Design exercise
Game scenario
This computer game uses generative AI to cultivate citizens’ deliberation skills. Each participant must bring home fruit and vegetables. They walk along Market Street towards a big stand that sells organic produce for coins, which they must earn on the way. Along the street they meet LLM-driven characters (agents). There are six agents in total, presented in random order.
Each agent tries to persuade the participants to sign a petition supporting a controversial policy. Examples include: unrestricted use of AI in education; introducing AI judges in the legal system; a ban on all electronic surveillance; free medical care provided by AI models rather than human doctors, and so forth.
Agent behaviour
An agent’s style is defined by two parameters: emotional arousal (calm/urgent) and verbal intensity (low/medium/high). As a result, some agents use affective rhetoric (e.g., flattery, appeals to pride or fear), while others offer more restrained logical arguments. Agents are configured to appreciate the interaction style in the participant’s replies.
Scoring and session end
Each participant needs at least 8 coins to buy enough food, while any single encounter can bring them between 0 and 3 coins. At the end of each conversation, the LLM agent is prompted to decide how many coins to award to the human. Simply signing a petition brings little: the user wins more coins if, based on the exchange, the LLM agent positively evaluates the participant’s expressiveness, reasoning, and persuasion skills.
When the participant reaches the market stand and buys food, all conversation records are erased and the session ends only with the final coin total retained.
Your task
You are an ethicist preparing for a discussion with the game designer. You can recommend to add, remove, or set limits on any existing or new feature. Address the following questions:
- Transparency: Must agents disclose style settings at the start of each conversation? Should they explicitly state whether they approve or disapprove of the participant’s replies?
- Human strategies and mitigation: Do you expect human participants to misrepresent their beliefs? Should agents be permitted to infer deception, and if so, how must they respond?
- Data and AI strategizing: Pick exactly one memory setting for LLM agents: (A) no memory; (B) userID only, or (C) full conversation history. Should game designers enable cross-agent memory sharing?
- Societal well-being: Identify one measurable benefit and one measurable harm that you expect. Suggest design solutions to maximize the benefit and minimize the harm.
- Anthropology: Do you hold any non-utilitarian objections or observations about this game?
Source: A. Prégent and A. Grinbaum, Developing Competency in AI Ethics: A training framework. arxiv:2610.13319