Essay

Designing agents people can understand

A blue diamond outline open at its upper right, with a gold square inside, a green half-circle to its left and three small green squares trailing off to the upper right

A shared framework and template could help teams create useful agent roles quickly, with their purpose, authority and limits clear to everyone involved.

Give an agent a name and a goal, and people will fill in the gaps differently. One person imagines an assistant that gathers options. Another expects it to choose among them. A third assumes it can contact colleagues and carry out the choice. The difference is authority, yet all three might describe the same thing as a “coordination agent.”

As teams create more agents, these gaps become consequential. An agent is part of a working relationship. It reads information that people supplied for particular purposes, helps shape decisions, and may act through tools that affect other people. Its design therefore needs to be understandable to those people, as well as to the people who build it.

We are exploring an agent design system to make that work repeatable. By system, we mean three connected things: a framework of questions that every agent role should answer, a template that records the answers, and a design process that develops the answers from a rough idea into an operating role. This is work in progress. Its value will depend on testing it with real agent ideas and revising it with the people who would use and be affected by those agents.

Begin with the work

The first question is what people are trying to accomplish. Describe how the work happens today, where it is difficult, and what a useful change would look like. Then ask how much of that change needs an agent.

This choice matters. Anthropic distinguishes workflows with predefined paths from agents that select their own steps. OpenAI identifies complex decisions, hard-to-maintain rules and unstructured information as promising areas for agents. Google’s People + AI Guidebook asks whether AI adds distinct value, and whether people want a task automated or their own ability augmented.

A design template can make that comparison routine. The answer might be a better human practice, a fixed workflow, a function added to an existing agent, or a new agent with room to choose its approach. A team can reach a useful design sooner when the options are visible at the start.

Make the role legible

We are considering five questions for an early agent sketch:

  1. Purpose: What change is the agent meant to help people achieve, and how would they recognise value?
  2. People: Who requests its work, collaborates with it, is accountable for it, and may be affected by its output without using it directly?
  3. Authority: What may it read, draft, recommend or do? Who authorises consequential actions, and who can correct or stop it?
  4. Information: Where does its context come from, how reliable is it, what may it remember, and where may its output travel?
  5. Evidence: What would show that it helps, what would show that it causes harm, and what should happen when it gets something wrong?

These questions turn a role name into a shared account of the work. They also reveal conflicts. The person who asks for an agent may differ from the person its output describes. A fast summary may help a manager while placing an unwanted burden on contributors whose work is summarised. NIST’s AI Risk Management Framework calls for attention to intended users, affected people, benefits, harms and human oversight. That wider view belongs in an agent’s design from the beginning.

Authority needs equally plain language. Consider an agent asked to prepare a meeting brief. It could read the shared agenda and project notes, draft a concise summary with links, and highlight unresolved questions. A human chair might decide what to send. If the notes disagree, the agent could flag the disagreement rather than inventing an agreement. The design changes materially if the agent can also invite participants, reveal private notes, or announce decisions.

OWASP’s Excessive Agency guidance shows why a system’s available functions, permissions and freedom to act must be considered separately. A clear role can distinguish reading from writing, drafting from sending, and suggesting from deciding. This gives builders concrete boundaries and gives people a way to understand what the agent will actually do.

Design for correction and learning

A useful template should ask for one ordinary case and one case where the agent is uncertain or wrong. When does it ask for clarification? When does it hand work back to a person? How can someone see where an answer came from, edit it, dismiss it or challenge an action?

Microsoft’s human–AI interaction guidelines include clear capability limits, easy correction and explanations of behavior. OpenAI’s agent guide recommends human intervention for sensitive actions and repeated failure. These are design questions as much as implementation checks: they shape the relationship between an agent and the people around it.

The same applies to success. Counting tasks completed can show activity, but it says little about whether people were better served. We would want to observe the quality of the result and the cost of getting it: whether people can correct mistakes, whether review work has shifted onto someone else, whether the agent has exposed information or weakened a human capability. Google warns about narrow measures and their downstream effects. Anthropic’s evaluation guidance also distinguishes an agent’s convincing report from the actual outcome in the environment where it acted.

Our proposed process would develop the design in stages. An early sketch captures purpose, people, possible actions and open questions. Research then compares possible forms and tests the uncertain parts with affected people. A concrete role card states permissions, decision rights, handoff points and how the agent will be evaluated. After it begins work, a named human reviews feedback, failures and changes to its remit.

The template makes each new role easier to start because the team has a shared set of questions. The framework helps people compare roles and understand how they relate. The process gives those answers room to mature as evidence arrives. Together, they could let us create agents rapidly while making each agent’s purpose, limits and contribution clear enough for people to work with it thoughtfully.

← All posts

Building AI that works with people, not around them?

Let’s Talk