What an AI agent actually is

There's a useful, specific definition underneath the hype: an agent is a model, wired to tools and memory, running inside a loop that lets it plan a step, act on it, observe what actually happened, and repeat, working toward a goal instead of replying once. Take away any one of those pieces and you have something else. No loop, and it's a chatbot. No tools, and it's a chatbot that can only talk about acting, not do it. No memory management, and it falls apart past a handful of steps. This guide walks through what actually makes something an agent, the reasoning pattern behind the loop, where agents earn their keep and where they don't, and why permissions matter the moment a model can take real actions instead of just describing them.

Chat, assistant, agent: a ladder of autonomy

Chat, assistant, and agent get used interchangeably in marketing copy, but they describe genuinely different amounts of autonomy: how much a system decides and does on its own before it hands control back to a person. A chatbot replies to what you type, one turn at a time, and stops there. An assistant works inside a specific task or app, with some surrounding context, but still hands back one answer per turn. An agent is different in kind, not just in degree: it can decide to take an action, check what actually happened, and decide what to do next, without you typing a new prompt for each step.

The jump that matters is the last one. A very good single reply is still just a reply: it doesn't check whether it was right, and it can't correct itself if it wasn't. An agent isn't a smarter model doing the same thing better. It's the same kind of model, wired to a loop and a set of tools, so it can work a goal across multiple steps instead of answering once and stopping.

The core loop: plan, act, observe, repeat

Under whatever name a product gives it, almost every agent runs the same four-stage loop. Learning the stages by name is worth it, because they explain both what an agent is doing at any given moment and why it sometimes takes several tool calls and real wall-clock time to finish something a person could describe in one sentence.

• Plan: given the goal and everything gathered so far, the model decides its next move, not the whole solution, just the next concrete step.
• Act: it calls a tool to actually carry out that move, running a command, editing a file, querying an API, searching the web.
• Observe: it reads the real result, an error message, a file's new contents, a search hit, not the outcome it assumed it would get.
• Repeat: that result becomes part of the next plan, and the loop runs again until the goal is met or the agent stops.

Reason then act: the pattern behind the loop

The "plan" step isn't new or mysterious. It comes from a specific, well-tested pattern called ReAct, short for reasoning and acting, described by Shunyu Yao and coauthors in a 2022 paper.¹ Before ReAct, reasoning (a model thinking through a problem in text, sometimes called chain-of-thought) and acting (a model generating a plan of actions) were mostly studied as separate skills. ReAct interleaves them instead: the model writes a short piece of reasoning, takes an action based on it, reads the result, then writes the next piece of reasoning informed by what it just learned. The reasoning half helps the model track its own plan and catch exceptions; the acting half lets it pull in real information instead of guessing at it.¹

The effect is measurable, not just plausible. On question answering (HotpotQA) and fact verification (Fever), letting the model check a simple Wikipedia API mid-reasoning cut down on the hallucination and compounding errors that plain chain-of-thought reasoning is prone to, since the model no longer had to hold every fact in its head, it could go check. On two interactive decision-making benchmarks, ALFWorld and WebShop, ReAct beat imitation-learning and reinforcement-learning baselines by an absolute success rate of 34 and 10 percentage points respectively, prompted with only one or two examples.¹ That's the empirical case for the whole idea: a model that checks its work mid-task does measurably better than one that only talks or only acts.

Memory: an agent's limited working memory

A single chat reply barely touches the context window, the fixed amount of text a model can hold in view at once. A multi-step agent burns through it fast, because every plan, tool call, and observation from every pass through the loop has to fit inside that same window, alongside the original goal. A twenty-step task can generate a long transcript, and once the window is full, the earliest parts drop out of view and stop influencing what the model does next.

Two common fixes: summarizing, which compresses older steps into a shorter recap and trades detail for space, and external memory, which writes key facts to a file or store outside the window and reads them back only when needed. Neither makes the window bigger; both are ways of budgeting a fixed resource. It's also why handing an agent one enormous goal tends to work worse than breaking it into smaller ones. A smaller goal produces a shorter loop, which leaves more of the window free for the parts of the task that actually matter.

Where agents genuinely help

Agents earn their keep on tasks that are genuinely multi-step, where the right next move depends on what the last one turned up: fixing a failing test suite, working through a multi-file refactor, researching a topic across several sources and reconciling what they say, or moving a request through a workflow that touches several systems in sequence. What these have in common is that no single prompt could specify the whole solution up front, because part of the information needed only shows up once the agent starts acting and reading results.

Where agents fail, or just add complexity

None of this makes an agent the default right answer. Anthropic's own guidance on building agents is direct about this: add the complexity of an agentic loop only when it demonstrably improves the outcome, because agentic systems trade latency and cost for better performance on harder tasks.² Every pass through the loop is another full model call, and because each call re-sends the conversation so far, a five-step task runs up more than five times the cost of a single reply, and takes proportionally longer. For a large share of real tasks, one well-built call with good context and a couple of examples does the job just as well, more cheaply and more predictably.²

Agents also fail in specific, recognizable ways. A wrong observation early in the loop can compound, since every later step reasons from it. A loop can spin, retrying a broken approach several times before giving up, or not giving up at all. And because each step carries some model randomness, the same task can take a different number of steps, or a different path, on two separate runs. None of that makes agents unreliable in principle. It means they need the same skepticism as any other model output, applied at every step instead of just once.

Permissions and human review: why they matter

A chat reply that's wrong costs you a moment of confusion. An agent that's wrong can delete a file, send a message, or spend money before anyone notices, because it isn't just producing text, it's calling tools that act on real systems. Anthropic frames the fix directly: an agent should get real feedback from its environment at each step, and it should be able to pause for human input at checkpoints or when it hits a blocker, running inside a sandboxed environment with guardrails matched to what it's allowed to touch.²

• Ask before acting on anything in a defined, listed category, rather than after the fact.
• Read-only by default: let the agent look and propose, and require a separate step before anything actually changes.
• Scoped tools: give it access to only what the task needs, not every system it could theoretically reach.
• A human review step before anything touching data, money, or another person goes live, since those are the actions that are hardest, or impossible, to undo.

Honest limits

An agent doesn't fix a badly scoped task, it just executes the bad scope faster and in more steps. It doesn't make a model more accurate at any single step either; it gives the model more chances to catch its own mistakes, which helps but never guarantees correctness. And more autonomy isn't automatically better: a system that can act on its own is also a system that can act on its own mistake, which is the entire reason the last section exists. The honest way to think about an agent isn't "smarter assistant." It's a model, some tools, some memory, and a loop, plan, act, observe, repeat, each part able to fail in its own ordinary way, wired together to work toward a goal instead of stopping after one answer.