Design a multi-step AI workflow

A workflow is a task broken into steps whose order was decided by the code you wrote, before the task ever ran. That is what separates it from an agent, where the model decides its own next step as it goes.¹ This guide is about designing the workflow side: taking a multi-step task and turning it into a pipeline you can build, test, and trust one stage at a time.

The example this guide follows

One task runs through every step below: turn a folder of raw customer reviews into a short weekly digest email, with a category and sentiment per review and a couple of real quotes for each category. It is ordinary business automation, not a research problem, which is exactly the kind of task a workflow is built for.

1. Break the task into discrete stages

Before picking a tool or writing a prompt, write the stages down as plain, one-verb descriptions: clean, classify, validate, extract, verify, draft. Each stage should do one job and produce one clear output. If describing a stage needs "and," classify and summarize, it is probably two stages wearing one name.

Stage boundaries matter because they are where you will add contracts, gates, and retries in the steps below. A task described as one giant stage, "read the reviews and write the digest," gives you nowhere to check the work until the very end, when a mistake is hardest to trace back to where it started.

2. Choose the unit for each stage

Not every stage needs a model call, and not every model call needs to be a free-form prompt. Four units cover almost everything:

In the running example, cleaning is deterministic code (no model needed to strip HTML), classifying and drafting are prompts (judgment calls with one clear output), and both validating steps are deterministic code again (a schema check does not need a model). None of the six stages need a sub-agent, because every one of them has a known shape before it runs.

3. Chain the stages: define the contract

A workflow is only as reliable as the handoffs between its stages. Decide, before you write a single prompt, exactly what each stage receives and what it must return, down to the field names. A structured contract, not a paragraph of prose, is what lets you check a handoff mechanically in the next step.

# stage 1 output -> stage 2 input
{
  "reviews": [
    { "id": "r_1042", "text": "Billing charged me twice this month." }
  ]
}

# stage 2 output -> stage 3 input
{
  "reviews": [
    { "id": "r_1042", "category": "billing", "sentiment": "negative" }
  ]
}

Ask the model for exactly this shape, structured data with named fields, not "a short paragraph about each review." A free-form answer is harder to validate, and harder for the next stage to consume without another model call just to parse it.

4. Add checks and gates between stages

A contract only helps if something enforces it. Anthropic's own guidance on this pattern is direct: you can add "programmatic checks (see 'gate' in the diagram below) on any intermediate steps to ensure that the process is still on track."¹ A gate sits between two stages and decides whether the pipeline continues, retries, or stops.

• A validation gate: deterministic code checking the contract, correct field names, allowed values, no empty required fields. This is what catches a classify stage returning "irate" when the schema only allows "negative."
• An evaluator gate: a second, narrower model call that scores a draft against a rubric and sends it back if it fails, the pattern behind "draft, then check the draft."
• A human gate: a person reviews the output before the pipeline is allowed to do anything irreversible, sending an email, in the running example.

In the running example, the verify-quotes gate is a validation gate: it confirms the quoted line the model pulled out is an exact substring of the original review. That one cheap, deterministic check is what stops an invented quote from reaching the email.

5. Handle failure and retries

Stages fail two different ways, and they need two different responses. A transient failure (a timeout, a rate limit, a dropped connection) is not the stage's fault; retry it, with a short delay that grows on each attempt, and give up after a small fixed number of tries. A structural failure (the output fails its gate twice in a row) will not be fixed by an identical third attempt; stop, and either fall back to a simpler path or hand it to a human.

def run_stage(fn, input, max_retries=2):
    for attempt in range(max_retries + 1):
        try:
            return fn(input)
        except TransientError:
            wait(backoff(attempt))
    raise StageFailed(fn.__name__)

result = run_stage(classify, reviews)
ok, reason = validate(result)
if not ok:
    result = run_stage(classify, reviews, hint=reason)  # one retry, with the failure named
    ok, reason = validate(result)
if not ok:
    escalate_to_human(reviews, reason)

Two other habits pay for themselves quickly: make each stage idempotent, safe to run twice, so a retry after a partial failure does not send the digest email a second time; and log every attempt, input, output, and gate result, so a failure three stages downstream can be traced back to the stage that actually caused it.

6. Decide: workflow, or a single agent

Everything above assumes the stages, and their order, are known before the pipeline runs. That is what makes it a workflow: "systems where LLMs and tools are orchestrated through predefined code paths," as opposed to an agent, "systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks."¹ Anthropic's framing of which to prefer by default is direct: workflows "offer predictability and consistency for well-defined tasks."¹ Most tasks that feel like they need an agent turn out to be workflows once you break them into stages the way step 1 does.

Five recurring shapes cover almost every workflow you will design. The running example already uses two of them:

• Prompt chaining: "decomposes a task into a sequence of steps, where each LLM call processes the output of the previous one,"¹ gated at each handoff. This is the whole shape of the running example: classify, then extract, then draft, each stage consuming the last one's output.
• Routing: "classifies an input and directs it to a specialized followup task,"¹ useful the moment your classify stage needs to send different categories down different paths, a billing complaint drafted differently than a feature request.
• Parallelization: several calls run at once, either splitting independent work or having a few calls vote on the same input, a fit if you are classifying a thousand reviews and do not need them done in order.
• Orchestrator-workers: one call looks at the input and decides, at runtime, how to split it into subtasks, then hands each to a worker. Reach for this only when you genuinely cannot enumerate the split in advance, unlike routing, where the categories are already known.
• Evaluator-optimizer: one call drafts, a second call checks it against a rubric, and the two go back and forth until the draft passes, with a round cap so it cannot loop forever. This is the evaluator gate from step 4, made into its own pattern.

If, once you have named your stages, one of them still cannot be written down in advance, the number of sub-issues in a bug report is unknown until you start reading them, that single stage is the one to build as a bounded sub-agent, not a reason to rebuild the whole pipeline as one. An agent earns its cost by handling exactly that kind of open-ended stage, and it can still "pause for human feedback at checkpoints or when encountering blockers"¹ the same way a workflow gate does. Wrap it in guardrails and drop it into the fixed pipeline around it.

For the vocabulary behind workflow and agent as a question of control, not capability, see Agent vs script vs chat: when to use which. For wiring a single tool into a working loop, Build your first agent covers the smallest agent that works. For the pieces an agent is built from, models, tools, memory, and the loop, see What an AI agent actually is.