Structured output and grounding

A chat answer is meant for a person to read. The moment you want code to use the result, prose is a problem. Code cannot reliably pull a price or a date out of a paragraph. What it can parse is a fixed, predictable shape: JSON, a table, a fixed set of fields. Asking for that shape is called structured output.

Why a schema, not just "give me JSON"

Saying "respond in JSON" is a start, but it still leaves the model guessing at field names, types, and which fields are required. A schema pins that down: the exact keys, their types, and the shape of any nested data. With a schema, two runs of the same prompt produce output your code can parse the same way every time, instead of the model inventing slightly different field names each run.

Extract the product name, price in USD, and in-stock status
from this listing. Respond with ONLY this JSON shape, no other text:

{
  "name": <string>,
  "price_usd": <number>,
  "in_stock": <boolean>
}

Listing:
"Wireless Mouse M2 -- $24.99 -- 14 left in warehouse"

The expected output for that prompt is short and exact, nothing else on the page:

{
  "name": "Wireless Mouse M2",
  "price_usd": 24.99,
  "in_stock": true
}

What to specify

• The exact keys you want, spelled the way your code expects them.
• The type of each value: string, number, boolean, array, or a nested object.
• Whether a field is required or can be left out.
• "No other text" or "JSON only" so the model does not wrap the answer in an explanation.

This is not limited to JSON. A table with named columns, or a fixed list of labeled fields ("Name:", "Price:", "In stock:"), works the same way: you are trading the model's freedom to phrase things however it likes for a shape your code can depend on.

A schema tells the model what shape you want. It does not force the model to deliver it. AI Foundations covered hallucination: a model can be confidently wrong. The structured-output version of that same risk is a response that is malformed, missing a field, or holding the wrong type, delivered with no warning that anything is off. Treat every response as unverified until your code checks it.

What "malformed" looks like

• Broken JSON: a missing brace, a trailing comma, or an unescaped quote inside a string.
• A missing field your code expects to be there.
• The wrong type: a price returned as the string "24.99" instead of a number.
• Extra text around the data, like "Sure, here is the JSON:" before the actual object.
• A value that parses fine but is wrong: a price of -5, or a date that does not exist.

The first four are parsing problems, catchable by checking the shape. The last one is a content problem: the data parses cleanly but is not sensible. Both need a check before your code trusts the result.

A basic validate-retry-fail loop

1. 1Parse the response against your schema. Confirm every required field is present and has the right type.
2. 2If it fails, do not guess or patch it silently. Send it back with a short, specific correction: "That was not valid JSON, missing a closing brace. Return only the corrected JSON."
3. 3Retry a small, fixed number of times, such as 2 or 3. Do not retry forever.
4. 4If it still fails after your retry budget, fail gracefully: show the user a clear message, log the raw response for debugging, and stop, instead of feeding bad data further into your app.

That last step matters as much as the first. A silent failure that lets bad data flow downstream, an empty string treated as a valid name, a null price recorded as zero, is often worse than an error message, because nobody notices until it has already caused a mess somewhere else.

A model only knows two things: what it learned during training, frozen at its knowledge cutoff, and whatever text is sitting in the prompt right now. If you want it to answer questions about your own documents, a product catalog, internal notes, a support wiki, training data will not help, because the model has never seen that text. The fix is to hand it the text directly. That is called grounding.

The basic pattern

For a short document, grounding is simple: paste the whole thing into the prompt and ask the model to answer using only what you gave it. For a large collection, a full library of documents, you cannot paste all of it into one prompt; the context window will not hold it, and most of it is irrelevant to any one question anyway. So you narrow it down first: find the pieces that are actually relevant to the question, and only put those in the prompt.

This search-then-answer pattern has a name: retrieval-augmented generation, or RAG. "Retrieval" is the search step. "Augmented generation" means the model's answer is generated with that retrieved text added in. Whole systems exist to make the retrieval step fast and accurate over huge document sets, but the underlying idea is exactly the flow above: find it, add it, answer from it.

Instructing the model to stay in bounds

Retrieval only does half the job. You still have to tell the model to actually use the text you gave it, rather than filling gaps from its own training. A grounding instruction says so directly: "Answer using only the text below. If the answer is not in the text, say you do not know." That last clause matters: without it, a model will often reach for a plausible-sounding guess instead of admitting the provided text does not cover the question.