85 agentic tools audited for security incidents · Talk to the lab →
MotionticBook a call
Glossary

What are structured outputs?

Structured outputs is an API feature that constrains a language model's response to a supplied JSON schema through constrained decoding, so the output follows the schema by construction rather than by instruction.

Asking a model for JSON in the prompt works most of the time, which is not good enough for code that parses the result thousands of times a day. Structured outputs move the guarantee from the prompt into decoding: the schema is compiled into a grammar, and at each step the model can only produce tokens that keep the output valid.

OpenAI introduced the feature on 6 August 2024. On its own evals of complex JSON schema following, gpt-4o-2024-08-06 with structured outputs scored 100%, against under 40% for gpt-4-0613, and OpenAI noted that the earlier JSON mode produced valid JSON without guaranteeing it matched a schema. Anthropic's version, first released in beta in November 2025, is now generally available on current Claude models.

Two modes

  • JSON output

    The response body itself is JSON matching the schema. Used for extraction, classification and any result that code consumes directly.

  • Strict tool use

    Tool names and arguments are validated against each tool's schema, so an agent's tool calls always parse. The two modes can be combined in one request.

Limits worth knowing

  • Not every schema feature

    Anthropic's docs list unsupported features, including recursive schemas, numeric minimum and maximum, and string length limits. Those checks still have to happen in code.

  • Complexity caps

    Anthropic limits a request to 20 strict tools, 24 optional parameters and 16 union-type parameters across strict schemas.

  • First-call latency

    The grammar is compiled on first use and cached for 24 hours, so the first request with a new schema is slower.

  • Edge cases

    A refusal or a response cut off at max_tokens can still break the schema, so the stop reason has to be checked.

Valid is not correct

A schema guarantees shape, not truth. A field typed as a string will contain a string, which may be an invented customer ID. Business rules, cross-field checks and lookups against real records still need validation in code, and the values need evals.

Structured outputs also replaced older workarounds. Anthropic stopped supporting prefilled assistant responses from Claude 4.6 onward and points to structured outputs as the replacement for prefills used to force JSON.

How Motiontic's engineers apply it

Motiontic's engineers define schemas in the application's own types, such as Pydantic or Zod, use strict mode on every tool an agent can call, and validate values against real records before anything is written. Eval sets score field-level accuracy, not only whether the JSON parsed.

Ways to get JSON from a model

Ways to get JSON from a model
Asking in the promptJSON modeStructured outputs
Valid JSONUsuallyYesYes
Matches the schemaUsuallyNot guaranteedYes, by constrained decoding
Correct valuesNot guaranteedNot guaranteedNot guaranteed
Extra costRetries on parse failuresRetries on schema failuresSlower first call per schema

Common questions

What are structured outputs in an LLM API?

A feature that makes the model's response match a JSON schema supplied with the request, enforced during decoding rather than requested in the prompt.

What is the difference between JSON mode and structured outputs?

JSON mode makes the output valid JSON. Structured outputs make it match a specific schema, with the required fields and types. OpenAI introduced structured outputs because JSON mode did not guarantee schema conformance.

Do structured outputs stop hallucinations?

No. They guarantee the shape of the output, not the truth of the values. A well-formed field can still hold an invented name or number, so values need validation and evals.

Should agents use strict tool schemas?

In most production agents, yes. Strict schemas remove a whole class of failed tool calls caused by malformed arguments, leaving the harder problem of choosing the right values.

Running agents that have to hold up?

Describe the agent, the systems it touches and what it must never get wrong. The first call is with a lab engineer, who will say plainly whether it is ready for production, and whether Motiontic is the right team.