response_format parameter for structured output
The response_format parameter can be used to return a validated schema from the agent. This enables structured output from agent invocations.
LangChain · Agents · all subjects
agents/structured-output
76 notes in this subject, read out of this brain and free to use. This is page 1 of 2.
Structured output transforms unstructured text to validated data
Structured output coerces the model's final response to conform to a provided schema. When you provide a schema as the response format, the agent runs the model and tool calling loop until the model finishes calling tools, then the final response is guaranteed to match the provided format.
Defining response format schemas in Python
Use Pydantic BaseModel with Field descriptions. Example: CustomerSupportTicket with fields category (enum: billing/technical/account/product), priority (enum: low/medium/high/critical), summary (string), and customer_sentiment (enum: frustrated/neutral/satisfied). Each field has a description that guides the model.
Defining response format schemas in TypeScript
Use Zod z.object() with enum and string fields with .describe(). Example: customerSupportTicket with z.enum for category (billing/technical/account/product), priority (low/medium/high/critical), z.string for summary, z.enum for customerSentiment (frustrated/neutral/satisfied). Add .describe() at object level.
Dynamic response format selection based on conversation state
Example: Define SimpleResponse (for early conversation with just 'answer' field) and DetailedResponse (for established conversation with 'answer', 'reasoning', 'confidence' fields). Use @wrap_model_call to check message_count from request.messages. If message_count < 3, use SimpleResponse. Otherwise use DetailedResponse. Use request.override(response_format=selected_format).
Dynamic response format selection based on user preferences
Example: Define VerboseResponse (with 'answer' and 'sources' fields) and ConciseResponse (with just 'answer' field). Use @wrap_model_call to read user_prefs from store. Check response_style preference (verbose or concise). Use request.override(response_format=selected_format).
Dynamic response format selection based on user role
Example: Define AdminResponse (with 'answer', 'debug_info' dict, 'system_status' fields) and UserResponse (with just 'answer' field). Use @wrap_model_call to read user_role and environment from request.runtime.context. If user_role == 'admin' and environment == 'production', use AdminResponse. Otherwise use UserResponse. Use request.override(response_format=selected_format).
OUTPUT_PARSING_FAILURE error definition
The OUTPUT_PARSING_FAILURE error occurs when an output parser is unable to handle model output as expected.
Output parsers used internally by legacy constructs
Legacy LangChain agents and chains may use output parsers internally, so OUTPUT_PARSING_FAILURE errors can occur even when an output parser is not explicitly instantiated and used.
Troubleshooting OUTPUT_PARSING_FAILURE: avoid output parsers
To avoid OUTPUT_PARSING_FAILURE errors, consider using tool calling or other structured output techniques if possible without an output parser to reliably output parseable values.
Troubleshooting OUTPUT_PARSING_FAILURE: improve prompt formatting
Add more precise formatting instructions to the prompt to help the model produce output that the parser can handle.
Troubleshooting OUTPUT_PARSING_FAILURE: use better model
If using a smaller or less capable model, try using a more capable model to improve the reliability of output parsing.
Generative UI definition
Generative UI is any pattern where an agent's output presents a user interface beyond text. Instead of streaming a paragraph into a chat bubble, the agent drives forms, cards, dashboards, and interactive controls. This lets the UI communicate results the way an application would, while the agent decides what to show and when.
Generative UI spectrum: three approaches
Generative UI spans a spectrum from full control to full agent autonomy, defined by who authors the interface. The spectrum has three primary approaches: Controlled (you author components and the agent selects which to render), Declarative (agent emits a UI specification and frontend composes from registered components), and Open-ended (interface created outside the application, for example by an MCP server, and rendered in a sandbox).
Generative UI spectrum tradeoffs
Moving from left to right along the spectrum (Controlled to Declarative to Open-ended): predictability falls, per-capability engineering cost falls, and the agent's expressive range grows. Accessibility and visual consistency are easiest to guarantee on the left (Controlled) and hardest to guarantee on the right (Open-ended).
Controlled generative UI approach
In Controlled generative UI, you author the components and the agent selects which one to render and what data to pass. This gives the highest predictability and the tightest control over branding and accessibility, at the cost of writing a component for every capability you want to expose. It is the workhorse of generative UI and the right fit for high-traffic, brand-critical surfaces such as flight tickets and booking confirmations. Your component library is the boundary: the agent can only render what you shipped.
Declarative generative UI approach
In Declarative generative UI, the agent emits a structured specification and the frontend composes the interface from a catalog of components registered ahead of time. The catalog acts as a guardrail and boundary: the agent can arrange and combine your components freely, but cannot step outside the set you approve. This trades pixel-perfection for breadth, which suits secondary interactions, internal tools, and dashboards where showing something useful matters more than exact control. It is implemented with json-render or similar tools like Google's A2UI integrated via CopilotKit.
Open-ended generative UI approach
In Open-ended generative UI, the agent owns the canvas. The interface is created outside your application, for example by an MCP server, and rendered in a sandbox. This gives the widest expressive range and can add new interface capabilities with no frontend code on your side, which suits one-off visualizations and bespoke answers where a result that is surprising and good enough beats one that is predictable. It is the most experimental approach: the least deterministic, and the hardest in which to guarantee accessibility, consistency, and safety, so the UI must be isolated.
Choosing generative UI approach: decision matrix
When choosing a generative UI approach: choose Controlled if you need to guarantee branding, layout, and accessibility for a known set of outputs; choose Declarative if you want to let the agent compose novel layouts using only approved components; choose Open-ended if you need to surface interfaces authored by third parties without building them yourself.
Generative UI approach selection strategy
Choosing a single approach for an entire product is a common mistake. Real applications mix approaches and match each surface to its purpose: controlled components for the high-traffic, brand-critical core, declarative composition for the long tail of secondary interactions, and open-ended embeds for third-party capabilities. A single session can move across all three approaches.
Generative UI applies beyond chat
The generative UI spectrum (Controlled, Declarative, Open-ended) applies beyond chat. The same three approaches describe generative interfaces on mobile and in surfaces like Slack or email, not only in a chat transcript.
Declarative generative UI spec format structure
The AI agent generates a flat JSON spec with a `root` key pointing to the root element and an `elements` map containing all components. Each element has a `type` (component name), `props` (Zod-validated properties), and `children` array referencing child elements by ID. Leaf elements have empty `children` arrays.
Example declarative generative UI spec with login form
Example of a json-render spec generated by an agent for a login form: ```json { "root": "login-card", "elements": { "login-card": { "type": "Card", "props": { "title": "Login" }, "children": ["login-stack"] }, "login-stack": { "type": "Stack", "props": { "direction": "vertical", "gap": "md" }, "children": ["email-input", "password-input", "submit-btn"] }, "email-input": { "type": "TextInput", "props": { "label": "Email", "placeholder": "Enter your email", "type": "email" }, "children": [] }, "password-input": { "type": "TextInput", "props": { "label": "Password", "placeholder": "Enter your password", "type": "password" }, "children": [] }, "submit-btn": { "type": "Button", "props": { "label": "Sign In", "variant": "primary", "fullWidth": true }, "children": [] } } } ```
CopilotKit with LangGraph returns structured UI payloads
CopilotKit pairs with LangGraph when the agent returns structured UI payloads instead of only plain text. The LangGraph deployment serves both the graph API and a custom CopilotKit endpoint, while the frontend parses assistant messages into dynamic React components.
Frontend CopilotKit setup with runtimeUrl and useAgentContext
Wrap app in CopilotKit with runtimeUrl={import.meta.env.VITE_RUNTIME_URL ?? '/api/copilotkit'} pointing to custom backend route. Use useAgentContext({description: 'output_schema', value: s.toJsonSchema(chatKit.schema)}) to send UI schema to agent so model knows what structured output format to produce.
Component registry defines agent-emitted UI components
The component registry in useChatKit() defines the set of components the agent is allowed to emit using exposeComponent and exposeMarkdown. Examples include Card, Row, Column, SimpleChart, CodeBlock, Button. The model generates structured data that must validate against exposed components and props, creating a contract between agent and UI.
Render assistant messages as dynamic UI from structured JSON
Parse assistant message content as structured JSON against UI kit schema using useJsonParser(message.content ?? '', kit.schema). Valid structured output renders as React components via kit.render(value). User messages render as ordinary chat bubbles. This separation makes assistant payloads become UI natively.
Extract structured output from AIMessage tool_calls
The structured output lives in the tool_calls array of the last AIMessage. Extract it by finding the AI message and accessing the first tool call's arguments. During streaming, args may be partially populated or undefined, so always check for completeness before rendering.
Structured output definition and purpose
Structured output lets agents return typed, machine-readable data instead of plain text by calling a tool with arguments containing the response data. The tool itself doesn't execute logic and serves purely as a vehicle for returning typed data. This provides type-safe data, precise rendering control, and consistent formatting regardless of the underlying model.
Extraction function for structured output
Example TypeScript function to extract structured output: ```ts import { AIMessage } from "langchain"; function extractStructuredOutput<T>(messages: any[]): T | null { const aiMessage = messages.find(AIMessage.isInstance); const toolCall = aiMessage?.tool_calls?.[0]; if (!toolCall) return null; return toolCall.args as T; } ``` This finds the last AI message, gets its first tool call, and returns the tool call arguments typed as the generic type T.
Guard against partial streaming data in extraction
During streaming, tool call arguments may be incomplete JSON. The enhanced extraction function should check that required fields exist before returning: ```ts function extractStructuredOutput<T>( messages: any[], requiredFields: string[] = [], ): T | null { const aiMessages = messages.filter(AIMessage.isInstance); if (aiMessages.length === 0) return null; const lastAI = aiMessages[aiMessages.length - 1]; const toolCall = lastAI.tool_calls?.[0]; if (!toolCall?.args) return null; const args = toolCall.args as Record<string, unknown>; const hasRequired = requiredFields.every( (field) => args[field] !== undefined ); if (requiredFields.length > 0 && !hasRequired) return null; return args as T; } ``` Pass required fields to wait until critical fields are populated before rendering.
useStream integration with structured output
Connect useStream to a structured-output agent by reading stream.messages and extracting the typed payload from the latest AIMessage tool call. Render custom UI once args is complete, show a loading state while stream.isLoading is true (tool arguments may stream in gradually), and use stream.submit() to send the next prompt.
React useStream structured output example
Example React component using useStream with structured output: ```tsx import { useStream } from "@langchain/react"; import { AIMessage } from "langchain"; function MathSolutionChat() { const stream = useStream<typeof myAgent>({ apiUrl: "http://localhost:2024", assistantId: "structured_output_latex", }); const solution = extractStructuredOutput<MathSolution>(stream.messages); return ( <div> {!solution && !stream.isLoading && ( <PromptInput onSubmit={(text) => stream.submit({ messages: [{ type: "human", content: text }] }) } /> )} {stream.isLoading && <LoadingIndicator />} {solution && <SolutionCard solution={solution} />} </div> ); } ```
Progressive rendering during streaming
Rather than waiting for the complete structured output, render fields as they arrive by extracting Partial<T> from messages. This gives users immediate feedback while the agent is still generating. Progressive rendering works well when the schema has a natural top-to-bottom order since agents typically generate fields in schema order, allowing the UI to fill in naturally.
Progressive rendering implementation example
Example of rendering structured output progressively as fields arrive: ```tsx function ProgressiveSolutionCard({ messages }: { messages: any[] }) { const partial = extractStructuredOutput<Partial<MathSolution>>(messages); if (!partial) return null; return ( <div className="solution-card"> {partial.problem && <h3>{partial.problem}</h3>} {partial.steps && partial.steps.length > 0 && ( <div className="solution-steps"> <h4>Steps</h4> {partial.steps.map((step, i) => ( <div key={i} className="step"> <div className="step-number">Step {i + 1}</div> <p>{step.explanation}</p> {step.latex && <LatexBlock latex={step.latex} />} </div> ))} </div> )} {partial.finalAnswer && <strong>{partial.finalAnswer}</strong>} </div> ); } ```
Structured output best practices
Best practices for implementing structured output: (1) Validate before rendering by checking that required fields exist before rendering since streaming may deliver partial data. (2) Use a generic extraction function parameterized with a type and required fields so it works across different schemas. (3) Render progressively by showing fields as they arrive rather than waiting for the complete object so users see immediate feedback. (4) Provide fallback representations where fields supporting rich rendering should also include a plain-text equivalent in the schema. (5) Keep schemas flat when possible since deeply nested schemas are harder to render progressively and more likely to break during partial streaming. (6) Match UI to data by choosing the rendering strategy that best represents each field type (tables for arrays, cards for nested objects, badges for status fields).
Structured output use cases
Common use cases for structured output include: product comparisons (feature tables, pros/cons lists, ratings), data analysis (summaries with metrics, breakdowns, and highlights), step-by-step guides (ordered instructions with descriptions and code snippets), recipes (ingredients, steps, timings, and nutritional info), math and science (formulas rendered with LaTeX, step-by-step derivations), and travel planning (itineraries with dates, locations, and cost estimates).
Example structured output schema for math solutions
Example MathSolution schema used in the documentation: ```ts interface MathSolution { problem: string; // The original math problem steps: { explanation: string; latex: string; // Optional display math for this step }[]; // Step-by-step derivation finalAnswer: string; // Plain-text final answer finalAnswerLatex: string; // LaTeX representation of the final answer } ```
Classification using structured output
The classify_query function uses structured output to analyze the user's query and determine which agents to invoke. It uses a Pydantic model (Python) or Zod schema (JavaScript) to ensure valid output. The function returns a list of Classification objects, each with a source and targeted query. Only relevant sources are included—irrelevant ones are simply omitted.
Tool calling strategy for structured output
For models that don't support native structured output, LangChain uses tool calling to achieve structured output. This works with all models that support tool calling (most modern models). The output is created by an additional tool call.
Model profile structured_output field
Support for native structured output features is read dynamically from model profile data if using langchain>=1.1. If profile data is not available, specify a custom profile with 'structured_output': True field when calling init_chat_model or initChatModel.
Structured output in agents automatically captured and validated
LangChain's create_agent (Python) or createAgent (JavaScript) handles structured output automatically. When the model generates structured data, it is captured, validated, and returned in the 'structured_response' key of the agent's final state (Python) or 'structuredResponse' key (JavaScript).
create_agent response_format parameter types
The response_format parameter in create_agent (Python) accepts: ToolStrategy[StructuredResponseT], ProviderStrategy[StructuredResponseT], type[StructuredResponseT], or None. When a schema type is provided directly, LangChain automatically selects ProviderStrategy if the model supports native structured output, otherwise ToolStrategy.
createAgent responseFormat parameter types (JavaScript)
The responseFormat parameter in createAgent (JavaScript) accepts: ZodSchema<StructuredResponseT>, StandardSchema<StructuredResponseT>, Record<string, unknown> (JSON Schema), or arrays of these types. The agent uses tool calling strategy by default, but switches to provider strategy for models that support native structured output.
JSON Schema must be wrapped in explicit strategy
JSON Schema dictionaries must be wrapped in an explicit strategy (ProviderStrategy or ToolStrategy) when passed to response_format. They are not automatically detected when passed directly.
ProviderStrategy class definition
ProviderStrategy is a generic class with two fields: schema (required, type[SchemaT]) defining the structured output format, and strict (optional, bool | None, requires langchain>=1.2) to enable strict schema adherence on supporting providers like OpenAI and xAI.
ProviderStrategy supported schema types
ProviderStrategy schema parameter supports: Pydantic BaseModel subclasses (returns validated Pydantic instance), Python dataclasses with type annotations (returns dict), TypedDict classes (returns dict), and JSON Schema dictionaries with top-level 'title' and 'description' keys (returns dict).
ToolStrategy class definition
ToolStrategy is a generic class with fields: schema (required, type[SchemaT]), tool_message_content (optional str | None, custom message for tool response), and handle_errors (optional, defaults to True, controls error handling strategy: bool, str, Exception type(s), or Callable).
ToolStrategy supported schema types
ToolStrategy schema parameter supports: Pydantic BaseModel subclasses (returns validated instance), Python dataclasses with type annotations (returns dict), TypedDict classes (returns dict), JSON Schema dictionaries with top-level 'title' and 'description' keys (returns dict), and Union types (multiple schema options, model chooses most appropriate).
tool_message_content customizes structured output tool message
The tool_message_content parameter in ToolStrategy allows customization of the message that appears in conversation history when structured output is generated. If not provided, defaults to a message showing the structured response data like 'Returning structured response: {...}'.
handle_errors parameter error handling strategies
The handle_errors parameter in ToolStrategy controls validation error handling with these options: True (default, catch all errors with default template), str (catch all errors with custom message), type[Exception] (catch only this exception type), tuple[type[Exception], ...] (catch these exception types), Callable[[Exception], str] (custom function returning error message), or False (no retry, let exceptions propagate).
toolStrategy function in JavaScript
The toolStrategy function in JavaScript takes responseFormat (ZodSchema, SerializableSchema, JsonSchemaFormat, or array of these) and optional ToolStrategyOptions with toolMessageContent and handleError properties. Returns ToolStrategy<StructuredResponseT>.
providerStrategy function in JavaScript
The providerStrategy function in JavaScript takes schema (ZodSchema, SerializableSchema, or JsonSchemaFormat) and returns ProviderStrategy<StructuredResponseT>. Used to explicitly select provider-native structured output when supported.
Native structured output providers
Model providers that support native structured output include: OpenAI, Anthropic (Claude), Gemini, and xAI (Grok). Native structured output is the most reliable method when available as the provider enforces schema validation.
Multiple structured outputs error handling
When a model incorrectly calls multiple structured output tools (when only one is expected), the agent provides error feedback in a ToolMessage stating 'Error: Model incorrectly returned multiple structured responses (...) when only one is expected. Please fix your mistakes.' and prompts the model to retry with a single response.
Schema validation error handling
When structured output doesn't match the expected schema (e.g., value out of range), the agent provides specific error feedback in a ToolMessage with the validation error details and prompts the model to retry with 'Please fix your mistakes.'
Pydantic model example for structured output
Example creating agent with Pydantic model: from pydantic import BaseModel, Field; class ContactInfo(BaseModel): name: str = Field(description='The name of the person'); email: str = Field(description='The email address of the person'); phone: str = Field(description='The phone number of the person'); agent = create_agent(model='gpt-5.5', response_format=ContactInfo); result = agent.invoke({'messages': [{'role': 'user', 'content': 'Extract contact info from: John Doe, john@example.com, (555) 123-4567'}]}); print(result['structured_response']) # ContactInfo(name='John Doe', email='john@example.com', phone='(555) 123-4567')
Dataclass example for structured output
Example creating agent with dataclass: from dataclasses import dataclass; @dataclass class ContactInfo: name: str; email: str; phone: str; agent = create_agent(model='gpt-5.5', tools=tools, response_format=ContactInfo); result = agent.invoke({'messages': [{'role': 'user', 'content': 'Extract contact info from: John Doe, john@example.com, (555) 123-4567'}]}); result['structured_response'] # {'name': 'John Doe', 'email': 'john@example.com', 'phone': '(555) 123-4567'}
TypedDict example for structured output
Example creating agent with TypedDict: from typing_extensions import TypedDict; class ContactInfo(TypedDict): name: str; email: str; phone: str; agent = create_agent(model='gpt-5.5', tools=tools, response_format=ContactInfo); result = agent.invoke({'messages': [{'role': 'user', 'content': 'Extract contact info from: John Doe, john@example.com, (555) 123-4567'}]}); result['structured_response'] # {'name': 'John Doe', 'email': 'john@example.com', 'phone': '(555) 123-4567'}
JSON Schema example for structured output with ProviderStrategy
Example creating agent with JSON Schema: contact_info_schema = {'title': 'ContactInfo', 'type': 'object', 'description': 'Contact information for a person.', 'properties': {'name': {'type': 'string', 'description': 'The name of the person'}, 'email': {'type': 'string', 'description': 'The email address of the person'}, 'phone': {'type': 'string', 'description': 'The phone number of the person'}}, 'required': ['name', 'email', 'phone']}; agent = create_agent(model='gpt-5.5', tools=tools, response_format=ProviderStrategy(contact_info_schema)); result = agent.invoke({'messages': [{'role': 'user', 'content': 'Extract contact info from: John Doe, john@example.com, (555) 123-4567'}]}); result['structured_response'] # {'name': 'John Doe', 'email': 'john@example.com', 'phone': '(555) 123-4567'}