agents/middleware

224 notes in this subject, read out of this brain and free to use. This is page 4 of 4.

wrap_model_call middleware for step-based configuration

The @wrap_model_call decorator creates middleware that reads current_step from state and applies the appropriate configuration. It: gets current_step (defaulting to initial step), looks up STEP_CONFIG, validates required state fields exist, formats prompt with state values, then calls request.override(system_prompt=..., tools=...) to apply the configuration.

State machine pattern for customer support

The state machine pattern implements workflows where an agent's behavior changes across different states. A single agent dynamically changes its configuration (system prompt and available tools) based on the current state, rather than creating multiple separate agents. This differs from the subagents pattern where sub-agents are called as tools.

Request.override for dynamic agent reconfiguration

The request.override() method (used in middleware) allows changing the system prompt and available tools before the model call. It returns a modified request without mutating the original. This is key to the state machine pattern: override is called once per turn based on current_step, not at initialization.

JavaScript implementation uses createMiddleware and Command from langgraph

In JavaScript, createMiddleware defines a middleware with name, stateSchema, and wrapModelCall function. The wrapModelCall receives request (with state property) and handler, and must return the result of handler() after modifications. Command is imported from '@langchain/langgraph', not langchain.

Step configuration dictionary structure

Steps are configured in a dictionary mapping step names to their configuration. Each step entry contains: prompt (string, can use format variables like {warranty_status}), tools (list of available tools), and requires (list of state fields that must be set before this step). This makes steps reusable and dependencies clear.

Stateless router implementation

A stateless router addresses each request independently with no memory between calls. For multi-turn conversations, use stateful routers instead.

Router architecture pattern

In the router architecture, a routing step classifies input and directs it to specialized agents. This is useful when you have distinct verticals (separate knowledge domains that each require their own agent). The router decomposes the query, zero or more specialized agents are invoked in parallel, and results are synthesized into a coherent response.

Router pattern use cases

Use the router pattern when you have distinct verticals (separate knowledge domains that each require their own agent), need to query multiple sources in parallel, and want to synthesize results into a combined response.

Single agent routing with Command

Use Command to route to a single specialized agent. Command takes a goto parameter with the name of the target agent node to transition to.

Parallel agent routing with Send

Use Send to fan out to multiple specialized agents in parallel. Send takes the agent node name and state to pass to that agent, allowing multiple agents to be invoked concurrently based on query classification.

Tool wrapper for router multi-turn conversation Python

```python @tool def search_docs(query: str) -> str: """Search across multiple documentation sources.""" result = workflow.invoke({"query": query}) return result["final_answer"] # Conversational agent uses the router as a tool conversational_agent = create_agent( model, tools=[search_docs], prompt="You are a helpful assistant. Use search_docs to answer questions." ) ``` This example shows how to wrap a stateless router as a tool for a conversational agent to handle multi-turn conversations.

Tool wrapper for router multi-turn conversation TypeScript

```typescript const searchDocs = tool( async ({ query }) => { const result = await workflow.invoke({ query }); return result.finalAnswer; }, { name: "search_docs", description: "Search across multiple documentation sources", schema: z.object({ query: z.string().describe("The search query"), }), } ); // Conversational agent uses the router as a tool const conversationalAgent = createAgent({ model, tools: [searchDocs], systemPrompt: "You are a helpful assistant. Use search_docs to answer questions.", }); ``` This example shows how to wrap a stateless router as a tool for a conversational agent to handle multi-turn conversations in TypeScript.

Stateful router with tool wrapper

The simplest approach for multi-turn conversations: wrap the stateless router as a tool that a conversational agent can call. The conversational agent handles memory and context; the router stays stateless. This avoids the complexity of managing conversation history across multiple parallel agents.

Router vs Subagents pattern distinction

Router: A dedicated routing step (often a single LLM call or rule-based logic) that classifies the input and dispatches to agents. The router itself typically doesn't maintain conversation history or perform multi-turn orchestration—it's a preprocessing step. Subagents: A main supervisor agent dynamically decides which subagents to call as part of an ongoing conversation. The main agent maintains context, can call multiple subagents across turns, and orchestrates complex multi-step workflows. Use a router when you have clear input categories and want deterministic or lightweight classification. Use a supervisor when you need flexible, conversation-aware orchestration where the LLM decides what to do next based on evolving context.

Stateful router conversation experience pitfall

Stateful routers require custom history management. If the router switches between agents across turns, conversations may not feel fluid to end users when agents have different tones or prompts. With parallel invocation, you'll need to maintain history at the router level (inputs and synthesized outputs) and leverage this history in routing logic. Consider the handoffs pattern or subagents pattern instead—both provide clearer semantics for multi-turn conversations.

Single agent routing example Python

```python from langgraph.types import Command def classify_query(query: str) -> str: """Use LLM to classify query and determine the appropriate agent.""" # Classification logic here ... def route_query(state: State) -> Command: """Route to the appropriate agent based on query classification.""" active_agent = classify_query(state["query"]) # Route to the selected agent return Command(goto=active_agent) ``` This example shows how to route to a single specialized agent using Command with a goto parameter.

Single agent routing example TypeScript

```typescript import { z } from "zod"; import { Command } from "@langchain/langgraph"; const ClassificationResult = z.object({ query: z.string(), agent: z.string(), }); function classifyQuery(query: string): z.infer<typeof ClassificationResult> { // Use LLM to classify query and determine the appropriate agent // Classification logic here ... } function routeQuery(state: z.infer<typeof ClassificationResult>) { const classification = classifyQuery(state.query); // Route to the selected agent return new Command({ goto: classification.agent }); } ``` This example shows how to route to a single specialized agent using Command with a goto parameter in TypeScript.

Multiple agents parallel routing example Python

```python from typing import TypedDict from langgraph.types import Send class ClassificationResult(TypedDict): query: str agent: str def classify_query(query: str) -> list[ClassificationResult]: """Use LLM to classify query and determine which agents to invoke.""" # Classification logic here ... def route_query(state: State): """Route to relevant agents based on query classification.""" classifications = classify_query(state["query"]) # Fan out to selected agents in parallel return [ Send(c["agent"], {"query": c["query"]}) for c in classifications ] ``` This example shows how to fan out to multiple specialized agents in parallel using Send.

Multiple agents parallel routing example TypeScript

```typescript import { z } from "zod"; import { Command } from "@langchain/langgraph"; const ClassificationResult = z.object({ query: z.string(), agent: z.string(), }); function classifyQuery(query: string): z.infer<typeof ClassificationResult>[] { // Use LLM to classify query and determine the appropriate agent // Classification logic here ... } function routeQuery(state: typeof State.State) { const classifications = classifyQuery(state.query); // Fan out to selected agents in parallel return classifications.map( (c) => new Send(c.agent, { query: c.query }) ); } ``` This example shows how to fan out to multiple specialized agents in parallel using Send in TypeScript.

Skill middleware architecture

Skill middleware is custom middleware that injects skill descriptions into the system prompt, making skills discoverable without loading full content upfront. The middleware (1) builds a skills prompt listing all available skills with their descriptions in the format '- **skill_name**: description', (2) creates a skills addendum instructing the agent to use the load_skill tool when needing detailed information, (3) registers the load_skill tool as a class variable to make it available to the agent, (4) appends the skills addendum to the system message in the wrap_model_call method. In production, skill loading should occur in the before_agent hook to allow periodic refreshes when new skills are added or existing ones are modified.

Middleware in create_agent harness

The create_agent harness includes middleware that shapes agent behavior. Middleware capabilities include guardrails, retries, routing, and custom tool policies.

Python auth gate middleware with execution and server info example

Example showing how to use execution_info and server_info in Python middleware: from langchain.agents import AgentState from langchain.agents.middleware import before_model from langgraph.runtime import Runtime @before_model def auth_gate(state: AgentState, runtime: Runtime) -> dict | None: """Block unauthenticated users when running on LangGraph Server.""" server = runtime.server_info if server is not None and server.user is None: raise ValueError("Authentication required") print(f"Thread: {runtime.execution_info.thread_id}") return None This demonstrates checking for authenticated users via runtime.server_info.user and accessing runtime.execution_info.thread_id. Requires deepagents>=0.5.0 (or langgraph>=1.1.5).

Runtime object components and structure

LangChain's create_agent runs on LangGraph's runtime under the hood. The Runtime object exposes five main components: Context (static information like user id, db connections, or other dependencies for an agent invocation), Store (a BaseStore instance used for long-term memory), Stream writer (an object used for streaming information via the 'custom' stream mode), Execution info (identity and retry information for the current execution including thread ID, run ID, attempt number), and Server info (server-specific metadata when running on LangGraph Server including assistant ID, graph ID, authenticated user).

Access runtime in middleware

You can access runtime information in middleware to create dynamic prompts, modify messages, or control agent behavior based on user context. In Python node-style hooks, use the Runtime parameter to access the Runtime object. For Python wrap-style hooks, the Runtime object is available inside the ModelRequest parameter. In JavaScript, use the runtime parameter to access the Runtime object inside middleware.

Python dynamic prompt middleware with runtime example

Example showing how to use runtime in Python middleware with dynamic_prompt decorator: from dataclasses import dataclass from langchain.messages import AnyMessage from langchain.agents import create_agent, AgentState from langchain.agents.middleware import dynamic_prompt, ModelRequest @dataclass class Context: user_name: str @dynamic_prompt def dynamic_system_prompt(request: ModelRequest) -> str: user_name = request.runtime.context.user_name system_prompt = f"You are a helpful assistant. Address the user as {user_name}." return system_prompt agent = create_agent( model="gpt-5-nano", tools=[...], middleware=[dynamic_system_prompt], context_schema=Context ) agent.invoke( {"messages": [{"role": "user", "content": "What's my name?"}]}, context=Context(user_name="John Smith") ) This demonstrates accessing runtime.context inside a dynamic_prompt middleware.

Python before_model and after_model middleware with runtime example

Example showing how to use runtime in Python middleware with before_model and after_model decorators: from dataclasses import dataclass from langchain.agents import create_agent, AgentState from langchain.agents.middleware import before_model, after_model from langgraph.runtime import Runtime @dataclass class Context: user_name: str @before_model def log_before_model(state: AgentState, runtime: Runtime[Context]) -> dict | None: print(f"Processing request for user: {runtime.context.user_name}") return None @after_model def log_after_model(state: AgentState, runtime: Runtime[Context]) -> dict | None: print(f"Completed request for user: {runtime.context.user_name}") return None agent = create_agent( model="gpt-5-nano", tools=[...], middleware=[log_before_model, log_after_model], context_schema=Context ) agent.invoke( {"messages": [{"role": "user", "content": "What's my name?"}]}, context=Context(user_name="John Smith") ) This demonstrates accessing runtime.context.user_name in before_model and after_model hooks.

JavaScript middleware with runtime example

Example showing how to use runtime in JavaScript middleware: import * as z from "zod"; import { createAgent, createMiddleware, SystemMessage } from "langchain"; const contextSchema = z.object({ userName: z.string(), }); const dynamicPromptMiddleware = createMiddleware({ name: "DynamicPrompt", contextSchema, beforeModel: (state, runtime) => { const userName = runtime.context?.userName; if (!userName) { throw new Error("userName is required"); } const systemMsg = `You are a helpful assistant. Address the user as ${userName}.`; return { messages: [new SystemMessage(systemMsg), ...state.messages], }; }, }); const loggingMiddleware = createMiddleware({ name: "Logging", contextSchema, beforeModel: (state, runtime) => { console.log(`Processing request for user: ${runtime.context?.userName}`); return; }, afterModel: (state, runtime) => { console.log(`Completed request for user: ${runtime.context?.userName}`); return; }, }); const agent = createAgent({ model: "gpt-5.5", tools: [...], middleware: [dynamicPromptMiddleware, loggingMiddleware], contextSchema, }); const result = await agent.invoke( { messages: [{ role: "user", content: "What's my name?" }] }, { context: { userName: "John Smith" } } ); This demonstrates accessing runtime.context in beforeModel and afterModel hooks in JavaScript.

Access execution info and server info in middleware

Middleware hooks can access runtime.execution_info and runtime.server_info (Python) or runtime.executionInfo and runtime.serverInfo (JavaScript). This allows middleware to use thread ID, run ID, assistant ID, and authenticated user information from the runtime.

JavaScript auth gate middleware with execution and server info example

Example showing how to use executionInfo and serverInfo in JavaScript middleware: import { createMiddleware } from "langchain"; const authGate = createMiddleware({ name: "AuthGate", beforeModel: (state, runtime) => { const server = runtime.serverInfo; if (server != null && server.user == null) { throw new Error("Authentication required"); } console.log(`Thread: ${runtime.executionInfo.threadId}`); return; }, }); This demonstrates checking for authenticated users via runtime.serverInfo.user and accessing runtime.executionInfo.threadId. Requires deepagents>=1.9.0 (or @langchain/langgraph>=1.2.8).

Before_model middleware execution point

The `@before_model` middleware decorator processes messages before the model is called. In the agent execution flow, before_model runs after __start__ and before the model, allowing preprocessing of state and messages. It can return None (no changes) or a dict with updates.

After_model middleware execution point

The `@after_model` middleware decorator processes messages after the model is called. In the agent execution flow, after_model runs after the model and before tools/end, allowing postprocessing of model output. It can return None (no changes) or a dict with updates.

After_model validate response example

Example using `@after_model` to validate and filter responses: ```python @after_model def validate_response(state: AgentState, runtime: Runtime) -> dict | None: STOP_WORDS = ["password", "secret"] last_message = state["messages"][-1] if any(word in last_message.content for word in STOP_WORDS): return {"messages": [RemoveMessage(id=last_message.id)]} return None ```

SQL agent human-in-the-loop middleware

LangChain agents support built-in human-in-the-loop middleware to add oversight to agent tool calls. For SQL agents, this can be configured to pause for human review before executing the sql_db_query (or execute_sql in JavaScript) tool. A checkpointer must be added to the agent to allow execution to be paused and resumed.

Resuming SQL agent execution with Command

After human-in-the-loop middleware pauses execution pending approval, execution can be resumed using the Command function, allowing the human to accept or reject the tool call (such as a SQL query).

Tool error handling with middleware

Handle tool errors using LangChain agent middleware to retry failed tool calls or return custom error messages.

Human-in-the-loop middleware configuration for supervisors

Human-in-the-loop review can be added to supervisor systems using HumanInTheLoopMiddleware on sub-agents with interrupt_on parameter to specify which tools should interrupt (e.g., interrupt_on={'create_calendar_event': True}). A checkpointer (such as InMemorySaver) must be added only to the top-level supervisor agent to pause and resume execution. Sub-agents with middleware can use all response types: approve, edit, reject.

Interrupts in supervisor pattern with human review

When human-in-the-loop review is enabled, interrupted execution stops and the supervisor gathers interrupt events. Each interrupt has an ID and contains action_requests with tool information and arguments. To resume, pass a Command with resume parameter containing a dictionary mapping interrupt IDs to decisions (type: 'approve', 'edit', or 'reject'), and optionally edited_action for edit decisions.

Dynamic model selection with wrap_model_call decorator

To use dynamic model selection, create middleware using the @wrap_model_call decorator that modifies the model in the request. The decorator receives a ModelRequest and handler. Inside the function, you can access request.state to make decisions about which model to use, then call handler(request.override(model=new_model)) to use that model.

Example: Dynamic model selection middleware in Python

```python from langchain_openai import ChatOpenAI from langchain.agents import create_agent from langchain.agents.middleware import wrap_model_call, ModelRequest, ModelResponse basic_model = ChatOpenAI(model="gpt-5.4-mini") advanced_model = ChatOpenAI(model="gpt-5.5") @wrap_model_call def dynamic_model_selection(request: ModelRequest, handler) -> ModelResponse: """Choose model based on conversation complexity.""" message_count = len(request.state["messages"]) if message_count > 10: model = advanced_model else: model = basic_model return handler(request.override(model=model)) agent = create_agent( model=basic_model, tools=tools, middleware=[dynamic_model_selection] ) ```

Dynamic model selection with createMiddleware in TypeScript

In TypeScript, create middleware with createMiddleware that has a wrapModelCall property to modify the model in the request. Example: const dynamicModelSelection = createMiddleware({name: 'DynamicModelSelection', wrapModelCall: (request, handler) => {const messageCount = request.messages.length; return handler({...request, model: messageCount > 10 ? advancedModel : basicModel});}});

Pre-bound models restriction with dynamic model selection

Pre-bound models (models with bind_tools already called) are not supported when using structured output with dynamic model selection. If you need dynamic model selection with structured output, ensure the models passed to the middleware are not pre-bound.

Permissions parameter in agent definition

Pass filesystem permission rules in `permissions` to control which paths the agent's built-in filesystem tools can read or write.

Human-in-the-loop parameter in agent definition

Set `interrupt_on` (Python) or `interruptOn` (JavaScript) to pause before selected tool calls. Use this for actions that require a person to approve, edit, or reject the call before it runs.

Middleware parameter in agent definition

Pass middleware in the `middleware` list (Python) or `middleware` array (JavaScript) to add behavior around model calls, tool calls, and the agent lifecycle. Middleware runs in the order specified (list order in Python, array order in JavaScript).

Give your agent this brain