agents/middleware

224 notes in this subject, read out of this brain and free to use. This is page 1 of 4.

Middleware is the primitive for agent customization

create_agent is highly extensible through middleware. Each middleware piece handles one concern, hooks into the agent loop at the right moment, and composes freely with any other. Common patterns are prebuilt as first-class middleware, and anything else can be built as custom middleware.

SkillsMiddleware configuration

SkillsMiddleware allows agents to access domain knowledge stored in skill files. With LangSmithSandbox, skill paths resolve on the sandbox filesystem, not the local machine. Skills must be uploaded to the sandbox before configuring SkillsMiddleware.

Skills provide on-demand domain knowledge

Skills provide a way to give an agent on-demand domain knowledge when needed using progressive disclosure. Skills can include multi-step workflows, rules, and conventions. By placing this information in a skill, it is not added to the system prompt by default, ensuring tokens are only used when the skill information is needed for a task. When the agent starts, it sees only lightweight metadata about each skill. When a task needs a skill, the agent loads the full skill file on demand.

Middleware lifecycle context hooks

Middleware is the mechanism for implementing life-cycle context between core agent steps. Middleware can hook into any step in the agent cycle and either: (1) update context by modifying state and store to persist changes, update conversation history, or save insights, or (2) jump in the lifecycle by moving to different steps based on context, such as skipping tool execution if a condition is met or repeating the model call with modified context.

Middleware enables context engineering

LangChain middleware is the mechanism under the hood that makes context engineering practical for developers. Middleware allows you to hook into any step in the agent lifecycle to update context and jump to a different step in the agent lifecycle.

Built-in middleware examples

LangChain offers built-in middleware including SummarizationMiddleware (for conversation history summarization) and LLMToolSelectorMiddleware (for intelligent tool selection).

PIIMiddleware with apply_to_output example

Example using PIIMiddleware to redact PII from stream output: ```py from langchain.agents import create_agent from langchain.agents.middleware import PIIMiddleware agent = create_agent( model="gpt-5-nano", tools=[], middleware=[ PIIMiddleware("email", strategy="redact", apply_to_output=True), ], ) ```

PIIMiddleware applies_to_output closes window for PII leakage in streamed output

The built-in PIIMiddleware uses transformer registration to redact PII from streamed wire output. With apply_to_output=True, its registered transformer scrubs detected PII from text deltas, tool-call args, tool outputs, and state snapshots before they leave the run, closing the window where after_model state-level redaction would otherwise let raw PII through to live readers of stream_events(version="v3").

Transformer merge order in create_agent

At compile time, create_agent merges middleware-registered transformer factories with anything passed to transformers= argument. The final order is: (1) built-in ToolCallTransformer, (2) middleware-registered factories in middleware order, (3) caller-supplied transformers= from create_agent. This keeps the built-in tool-call projection in front of consumer transformers and gives caller-supplied entries the final word.

AgentMiddleware.transformers for stream transformer registration

Middleware can declare stream transformer factories by setting the transformers attribute on an AgentMiddleware subclass to a sequence of factories. Each factory has the shape Callable[[tuple[str, ...]], StreamTransformer] and is invoked as factory(scope), where scope is the mini-mux scope tuple (empty for root, non-empty for subgraphs). Returning a fresh transformer per call keeps each subgraph isolated. Requires langchain>=1.3.2.

AgentMiddleware transformer registration example

Example registering transformers on middleware: ```py from langchain.agents import create_agent from langchain.agents.middleware import AgentMiddleware class ToolActivityMiddleware(AgentMiddleware): transformers = (ToolActivityTransformer,) agent = create_agent( model="gpt-5-nano", tools=[get_weather], middleware=[ToolActivityMiddleware()], ) ```

How LangGraph interrupts work in HITL

When a LangGraph agent hits an interrupt: (1) the agent stops executing and emits an interrupt payload, (2) the useStream hook surfaces the interrupt via stream.interrupt, (3) the UI renders a review card with approve/reject/edit options, (4) the user makes a decision, (5) code calls stream.submit() with a resume command, (6) the agent picks up where it left off. The frontend SDK keeps the interrupt alongside the rest of the thread state, so the UI can render it inline in the transcript, in a review queue, in an admin dashboard, or in a modal.

HITL pattern definition and use cases

Human-in-the-Loop (HITL) is a pattern that pauses agent execution when the agent is about to send an email, delete a record, execute a financial transaction, or perform any irreversible operation, requiring a human to review and approve the action first. The pause is durable, built on LangGraph interrupts and checkpoints, allowing users to refresh the page or have reviewers answer from different components while the agent resumes from the exact point where execution stopped.

useStream hook setup for HITL agents

Connect the useStream hook to a human-in-the-loop agent. When the graph hits an interrupt, the hook exposes the pending payload on stream.interrupt. Render an approval card while that value is set, then resume the run with stream.submit(null, { command: { resume: response } }) after the user approves, rejects, edits, or responds to the action.

HITL best practices: make approve the easiest path

If the action looks correct, approving should be a single click. Reserve multi-step flows for reject/edit decisions.

Custom interrupt form interface example

export interface FormField { name: string; label: string; type: "select" | "checkbox" | "textarea" | "currency"; options?: string[]; default?: unknown; } export interface ReviewDecision { approved: boolean; values?: Record<string, unknown>; } export interface InterruptCard { formType: "flight-booking" | "refund-approval" | "content-review"; tool: string; title: string; context: Record<string, unknown>; fields: FormField[]; resolved?: boolean; decision?: ReviewDecision; } Each tool should have a distinct formType so the frontend can switch on it and render the matching form.

Custom interrupt forms for tool-specific reviews

When the generic approve/reject/edit/respond card is insufficient, tools can raise interrupt() from inside the tool with a custom payload describing the exact form the UI should render. Each tool can surface a completely different interface. interrupt() accepts any JSON-serializable value and is generic over input and return types: interrupt<I, R>(value: I): R. This allows providing a form type, title, context, and fields to collect. Export types from the agent module so the frontend can import them and stay in sync.

HITL decision type: Respond

The respond decision type is used when the tool is intentionally a placeholder for human input, such as an ask_user tool. Call stream.submit with response: { decisions: [{ type: "respond", message: "Blue." }] }. The message becomes the tool result and the tool itself is not executed. Do not use respond to deny a proposed action, because it is returned to the model as a successful tool result.

Building approval cards for multiple pending actions

An interrupt can contain multiple actionRequests when the agent wants to perform several actions at once. Render a card for each action and collect all decisions before resuming. The resume payload is a single HITLResponse with one decision per pending action. Example: const resume: HITLResponse = { decisions: actionRequests.map(() => ({ type: "approve" })) }; await stream.submit(null, { command: { resume } });

HITL best practices: log all decisions

For audit trails, log every approve/reject/edit decision with timestamps and the user who made the decision.

HITL decision type: Edit

When a user modifies action arguments before approving, call stream.submit with response: { decisions: [{ type: "edit", editedAction: { name: actionRequest.name, args: { ...actionRequest.args, modifiedField: newValue } } }] }. The agent runs the tool with the edited arguments.

Resume flow after HITL decision

After the user makes a decision: (1) Call stream.submit(null, { command: { resume: hitlResponse } }), (2) The useStream hook sends the resume command to the LangGraph backend, (3) The agent receives the HITLResponse and continues execution with each decision being approve (agent continues executing), reject (tool not executed, agent receives rejection message), edit (agent runs tool with edited arguments), or respond (human's message returned as tool result), (4) The interrupt property resets to null as the agent resumes streaming. Multiple HITL checkpoints can be chained in a single agent run, handled independently.

Keep resolved card on screen with stream.respond example

import { AIMessage } from "langchain"; function handleResolve(decision: ReviewDecision) { const resolvedCard = { ...card, resolved: true, decision }; const cardMessage = new AIMessage({ content: `Review ${decision.approved ? "approved" : "declined"}.`, response_metadata: { cards: resolvedCard }, }); stream.respond(decision, { update: { messages: [cardMessage] } }); } // Render resolved card from message history: {stream.messages.map((msg) => { const card = (msg.response_metadata as { cards?: InterruptCard })?.cards; if (card) return <InterruptForm key={msg.id} card={card} readOnly />; return <Message key={msg.id} message={msg} />; })}

HITL best practices: show clear context

Always display what the agent wants to do and why. Include the action description and the full arguments so users have complete information for their decision.

Custom interrupt form in tool implementation

Inside a tool, raise interrupt() with a typed form spec: const decision = interrupt<InterruptCard, ReviewDecision>({ formType: "flight-booking", tool: "book_flight", title: "Confirm flight booking", context: { origin, destination, date, passengers }, fields: [{ name: "seatClass", label: "Seat class", type: "select", options: ["Economy", "Premium Economy", "Business"], default: "Economy" }, { name: "insurance", label: "Add trip insurance", type: "checkbox", default: false }] }); The returned decision contains the user's approved/declined status and edited form values. Check decision.approved before proceeding with the real work using decision.values.

Keep interrupt card visible with stream.respond()

When resolving a custom interrupt form, use stream.respond(decision, { update: { messages: [cardMessage] } }) to resolve the interrupt AND commit a message carrying the card to state atomically. This maps to LangGraph's Command(resume, update): one checkpoint, no extra state write. The card paints immediately and is reconciled by ID once the resumed run echoes the same message back. The backend never re-emits the card, so it stays rendered without flicker. Render the resolved card by reading it back off the message's response_metadata.

Custom interrupt form frontend rendering example

import { useStream } from "@langchain/react"; import type { InterruptCard, ReviewDecision } from "./agent"; function Chat() { const stream = useStream<typeof myAgent>({ apiUrl: AGENT_URL, assistantId: "hitl_interrupt_forms", }); const card = stream.interrupt?.value as InterruptCard | undefined; return ( <div> {stream.messages.map((msg) => ( <Message key={msg.id} message={msg} /> ))} {card && <InterruptForm card={card} onResolve={handleResolve} />} </div> ); } // InterruptForm renders a flight/refund/content card based on card.formType, // collects card.fields, and calls onResolve with the user's decision and edited values.

Rendering custom interrupt forms on frontend

On the client, the card arrives as stream.interrupt.value. Import InterruptCard and ReviewDecision types from the agent module so the form and payload stay in sync. Switch on card.formType to pick the right form component and feed card.fields into inputs. Import and use the types to ensure type safety between frontend and backend.

HITLRequest interface structure

interface HITLRequest { actionRequests: ActionRequest[]; reviewConfigs: ReviewConfig[]; } interface ActionRequest { name: string; args: Record<string, unknown>; description?: string; } interface ReviewConfig { allowedDecisions: ("approve" | "reject" | "edit" | "respond")[]; } actionRequests is an array of pending actions the agent wants to perform. Each ActionRequest has name (the action name like "send_email" or "delete_record"), args (structured arguments for the action), and optional description (human-readable description of what the action does). reviewConfigs contains per-action configuration controlling which decisions are allowed. reviewConfigs[].allowedDecisions specifies which buttons to show: "approve", "reject", "edit", or "respond".

HITL best practices: validate edited arguments

When users edit action arguments, validate the JSON structure before sending. Show inline errors for malformed input.

HITL decision type: Reject

When a user rejects an action, call stream.submit with response: { decisions: [{ type: "reject", message: "reason" }] }. The tool is not executed. The agent receives the rejection reason and can decide how to proceed. If message is omitted, the backend uses a default message telling the model the tool was not executed and not to retry unless the user asks. For side-effecting tools, pass a clear message telling the agent whether to abandon the action, ask a follow-up question, or try a safer alternative.

Angular useStream HITL example

import { Component } from "@angular/core"; import { injectStream } from "@langchain/angular"; import type { HITLResponse } from "langchain"; const AGENT_URL = "http://localhost:2024"; @Component({ selector: "app-chat", template: ` @for (msg of stream.messages(); track msg.id) { <app-message [message]="msg" /> } @if (stream.interrupt()) { <app-approval-card [interrupt]="stream.interrupt()" (respond)="handleRespond($event)" /> } `, }) export class ChatComponent { stream = injectStream<typeof myAgent>({ apiUrl: AGENT_URL, assistantId: "human_in_the_loop", }); handleRespond(response: HITLResponse) { this.stream.submit(null, { command: { resume: response } }); } }

HITL best practices: set timeouts thoughtfully

Long-running agents should not block indefinitely on human review. Consider showing how long the agent has been waiting.

Svelte useStream HITL example

Vue useStream HITL example

Multiple pending actions handling code example

async function approveAll() { const resume: HITLResponse = { decisions: actionRequests.map(() => ({ type: "approve" })), }; await stream.submit(null, { command: { resume } }); } async function rejectOne(index: number, message: string) { const resume: HITLResponse = { decisions: actionRequests.map((_, i) => i === index ? { type: "reject", message } : { type: "reject", message: "Rejected along with other actions" }, ), }; await stream.submit(null, { command: { resume } }); } async function editOne(index: number, editedArgs: Record<string, unknown>) { const originalAction = actionRequests[index]; const resume: HITLResponse = { decisions: actionRequests.map((_, i) => i === index ? { type: "edit", editedAction: { name: originalAction.name, args: editedArgs }, } : { type: "approve" }, ), }; await stream.submit(null, { command: { resume } }); }

Multi-action review component example

Custom interrupt form tool implementation example

import { createAgent, tool } from "langchain"; import { interrupt } from "@langchain/langgraph"; import { z } from "zod"; const bookFlight = tool( async ({ origin, destination, date, passengers }) => { const decision = interrupt<InterruptCard, ReviewDecision>({ formType: "flight-booking", tool: "book_flight", title: "Confirm flight booking", context: { origin, destination, date, passengers }, fields: [ { name: "seatClass", label: "Seat class", type: "select", options: ["Economy", "Premium Economy", "Business"], default: "Economy", }, { name: "insurance", label: "Add trip insurance", type: "checkbox", default: false }, ], }); if (!decision.approved) { return `Booking cancelled. No flight from ${origin} to ${destination} was reserved.`; } const seatClass = String(decision.values?.seatClass ?? "Economy"); return `Flight booked from ${origin} to ${destination} in ${seatClass}.`; }, { name: "book_flight", description: "Book a flight. Requires human confirmation of trip details.", schema: z.object({ origin: z.string(), destination: z.string(), date: z.string(), passengers: z.number().int().min(1), }), }, );

React useStream HITL example

import { useStream } from "@langchain/react"; const AGENT_URL = "http://localhost:2024"; export function Chat() { const stream = useStream<typeof myAgent>({ apiUrl: AGENT_URL, assistantId: "human_in_the_loop", }); const interrupt = stream.interrupt; return ( <div> {stream.messages.map((msg) => ( <Message key={msg.id} message={msg} /> ))} {interrupt && ( <ApprovalCard interrupt={interrupt} onRespond={(response) => stream.submit(null, { command: { resume: response } }) } /> )} </div> ); }

HITL best practices: persist interrupt state

If the user refreshes the page, the interrupt should still be visible. useStream handles this via the thread's checkpoint.

HITL decision type: Approve

When a user approves an action, call stream.submit with response: { decisions: [{ type: "approve" }] }. The action proceeds as-is without modification.

Python middleware to map CopilotKit context to structured output

Use @wrap_model_call decorator to create apply_structured_output_schema middleware. It reads output_schema from runtime context or copilotkit state context, normalizes it if a string (parse JSON), and applies it to request via request.override(response_format=ProviderStrategy(schema=schema, strict=True)). Use @before_agent decorator for normalize_context to flatten copilotkit context from list format to dict.

CopilotKitMiddleware bridges LangGraph and CopilotKit state

CopilotKitMiddleware allows a LangGraph graph, LangChain agent, or Deep Agent to speak the Agent UI (AG-UI) wire protocol, stream tool and message events to a chat UI, and read or write the shared CopilotKit slice of state. Add it to the middleware list for create_agent or create_deep_agent.

Python FastAPI app with CopilotKit AG-UI bridge example

from typing import Any, TypedDict; from ag_ui_langgraph import add_langgraph_fastapi_endpoint; from copilotkit import CopilotKitMiddleware, CopilotKitState, LangGraphAGUIAgent; from fastapi import FastAPI; from langchain.agents import create_agent; class AgentState(CopilotKitState): pass; class AgentContext(TypedDict, total=False): output_schema: dict[str, Any]; agent = create_agent(model='openai:gpt-5.5', middleware=[normalize_context, CopilotKitMiddleware(), apply_structured_output_schema], context_schema=AgentContext, state_schema=AgentState, system_prompt='You are a helpful UI assistant. Build visual responses using the available components.'); app = FastAPI(); add_langgraph_fastapi_endpoint(app=app, agent=LangGraphAGUIAgent(name='copilotkit_shadify', description='A UI assistant that returns structured component payloads.', graph=agent), path='/');

Extend LangGraph deployment with custom HTTP endpoint in langgraph.json

In langgraph.json, point http.app at custom app entrypoint. Python example: {"dependencies": ["."], "graphs": {"copilotkit_shadify": "./main.py:agent"}, "http": {"app": "./main.py:app"}}. This mounts a CopilotKit-aware runtime without replacing the underlying LangGraph deployment.

Deep Agent with CopilotKitMiddleware example

Example: from deepagents import create_deep_agent; from copilotkit import CopilotKitMiddleware; from langgraph.checkpoint.memory import MemorySaver; agent = create_deep_agent(model='openai:gpt-5.5', tools=[get_weather], middleware=[CopilotKitMiddleware()], system_prompt='You are a helpful research assistant.', checkpointer=MemorySaver()). This shows adding CopilotKitMiddleware to the middleware list for routing frontend tool calls and aligning chat state.

CopilotKit server-side components and their roles

Server-side components: CopilotKitMiddleware merges CopilotKit and AG-UI state and requests into agents. CopilotKitState is a custom state subclass extending it so CopilotKit key is part of graph state. LangGraphAGUIAgent bundles a compiled graph with name and description for runtime. add_langgraph_fastapi_endpoint (from ag-ui-langgraph) wires a FastAPI app so CopilotKit can run the graph on the same LangGraph process.

CopilotKit architecture: React app, custom endpoint, LangGraph deployment

CopilotKit sits between the React app and the LangGraph deployment. The frontend sends conversation state to a custom /api/copilotkit route mounted alongside the graph API, that route forwards the request to LangGraph, and the response comes back with both assistant messages and any structured UI payloads the component registry can render.

onCreated callback for chaining submissions

The onCreated callback fires when a new run is created, giving a hook to submit follow-up messages programmatically. This is useful for building multi-step workflows where the next question depends on the previous submission being accepted.

Clear entire queue

Remove all pending messages at once using await queue.clear(). This is useful when the user changes context or wants to start over.

Cancel single queue entry

Remove a specific message from the queue by its ID using await queue.cancel(entryId). The agent will skip it and move to the next entry.

QueueList component example

React message queue example

import { useStream, useSubmissionQueue } from "@langchain/react"; function Chat() { const stream = useStream<typeof myAgent>({ apiUrl: "http://localhost:2024", assistantId: "simple_agent", }); const queue = useSubmissionQueue(stream); const handleSubmit = (text: string) => { stream.submit({ messages: [{ type: "human", content: text }], }); }; const pendingCount = queue.size; const entries = queue.entries; return ( <div> <MessageList messages={stream.messages} /> {pendingCount > 0 && <QueueList entries={entries} queue={queue} />} <ChatInput onSubmit={handleSubmit} /> </div> ); }

SubmissionQueueEntry object fields

Each SubmissionQueueEntry object contains the following fields: `id` (string, unique identifier for this queue entry), `values` (object, the input values including messages that were submitted), `options` (object, any additional options passed with the submission), `createdAt` (string, ISO timestamp of when the entry was created).

useStream and useSubmissionQueue setup

Connect useStream to your agent, then pair it with the submission queue helper for your framework. Call stream.submit() to send messages while a run is in progress; pass multitaskStrategy: "enqueue" on submissions that should wait behind the active request. Read queue.entries and queue.size to render pending work, and use queue.cancel() or queue.clear() to remove items before they start processing.

Queue object properties and methods

The queue object has the following properties and methods: `queue.entries` (SubmissionQueueEntry[], array of all pending queue entries), `queue.size` (number, number of entries currently in the queue), `queue.cancel(id)` ((id: string) => Promise<void>, cancel a specific queued entry by ID), `queue.clear()` (() => Promise<void>, cancel all queued entries).

multitaskStrategy enqueue parameter

Pass `multitaskStrategy: "enqueue"` when you want a submission to wait behind the currently running request. While the agent is processing, queued submissions are added to the active thread's queue. Once the current run completes, the next queued message is dispatched automatically.

Queue cancellation only affects unstarted messages

Cancelling a queue entry only affects messages that have not yet started processing. If the agent is already working on a message, cancelling it from the queue has no effect. Use stream.stop() to interrupt the current run.

Message queues definition and purpose

Message queuing lets users send multiple messages in rapid succession without waiting for the agent to finish processing the current one. Each message is accepted immediately, queued for the active thread, and processed sequentially, giving full visibility and control over the pending work.

onCreated callback example

stream.submit( { messages: [{ type: "human", content: "What is quantum computing?" }] }, { onCreated(run) { console.log("Run created:", run.runId); // Chain a follow-up stream.submit({ messages: [{ type: "human", content: "Give me a simple analogy." }], }); }, } );

Give your agent this brain