agents/middleware/context-management

34 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.

Context management middleware strategies

Context management middleware includes summarization to compress history before context window overflow, memory to load persistent instructions at startup so knowledge carries across sessions, and skills to surface domain knowledge on demand rather than loading everything upfront. This is provided by SummarizationMiddleware and MemoryMiddleware.

SummarizationMiddleware for context management

SummarizationMiddleware compresses older turns when history grows too large, so the agent keeps working without manually trimming messages. This becomes important in multi-turn sessions with follow-up questions that trigger more file reads or script runs, preventing context window overflow.

SummarizationMiddleware persistent context updates

SummarizationMiddleware automatically handles conversation history summarization as a life-cycle context pattern. Unlike transient message trimming in model context, summarization persistently updates state by replacing old messages with a summary that is saved for all future turns. When the conversation exceeds a token limit, it: (1) summarizes older messages using a separate LLM call, (2) replaces them with a summary message in State (permanently), and (3) keeps recent messages intact for context.

SummarizationMiddleware configuration parameters

SummarizationMiddleware in Python accepts three parameters: `model` (string, the LLM model to use for summarization), `trigger` (dict with 'tokens' key specifying the token threshold to trigger summarization), and `keep` (tuple with namespace and count, e.g. ('messages', 20) to keep the 20 most recent messages). In JavaScript/TypeScript, the configuration uses `{ model: string, trigger: { tokens: number }, keep: { messages: number } }`.

Transient vs persistent context changes

Model context changes are transient (per-call only), affecting only the current agent step. Life-cycle context changes persist to state, affecting all future turns. This distinction is important when choosing between model-level context engineering and middleware-based approaches.

ContextEditingMiddleware purpose

ContextEditingMiddleware manages conversation context by clearing older tool call outputs when token limits are reached, while preserving recent results. This helps keep context windows manageable in long conversations with many tool calls. It is useful for long conversations with many tool calls that exceed token limits, reducing token costs by removing older tool outputs no longer relevant, and maintaining only recent N tool results in context.

ClearToolUsesEdit configuration parameters

ClearToolUsesEdit accepts: trigger (number, default 100000) token count that triggers the edit when conversation exceeds this threshold; clear_at_least (number, default 0) minimum tokens to reclaim when edit runs, if 0 clears as much as needed; keep (number, default 3) number of most recent tool results preserved and never cleared; clear_tool_inputs (boolean, default False) whether to clear tool call parameters on AI message by replacing with empty objects; exclude_tools (list[string], default empty) tool names excluded from clearing; placeholder (string, default "[cleared]") text inserted for cleared tool outputs.

ContextEditingMiddleware example - basic setup

from langchain.agents import create_agent from langchain.agents.middleware import ContextEditingMiddleware, ClearToolUsesEdit agent = create_agent( model="gpt-5.5", tools=[search_tool, your_calculator_tool, database_tool], middleware=[ ContextEditingMiddleware( edits=[ ClearToolUsesEdit( trigger=2000, keep=3, clear_tool_inputs=False, exclude_tools=[], placeholder="[cleared]", ), ], ), ], ) This shows basic context editing triggered at 2000 tokens, keeping 3 most recent results.

FilesystemMiddleware from Deep Agents purpose

FilesystemMiddleware from Deep Agents provides four tools for interacting with both short-term and long-term memory: ls (list files in filesystem), read_file (read entire file or certain number of lines), write_file (write new file to filesystem), edit_file (edit existing file). It addresses the main challenge of context engineering in building effective agents, particularly useful when tools return variable-length results.

FilesystemMiddleware configuration parameters

FilesystemMiddleware accepts: backend (optional custom backend, defaults to StateBackend), system_prompt (optional custom addition to system prompt), custom_tool_descriptions (optional dictionary with custom descriptions for filesystem tools like ls, read_file), tools (optional allowlist restricting which filesystem tools are exposed, e.g., ["read_file", "ls", "glob", "grep"]).

FilesystemMiddleware persistent storage with CompositeBackend

To enable persistent storage across threads with FilesystemMiddleware, configure a CompositeBackend that routes specific paths to a StoreBackend. Example: CompositeBackend with default=StateBackend() and routes={"/memories/": StoreBackend()}. Files prefixed with /memories/ are saved to persistent storage and survive across different threads. Files without this prefix remain in ephemeral state storage.

FilesystemMiddleware persistent storage example

from langchain.agents import create_agent from deepagents.middleware import FilesystemMiddleware from deepagents.backends import CompositeBackend, StateBackend, StoreBackend from langgraph.store.memory import InMemoryStore store = InMemoryStore() agent = create_agent( model="claude-sonnet-4-6", store=store, middleware=[ FilesystemMiddleware( backend=CompositeBackend( default=StateBackend(), routes={"/memories/": StoreBackend()} ), ), ], ) This shows persistent storage configuration for FilesystemMiddleware.

ContextEditingMiddleware token counting methods

ContextEditingMiddleware accepts token_count_method parameter (string, default 'approximate') with options: 'approximate' for fast approximate counting, or 'model' for model-based token counting.

LangGraph state for information passing

LangGraph state can be used to pass information between workflow steps, allowing each part of the workflow to read and update structured fields, making it easy to share data and context across nodes.

Checkpointer persistence for state machine workflows

A checkpointer (e.g., InMemorySaver) is essential for state machines to maintain state across conversation turns. Without it, the current_step and other state fields would be lost between user messages, breaking the workflow. The checkpointer is passed to create_agent and paired with a thread_id in the config.

SummarizationMiddleware for managing message history growth

SummarizationMiddleware compresses conversation history as it grows to prevent token bloat. It takes parameters: model (chat model to use for summarization), trigger (when to summarize, e.g., ('tokens', 4000)), and keep (how many messages to preserve, e.g., ('messages', 10)). Multiple middleware can be chained in the middleware list.

Thread ID for conversation state persistence

Pass a thread_id in the config dict when invoking an agent with a checkpointer: config = {'configurable': {'thread_id': thread_id}}. Each thread_id maintains separate conversation history and state. This allows multiple conversations to run in parallel without interference.

Message history grows during state machine conversation

As the state machine progresses through turns, HumanMessage, ToolMessage, and AssistantMessage objects accumulate in the messages list. This can cause token bloat. Use SummarizationMiddleware or periodic message cleanup to manage this.

Wrapping stateless router as tool for conversation memory

To add conversation memory to a stateless router, wrap the router as a tool that a conversational agent can call. Create a search_knowledge_base tool that invokes the workflow and returns the final_answer. Use this tool in a conversational agent with a checkpointer (e.g., InMemorySaver) to maintain multi-turn conversation context. This approach keeps the router stateless while the conversational agent handles memory.

Stateful vs stateless routers

A stateless router handles each request independently with no memory between calls. For multi-turn conversations, use a stateful approach by wrapping the stateless router as a tool in a conversational agent. Full persistence approach stores message history at the router level, but adds complexity. Stateful routers may have consistency issues if different agents have different tones or prompts—consider handoffs or subagents patterns instead.

Stateful router with full persistence

If the router itself needs to maintain state, use persistence to store message history. When routing to an agent, fetch previous messages from state and selectively include them in the agent's context as a lever for context engineering.

Runtime context dependency injection pattern

Runtime context provides dependency injection for tools and middleware. Instead of hardcoding values or using global state, you can inject runtime dependencies (like database connections, user IDs, or configuration) when invoking your agent. This makes tools more testable, reusable, and flexible, and keeps things stateless.

Define agent context schema with context_schema parameter

When creating an agent with create_agent (Python) or createAgent (JavaScript), you can specify a context_schema (Python) or contextSchema (JavaScript) to define the structure of the context stored in the agent Runtime. This schema is then passed when invoking the agent.

Python agent context schema and invocation example

Example showing how to define and use context schema in Python: from dataclasses import dataclass from langchain.agents import create_agent @dataclass class Context: user_name: str agent = create_agent( model="gpt-5-nano", tools=[...], context_schema=Context ) agent.invoke( {"messages": [{"role": "user", "content": "What's my name?"}]}, context=Context(user_name="John Smith") ) This demonstrates creating an agent with a dataclass context schema and passing context during invocation.

JavaScript agent context schema and invocation example

Example showing how to define and use context schema in JavaScript: import * as z from "zod"; import { createAgent } from "langchain"; const contextSchema = z.object({ userName: z.string(), }); const agent = createAgent({ model: "gpt-5.5", tools: [...], contextSchema, }); const result = await agent.invoke( { messages: [{ role: "user", content: "What's my name?" }] }, { context: { userName: "John Smith" } } ); This demonstrates creating an agent with a Zod context schema and passing context during invocation.

PostgresSaver checkpoint for production

In production, use PostgresSaver from `langgraph.checkpoint.postgres` for database-backed checkpointing. Install with `pip install -U langgraph-checkpoint-postgres psycopg[binary]`. Initialize with `PostgresSaver.from_conn_string(DB_URI)` and call `checkpointer.setup()` to auto-create tables in PostgreSQL. The connection string format is `postgresql://postgres:postgres@localhost:5432/postgres?sslmode=disable`.

Trim messages strategy with before_model middleware

Use the `@before_model` middleware decorator to trim message history before the model is called. Keep only the last N messages or a subset based on token limits to fit the context window. A common strategy is to keep the first message (often a system message) and the most recent 3-4 messages.

RemoveMessage for managing history

Use `RemoveMessage` to delete specific messages or all messages from the graph state. For `RemoveMessage` to work, the state key must use the `add_messages` reducer, which the default `AgentState` provides. Use `RemoveMessage(id=m.id)` to remove specific messages or `RemoveMessage(id=REMOVE_ALL_MESSAGES)` to clear all messages.

Message history deletion pitfall

When deleting messages, ensure the resulting message history is valid. Some LLM providers expect message history to start with a user message, and most require assistant messages with tool calls to be followed by corresponding tool result messages.

Delete messages example with after_model

Example using `@after_model` middleware to delete old messages after model calls: ```python @after_model def delete_old_messages(state: AgentState, runtime: Runtime) -> dict | None: messages = state["messages"] if len(messages) > 2: return {"messages": [RemoveMessage(id=m.id) for m in messages[:2]]} return None ``` This removes the earliest messages while keeping recent ones.

SummarizationMiddleware for message history

Use `SummarizationMiddleware` to summarize earlier messages in conversation history and replace them with a summary instead of trimming or deleting. This preserves information from the message queue that would otherwise be lost. Configure with parameters: `model` (the model to use for summarization), `trigger` (tuple like ('tokens', 4000) for when to trigger), and `keep` (tuple like ('messages', 20) for how many messages to keep).

SummarizationMiddleware Python example

Example using `SummarizationMiddleware`: ```python from langchain.agents import create_agent from langchain.agents.middleware import SummarizationMiddleware from langgraph.checkpoint.memory import InMemorySaver checkpointer = InMemorySaver() agent = create_agent( model="gpt-5.5", tools=[...], middleware=[ SummarizationMiddleware( model="gpt-5.4-mini", trigger=("tokens", 4000), keep=("messages", 20) ) ], checkpointer=checkpointer, ) ```

Trim messages before_model example

Example using `@before_model` to trim messages: ```python @before_model def trim_messages(state: AgentState, runtime: Runtime) -> dict[str, Any] | None: messages = state["messages"] if len(messages) <= 3: return None first_msg = messages[0] recent_messages = messages[-3:] if len(messages) % 2 == 0 else messages[-4:] new_messages = [first_msg] + recent_messages return {"messages": [RemoveMessage(id=REMOVE_ALL_MESSAGES), *new_messages]} ```

Conversation memory in voice agents

Voice agents maintain conversation state across turns using a checkpointer and unique thread ID. The InMemorySaver() checkpointer stores conversation history. A unique thread_id is generated per conversation session using uuid7() in Python. This thread_id is passed in the configurable dict to astream_events() to allow the agent to reference previous exchanges.

Give your agent this brain