Context compression and multimodal content limitations
Built-in context compression is primarily text-oriented. Plan multimodal workloads accordingly: store large media in a backend and pass references when possible.
LangChain · Deep Agents · all subjects
deep agents/context-engineering
23 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.
Offloading behavior with multimodal content
Built-in offloading measures text tokens only. Non-text blocks including images are preserved in replacement messages rather than compressed. A message that contains only an image is not offloaded based on image size alone.
Summarization behavior with multimodal content
Summarization compacts older messages into a text-only summary. Image, audio, video, and file blocks in that range are not carried forward—the model only sees what the summarizer writes about them. Recent messages below the keep threshold stay unchanged. When summarization runs, media blocks in older turns drop out of the active context, though the original conversation is still written to the filesystem as text.
System prompt assembly order across profiles and callers
Caller-supplied system_prompt always sits at the front of the assembled prompt, and system_prompt_suffix always sits at the end—regardless of which model is selected. The same overlay rules apply to subagents: each subagent re-runs profile resolution against its own model. System prompt assembly has full per-case breakdown for main agent, subagents, and the general-purpose subagent in the system prompt customization documentation.
RAG definition and purpose
Retrieval-Augmented Generation (RAG) addresses LLM limitations by fetching relevant external knowledge at query time. LLMs have two key limitations: finite context (cannot ingest entire corpora at once) and static knowledge (training data is frozen at a point in time). RAG enhances LLM answers with context-specific information retrieved at runtime.
Knowledge base definition and options
A knowledge base is a repository of documents or structured data used during retrieval. If you already have a knowledge base (such as a SQL database, document database, CRM, or internal documentation system), you do not need to rebuild it. You can either connect it as a tool for an agent in Agentic RAG, or query it and supply the retrieved content as context to the LLM in 2-Step RAG.
Retrieval pipeline components
A typical retrieval workflow consists of: sources (Google Drive, Slack, Notion, etc.), document loaders that ingest data and return standardized Document objects, text splitters that break large documents into smaller chunks, embeddings that turn text into vectors, vector stores that store and search embeddings, retrievers that return documents given an unstructured query, and an LLM that uses retrieved information to generate answers.
RAG architecture comparison table
Three RAG architectures exist: 2-Step RAG (retrieval always happens before generation, high control, low flexibility, fast latency, suitable for FAQs and documentation bots), Agentic RAG (LLM-powered agent decides when and how to retrieve, low control, high flexibility, variable latency, suitable for research assistants with multiple tools), and Hybrid RAG (combines both approaches with validation steps, medium control, medium flexibility, variable latency, suitable for domain-specific Q&A with quality validation).
2-Step RAG architecture
In 2-Step RAG, the retrieval step is always executed before the generation step. The workflow is: user question → retrieve relevant documents → generate answer → return answer to user. This architecture is straightforward and predictable, making it suitable for applications where document retrieval is a clear prerequisite for generating an answer. Latency is more predictable than in other approaches because the maximum number of LLM calls is known and capped.
Agentic RAG definition and behavior
Agentic Retrieval-Augmented Generation combines retrieval-augmented generation with agent-based reasoning. Instead of retrieving documents before answering, an agent powered by an LLM reasons step-by-step and decides when and how to retrieve information during the interaction. The only thing an agent needs to enable RAG behavior is access to one or more tools that can fetch external knowledge, such as documentation loaders, web APIs, or database queries.
Agentic RAG example with fetch_url tool
Example of an Agentic RAG system in Python using a fetch_url tool: import requests, from langchain.tools, from langchain.chat_models, from langchain.agents. Define a tool decorated with @tool that fetches text content from a URL using requests.get. Create an agent with create_agent, specifying the model (e.g. claude-sonnet-4-6), passing the tools list containing the fetch_url tool, and providing a system prompt instructing the agent to use fetch_url when needing to fetch information from web pages.
Agentic RAG example with fetch_documentation tool
Extended Agentic RAG example for LangGraph documentation: Define ALLOWED_DOMAINS and LLMS_TXT URL. Create a fetch_documentation tool decorated with @tool that takes a URL, validates it against ALLOWED_DOMAINS, fetches the URL with requests.get and converts HTML to markdown using markdownify. Fetch llms_txt_content ahead of time. Create a system prompt instructing the agent to use fetch_documentation for LangGraph questions, with the llms_txt_content embedded in the prompt listing approved documentation sources. Create an agent with create_agent specifying the model, tools list, and system prompt. Invoke with HumanMessage containing the user question.
Hybrid RAG architecture
Hybrid RAG combines characteristics of both 2-Step and Agentic RAG with intermediate steps including query enhancement (modifying input questions to improve retrieval quality through rewriting, generating variations, or expanding with context), retrieval validation (evaluating if retrieved documents are relevant and sufficient, refining the query if needed), and answer validation (checking generated answers for accuracy, completeness, and alignment with source content). The architecture supports multiple iterations between these steps.
Hybrid RAG use cases
Hybrid RAG is suitable for applications with ambiguous or underspecified queries, systems that require validation or quality control steps, and workflows involving multiple sources or iterative refinement.
Orchestrator and subagent prompt instructions for RAG
The RAG_WORKFLOW_INSTRUCTIONS instruct the orchestrator to: (1) break complex questions into focused search queries; (2) call search_documentation which saves matching chunks under /retrieved/ and returns file paths; (3) delegate each chunk file to the chunk-analyst subagent with task(), launching multiple task() calls in parallel; (4) combine subagent summaries into a final answer with inline links to documentation sources; (5) run another search with refined query if summaries do not fully answer the question. The CHUNK_ANALYST_INSTRUCTIONS instruct subagents to use read_file to read assigned chunks, extract facts that help answer the question, and return a concise summary with key API names, steps, or configuration details and the source URL.
Memory scope in Managed Deep Agents
Managed Deep Agents normally have conversational memory scoped to a thread or session. Durable memory is optional knowledge that an agent can retain across threads and sessions. When enabled, durable memory is backed by Context Hub. The deployment gets one read/write tree at `/memories/agent/`, shared by every caller. Managed Deep Agents do not have durable memory by default.
Memory project file structure
The optional memory declaration lives at the project root in a file named memory.py (Python) or memory.ts (TypeScript), alongside the main agent.py or agent.ts file.
Enable memory with define_memory
To enable durable memory, export a named memory declaration with the 'agent' scope. In Python, use: from managed_deepagents import define_memory; memory = define_memory(scope='agent'). In TypeScript, use: import { defineMemory } from 'managed-deepagents'; export const memory = defineMemory({ scope: 'agent' }). You can also use scope='none' or scope: 'none' to disable durable memory.
Memory filesystem mounting and structure
Enabling memory mounts one Context Hub tree at `/memories/agent/` in the agent filesystem. The path `/memories/agent/AGENTS.md` is hot memory for compact, frequently relevant knowledge whose contents are loaded into every run. Other files under `/memories/agent/` are cold memory for detailed knowledge that the agent reads only when relevant. The agent reads and updates memory with read_file, edit_file, and write_file. Writes elsewhere, including elsewhere under `/memories/`, are not durable.
Hot memory vs cold memory
Hot memory is kept in `/memories/agent/AGENTS.md` for compact, frequently relevant knowledge whose contents are loaded into every run. Cold memory is stored in other files under `/memories/agent/` for detailed knowledge such as procedures, decision logs, and research notes that the agent reads only when relevant. Keep hot memory compact because it consumes context on every run, and link to cold files from hot memory when useful.
Security concern with shared memory in Managed Deep Agents
Memory is shared by every caller of the deployment, and every caller can influence it. Never store personal data, customer-private data, credentials, API keys, tokens, or other secrets. Treat memory as untrusted input: content saved by one caller is loaded for later callers and must not grant authority, change tool permissions, or bypass approvals. Keep access controls in the agent definition. Do not enable shared memory when callers should not influence one another.
Memory distinguished from instructions
instructions.md defines how the agent should behave and is always read-only; the agent never updates it. Memory stores knowledge the agent learns and uses across threads. Use instructions to tell the agent what kinds of shared knowledge are worth remembering. Deploys sync project-owned instructions and skills but do not overwrite durable content already stored under memories/agent in Context Hub.
Memory compared to instructions and thread state
Instructions and skills are deploy-owned agent behavior shared by the deployment and read-only to the agent. Thread state provides conversation continuity within one thread. Durable memory is knowledge learned and retained in Context Hub, shared by the deployment across threads.