agents

134 notes in this subject, read out of this brain and free to use. This is page 2 of 3.

Enable LangSmith tracing environment variables

To enable LangSmith tracing for agents, set two environment variables: LANGSMITH_TRACING=true and LANGSMITH_API_KEY=<your-api-key>. These are the only configuration steps required to start capturing traces.

LangSmith prerequisites

Before using LangSmith tracing, you must have a LangSmith account (available free at smith.langchain.com) and generate a LangSmith API key following the Create an API key guide.

Traces record full agent execution

Traces in LangSmith record every step of an agent's execution, from the initial user input to the final response, including all tool calls, model interactions, and decision points. This data helps debug issues, evaluate performance across different inputs, and monitor usage patterns in production.

Default LangSmith project name

By default, agent traces are logged to the LangSmith project named 'default'. Custom project names can be configured separately.

JavaScript createAgent example with LangSmith tracing

Example showing automatic tracing with createAgent: import { createAgent } from "@langchain/agents"; function sendEmail(to: string, subject: string, body: string): string { return `Email sent to ${to}`; } function searchWeb(query: string): string { return `Search results for: ${query}`; } const agent = createAgent({ model: "gpt-5.5", tools: [sendEmail, searchWeb], systemPrompt: "You are a helpful assistant that can send emails and search the web." }); const response = await agent.invoke({ messages: [{ role: "user", content: "Search for the latest AI news and email a summary to john@example.com" }] }); All steps are traced automatically when the agent runs.

LangChain agents built on LangGraph

LangChain's agents are built on top of LangGraph. This integration allows LangChain agents to take advantage of LangGraph's durable execution, human-in-the-loop support, persistence, and more.

Agent = Model + Harness definition

LangChain defines an agent as the combination of a model and a harness. The harness is everything around the model loop: the prompt, the tools, and any middleware that shapes behavior.

LangChain vs LangGraph vs Deep Agents comparison

Deep Agents is a batteries-included agent framework built on LangChain agents, featuring automatic context compression, a virtual filesystem, and subagent-spawning. LangChain agents provide a highly customizable harness easily tailored to use case and data. LangGraph is a low-level orchestration framework for advanced needs combining deterministic and agentic workflows.

LangSmith for agent debugging and observability

LangSmith enables tracing of agent requests, debugging of agent behavior, and evaluation of outputs. Setting LANGSMITH_TRACING=true and providing an API key enables tracing. LangSmith allows inspection of traces, tool calls, state transitions, and latency.

LangChain agent evolution with ReAct

In December 2022, the first general purpose agents were added to LangChain, based on the ReAct paper (Reasoning and Acting). These agents used LLMs to generate JSON representing tool calls, which was then parsed to determine what tools to call.

LangGraph becomes preferred architecture

In October 2024, LangGraph became the preferred way to build any AI application that is more than a single LLM call. As developers tried to improve reliability, they needed more control than high-level interfaces provided. Most chains and agents were marked as deprecated in LangChain with migration guides to LangGraph. One high-level agent abstraction was created in LangGraph with the same interface as the original ReAct agents from LangChain.

LangChain version 1.0.0 release

LangChain released version 1.0.0 on 2025-10-20 with two major changes: (1) Complete revamp of all chains and agents in `langchain`, replacing them with only one high-level abstraction: an agent abstraction built on top of LangGraph. Users can continue using old LangChain by installing `langchain-classic` (Python) or `@langchain/classic` (JavaScript). (2) A standard message content format where model APIs evolved from returning messages with simple content strings to more complex output types including reasoning blocks, citations, and server-side tool calls. LangChain evolved its message formats to standardize these across providers.

Deep Agents release as open-source agent harness

Deep Agents was released on 2026-03-15 (v0.5.3) as an open-source agent harness built on LangGraph. While LangChain provides flexible building blocks for custom agent architectures, Deep Agents offers a batteries-included option for complex, long-running tasks like research and coding. It adds built-in planning tools, a virtual filesystem with pluggable backends (in-memory, disk, LangGraph store, sandboxes), and subagent spawning for context isolation. Use Deep Agents for autonomous agents with predefined tools; use LangChain for full control over agent architecture.

SQL agent architecture and workflow

A SQL agent using LangChain follows this workflow: fetch available tables and schemas from the database, decide which tables are relevant to the question, fetch schemas for relevant tables, generate a query based on the question and schema information, double-check the query using an LLM, execute the query and return results, correct mistakes surfaced by the database engine until the query is successful, and formulate a response based on the results.

SQL agent security warning

Building Q&A systems on SQL databases requires executing model-generated SQL queries, which carries inherent risks. Database connection permissions must always be scoped as narrowly as possible for the agent's needs. This mitigates, though does not eliminate, the risks of building a model-driven system.

SQL agent minimal installation

For Python SQL agent development, install: langchain, langgraph. For JavaScript, install: langchain, @langchain/core, sqlite3, zod.

SQL agent database permission security

Database tools for SQL agents are minimal wrappers for demonstration only and are not intended to be secure or used in production. Use narrowly scoped database permissions and add application-specific validation before executing model-generated SQL.

Three approaches to testing LangChain agents

There are three main approaches to testing agentic applications: unit tests that exercise small, deterministic pieces in isolation using in-memory fakes for quick and deterministic assertion of exact behavior; integration tests that test the agent using real network calls to confirm components work together, credentials and schemas line up, and latency is acceptable; and evals that use evaluators to assess agent execution trajectory either via deterministic matching or an LLM judge.

Why integration testing is emphasized for agentic applications

Agentic applications tend to lean more on integration testing because they chain multiple components together and must deal with flakiness due to the nondeterministic nature of LLMs.

Unit testing approach for agents

Unit tests exercise small, deterministic pieces of an agent in isolation using in-memory fakes. This approach allows you to assert exact behavior quickly and deterministically without API calls.

Integration testing approach for agents

Integration tests test the agent using real network calls to confirm that components work together, credentials and schemas line up, and latency is acceptable.

Evals approach for agents

Evals use evaluators to assess an agent's execution trajectory, either via deterministic matching or an LLM judge.

Challenge with testing agentic applications

The model's black-box nature makes it hard to predict how a tweak in one part of an agent will affect the whole, which is why thorough testing is essential for building production-ready agents.

Separate integration tests from unit tests using pytest markers

Integration tests are slower and require API credentials, so keep them separate from unit tests. Use pytest markers to tag integration tests. In pytest.ini or pyproject.toml, define the integration marker and configure addopts to exclude integration tests from default runs with '-m "not integration"'. Run integration tests explicitly with 'pytest -m integration'.

Python integration test example with pytest marker

Example showing how to write a Python integration test: import pytest, decorate the test function with @pytest.mark.integration, create an agent with a real model like 'claude-sonnet-4-6', invoke it with messages, and assert on the structure of results. For pytest configuration, use either pytest.ini with [pytest] section defining markers and addopts, or pyproject.toml with [tool.pytest.ini_options] section.

Separate integration tests in JavaScript using file naming and vitest

In JavaScript/TypeScript, use a file naming convention to separate integration tests. Name integration test files '*.int.test.ts' and configure vitest to exclude them from default runs. In vitest.config.ts, create conditional configuration: if mode is 'int', include only '**/*.int.test.ts' files and set testTimeout to 100_000 ms; otherwise exclude '**/*.int.test.ts' and set testTimeout to 30_000 ms. Add 'dotenv/config' to setupFiles. In package.json, define scripts for 'test' and 'test:integration' (vitest --mode int).

JavaScript integration test configuration example

Example vitest.config.ts for separating integration tests: define config function that checks env.mode. For 'int' mode, return config with test section including testTimeout: 100_000, include: ['**/*.int.test.ts'], and setupFiles: ['dotenv/config']. For other modes, return config with testTimeout: 30_000, exclude: ['**/*.int.test.ts', ...configDefaults.exclude]. In package.json, add scripts with 'test': 'vitest' and 'test:integration': 'vitest --mode int'.

Load API keys from environment variables in integration tests

Integration tests require real API credentials. Load them from environment variables so keys stay out of source control. In Python, use a conftest.py fixture with autouse=True to validate required keys are available, skipping tests if keys are missing using pytest.skip(). For local development, store keys in a .env file and load them with python-dotenv's load_dotenv() in conftest.py. Add .env to .gitignore to avoid committing credentials. In CI, inject secrets through the provider's secrets management.

Python API key validation in conftest.py

Example Python conftest.py fixture for checking API keys: import os and pytest, create a fixture with @pytest.fixture(autouse=True) decorator and name check_api_keys, check if os.environ.get('OPENAI_API_KEY') is set, and if not, call pytest.skip('OPENAI_API_KEY not set'). For loading from .env, import load_dotenv from dotenv package and call load_dotenv() in conftest.py.

JavaScript API key loading and skipping with vitest

In JavaScript/TypeScript vitest, add 'dotenv/config' as a setupFiles entry in vitest.config.ts so environment variables load automatically from .env file. To skip tests when keys are missing, use test.skipIf(!process.env.OPENAI_API_KEY)('test name', async () => { ... }) syntax to conditionally skip tests based on missing environment variables.

Assert on response structure, not exact content in LLM tests

LLM responses vary between runs, so instead of asserting on exact output strings, verify the structural properties of the response: message types, tool call names, argument shapes, and message count. This approach makes tests reliable despite the nondeterministic nature of LLM outputs.

Python example: assert tool call structure in agent test

Example Python test asserting tool call structure: create an agent with a real model, invoke with HumanMessage content, extract messages from result, filter for tool_calls by checking hasattr(msg, 'tool_calls') and iterating through msg.tool_calls, assert that any tool call has name 'get_weather', assert that the last message is an AIMessage instance, and assert that the last message content has length > 0. This avoids asserting on exact response text.

JavaScript example: assert tool call structure in agent test

Example JavaScript test using custom test matchers: create an agent with model 'claude-sonnet-4-6' and tools, invoke with HumanMessage, find AIMessage with tool_calls using result.messages.find(), then use expect matchers: expect(aiMsg).toContainToolCall({ name: 'get_weather' }) and expect(result.messages.at(-1)).toBeAIMessage(). These custom matchers produce clear error messages on failure.

LangChain custom vitest matchers setup

LangChain ships custom vitest matchers for structural assertions. Set up by creating a vitest.setup.ts file that imports langchainMatchers from '@langchain/core/testing' and calls expect.extend(langchainMatchers). Reference this setup file in vitest.config.ts under test.setupFiles. TypeScript types are included automatically for autocomplete.

Message type matchers in vitest

LangChain vitest matchers for checking message types: toBeHumanMessage(), toBeAIMessage(), toBeSystemMessage(), and toBeToolMessage(). Call without arguments to check only type, pass a string to match content, or pass an object to match specific fields. Examples: expect(lastMessage).toBeAIMessage(), expect(lastMessage).toBeAIMessage('It\'s 72°F and sunny.'), expect(lastMessage).toBeAIMessage({ name: 'weather-bot' }), expect(toolMsg).toBeToolMessage({ tool_call_id: 'call_1' }).

Tool call assertion matchers in vitest

Three matchers for tool call assertions on AIMessage in vitest: toHaveToolCalls(expected) checks that AIMessage has exactly the given tool calls in any order, toHaveToolCallCount(n) checks that AIMessage has exactly n tool calls, toContainToolCall(expected) checks that AIMessage contains at least one matching tool call and supports .not modifier. Example: expect(aiMsg).toHaveToolCalls([{ name: 'get_weather', args: { city: 'San Francisco' } }, { name: 'get_weather', args: { city: 'New York' } }]); expect(aiMsg).toHaveToolCallCount(2); expect(aiMsg).toContainToolCall({ name: 'get_weather' }); expect(aiMsg).not.toContainToolCall({ name: 'send_email' });

Tool message assertion matcher in vitest

The toHaveToolMessages() matcher takes the full message array and checks ToolMessage instances within it, in order. Example: expect(response.messages).toHaveToolMessages([{ content: '72°F and sunny in San Francisco' }, { content: '68°F and cloudy in New York' }]);

Interrupt and structured response matchers in vitest

toHaveBeenInterrupted() checks for a __interrupt__ field in a LangGraph interrupt result, optionally matching the interrupt value. Example: expect(result).toHaveBeenInterrupted(); expect(result).toHaveBeenInterrupted('confirm_action');. toHaveStructuredResponse() checks for a structuredResponse field on the result, optionally matching specific fields. Example: expect(result).toHaveStructuredResponse(); expect(result).toHaveStructuredResponse({ name: 'Alice', age: 30 });

Complete matcher reference for vitest

LangChain vitest matchers: toBeHumanMessage(expected?) - check HumanMessage, optionally match content (string) or fields (object); toBeAIMessage(expected?) - check AIMessage, optionally match content or fields; toBeSystemMessage(expected?) - check SystemMessage, optionally match content or fields; toBeToolMessage(expected?) - check ToolMessage, optionally match content or fields like tool_call_id; toHaveToolCalls(expected) - check AIMessage has exactly the given tool calls (order-independent); toHaveToolCallCount(n) - check AIMessage has exactly n tool calls; toContainToolCall(expected) - check AIMessage contains at least one matching tool call, supports .not; toHaveToolMessages(expected) - check message array contains given ToolMessage instances in order; toHaveBeenInterrupted(value?) - check result has __interrupt__, optionally match value; toHaveStructuredResponse(expected?) - check result has structuredResponse, optionally match specific fields.

Practices to reduce cost and latency in LLM integration tests

Integration tests that call LLM APIs incur real costs. Practices to keep test suites fast and affordable: use smaller models like 'gemini-3.1-flash-lite' or equivalent for tests that only need to verify tool calling and response structure; set maxTokens to cap response length and avoid expensive long completions; limit test scope to test one behavior per test, avoid end-to-end scenarios that chain many LLM calls when a single-turn test suffices; run integration tests selectively using test separation to run only in CI or before deploy, not on every file save.

Python example: use smaller model with maxTokens limit

Example Python code reducing LLM test costs: agent = create_agent('gemini-3.1-flash-lite', tools=[get_weather], model_kwargs={'max_tokens': 256}). This uses a smaller model and caps response length to 256 tokens to reduce cost and latency.

JavaScript example: use smaller model with maxTokens limit

Example JavaScript code reducing LLM test costs: const agent = createAgent({ model: 'gemini-3.1-flash-lite', tools: [getWeather], modelArgs: { maxTokens: 256 } }). This uses a smaller model and caps response length to 256 tokens to reduce cost and latency.

Record and replay HTTP calls with vcrpy for Python tests

For Python tests that run frequently in CI, use vcrpy to record HTTP interactions on the first run and replay them on subsequent runs without making real API calls. This eliminates cost and latency after initial recording. The pytest-recording plugin integrates vcrpy with pytest. vcrpy records HTTP request/response pairs into YAML cassette files for replay.

Filter sensitive data from VCR cassettes in conftest.py

In Python conftest.py, set up vcr_config fixture with scope='session' to filter sensitive information from cassettes: return dict with 'filter_headers' list containing tuples like ('authorization', 'XXXX') and ('x-api-key', 'XXXX'), and 'filter_query_parameters' list containing tuples like ('api_key', 'XXXX') and ('key', 'XXXX'). This prevents API keys from being recorded in cassette files.

Configure pytest.ini or pyproject.toml for VCR recording

In pytest.ini, add vcr marker under [pytest] markers section and set addopts to '--record-mode=once'. In pyproject.toml under [tool.pytest.ini_options], add vcr to markers list and set addopts to '--record-mode=once'. The '--record-mode=once' option records HTTP interactions on the first run and replays them on subsequent runs without making real API calls.

Decorate tests with @pytest.mark.vcr() for HTTP recording

Decorate Python integration tests with @pytest.mark.vcr() decorator to enable VCR recording and replay. On first run, vcrpy makes real network calls and generates a cassette file in tests/cassettes/. Subsequent runs replay the recorded responses from the cassette file without making real API calls.

Python example: integration test with VCR recording

Example Python test with VCR recording: @pytest.mark.vcr() decorator on test function, create agent with 'claude-sonnet-4-6', invoke with HumanMessage, assert tool_calls structure by extracting and checking that any tool call has name 'get_weather'. First run creates cassette file in tests/cassettes/, subsequent runs replay recorded responses.

Cassette files become outdated when prompts or tools change

When you modify prompts, add new tools, or change expected trajectories in Python integration tests, saved VCR cassettes become outdated and existing tests will fail. Delete the corresponding cassette files and rerun the tests to record fresh interactions with the updated code.

Voice agent definition and use cases

Voice agents are agents that can engage in natural spoken conversations with users. They combine speech recognition, natural language processing, generative AI, and text-to-speech technologies to create seamless, natural conversations. They are suited for customer support, personal assistants, hands-free interfaces, and coaching and training use cases.

Three core tasks every voice agent must handle

Every voice agent needs to handle three tasks: (1) Listen - capture audio and transcribe it, (2) Think - interpret intent, reason, plan, and (3) Speak - generate audio and stream it back to the user.

STT > Agent > TTS sandwich architecture overview

The sandwich architecture composes three distinct components: speech-to-text (STT), a text-based LangChain agent, and text-to-speech (TTS). User audio flows to STT, output flows to the LangChain agent, and agent output flows to TTS to produce audio output.

Sandwich architecture pros and cons

Sandwich architecture pros: full control over each component (swap STT/TTS providers as needed), access to latest capabilities from modern text-modality models, transparent behavior with clear boundaries between components. Cons: requires orchestrating multiple services, additional complexity in managing the pipeline, conversion from speech to text loses information (e.g., tone, emotion).

Speech-to-speech (S2S) multimodal architecture overview

Speech-to-speech uses a multimodal model that processes audio input and generates audio output natively, without intermediate text conversion.

Speech-to-speech architecture pros and cons

S2S architecture pros: simpler architecture with fewer moving parts, typically lower latency for simple interactions, direct audio processing captures tone and other nuances of speech. Cons: limited model options, greater risk of provider lock-in, features may lag behind text-modality models, less transparency in how audio is processed, reduced controllability and customization options.

Sandwich architecture latency performance

The sandwich architecture can achieve sub-700ms latency with some STT and TTS providers while maintaining control over modular components.

STT event types

STT produces two event types: stt_chunk (partial transcripts provided as the STT service processes audio) and stt_output (final, formatted transcripts that trigger agent processing).

AssemblyAI client WebSocket configuration

The AssemblyAI client connects to wss://streaming.assemblyai.com/v3/ws with sample_rate parameter (e.g., 16000) and format_turns=true. Authorization header contains the API key. The WebSocket receives JSON messages; Turn type messages with turn_is_formatted=true are final transcripts, others are partial transcripts.

Cartesia TTS client WebSocket configuration

The CartesiaTTS client connects to wss://api.cartesia.ai/tts/websocket with api_key and cartesia_version query parameters. The client sends JSON payloads containing model_id, transcript text, voice configuration (mode and id), output_format (container, encoding, sample_rate), language, and context_id. Server responds with base64-encoded audio data.

Cartesia TTS client default configuration

CartesiaTTS default parameters: voice_id="f6ff7c0c-e396-40a9-a70b-f7607edb6937", model_id="sonic-3", sample_rate=24000, encoding="pcm_s16le". Context ID is generated as timestamp-based string (ctx_{timestamp}_{counter}) to track synthesis batches.

Voice pipeline WebSocket integration

In a WebSocket endpoint, create an async generator that yields audio bytes from websocket.receive_bytes(). Pass this to the pipeline via pipeline.atransform(). Iterate over output_stream and send tts_chunk events back to the client via websocket.send_bytes().

Give your agent this brain