streaming & observability

31 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.

Custom output rendering configuration levels

Custom output rendering can be configured at three levels with the following precedence (highest to lowest): annotation queue, dataset, tracing project. When custom rendering settings are applied at multiple levels, annotation queue settings take precedence over dataset settings, which take precedence over tracing project settings.

Configure custom rendering for tracing projects

To configure custom output rendering for a tracing project: (1) Navigate to the Tracing Projects page, (2) Click on an existing tracing project or create a new one, (3) In the edit tracing project pane, scroll to the Custom Output Rendering section, (4) Toggle Enable custom output rendering, (5) Enter the webpage URL in the URL field, (6) Click Save.

Configure custom rendering for datasets

To configure custom output rendering for a dataset: (1) Navigate to your dataset in the Datasets & Experiments page, (2) Click ⋮ (three-dot menu) in the top right corner, (3) Select Custom Output Rendering, (4) Toggle Enable custom output rendering, (5) Enter the webpage URL in the URL field, (6) Click Save.

Configure custom rendering for annotation queues

To configure custom output rendering for an annotation queue: (1) Navigate to the Annotation Queues page, (2) Click on an existing annotation queue or create a new one, (3) In the annotation queue settings pane, scroll to the Custom Output Rendering section, (4) Toggle Enable custom output rendering, (5) Enter the webpage URL in the URL field, (6) Click Save or Create.

Custom renderer postMessage API format

LangSmith sends messages to custom renderers via the postMessage API with the following structure: {type: "output" | "reference", data: {...}, metadata: {inputs: {...}}}. The type field indicates whether this is an actual output ("output") or a reference output ("reference"). The data field contains the output data itself with structure varying based on the application. The metadata.inputs field contains the input data that generated this output, provided for context.

postMessage retry mechanism for custom renderers

LangSmith uses an exponential backoff retry mechanism to ensure custom renderer pages receive data even if they load slowly. Messages are sent up to 6 times with increasing delays of 100ms, 200ms, 400ms, 800ms, 1600ms, and 3200ms.

Custom renderer implementation example

This example listens for incoming postMessage events and displays them on the page: ```html <!DOCTYPE html> <html> <head> <meta charset="UTF-8" /> <title>PostMessage Echo</title> <link rel="stylesheet" href="https://unpkg.com/sakura.css/css/sakura.css" /> </head> <body> <h1>PostMessage Messages</h1> <div id="messages"></div> <script> let count = 0; window.addEventListener("message", (event) => { count++; const header = document.createElement("h3"); header.appendChild(document.createTextNode(`Message ${count}`)); const code = document.createElement("code"); code.appendChild(document.createTextNode(JSON.stringify(event.data, null, 2))); const pre = document.createElement("pre"); pre.appendChild(code); document.getElementById("messages").appendChild(header); document.getElementById("messages").appendChild(pre); }); </script> </body> </html> ``` This example shows how to listen for postMessage events and display the received data as numbered messages formatted as JSON.

Where custom rendering appears in LangSmith

When enabled, custom rendering replaces the default output view in three locations: (1) Experiment comparison view when comparing outputs across multiple experiments, (2) Run detail panes when viewing runs that are associated with a dataset, (3) Annotation queues when reviewing runs in annotation queues.

Custom output rendering use cases

Custom output rendering is useful for domain-specific formatting such as displaying medical records, legal documents, or other specialized data types in their native format, and for creating custom visualizations like charts, graphs, or diagrams from numeric or structured output data.

Custom rendering scope for datasets and annotation queues

For datasets, custom rendering applies to all runs associated with that dataset, wherever they appear including in experiments, run detail panes, or annotation queues. For annotation queues, custom rendering applies to all runs within a specific annotation queue regardless of which dataset they come from.

Agent Server /metrics endpoint with Prometheus format

In v0.2.62, a /metrics endpoint was added to expose queue worker metrics for monitoring, including Prometheus-format run statistics.

Agent Server streaming modes

The Agent Server supports multiple stream modes for different use cases: 'values' (state updates), 'updates' (node-by-node output), 'events' (all internal events including messages), 'tasks' (task-specific events), and 'checkpoints' (checkpoint creation events). Stream modes can be selected via stream_mode parameter in `/stream` and `/join-stream` endpoints. Resumable streams use event IDs in `ms-seq` format for reconnection.

Run event ID format for resumable streams

Run stream event IDs for resumable streams use the format `ms-seq` instead of previous formats. The Agent Server retains backwards compatibility with old format but recommends using new format for new code. The `/join-stream` and `/stream` APIs follow SSE spec for last-event-id parameter, returning only new messages following the provided ID (not including the event with that ID).

Metrics reporting and observability

The Agent Server provides Prometheus metrics for language usage in graphs, middleware, and authentication. Metrics reporting accurately accounts for PostgreSQL and Redis connections with consistent statistics between gRPC and Python metrics. Executor metrics for Datadog are supported. Metrics are returned even during database connection issues via fallback mechanisms.

Agent Server HTTP request metrics

In v0.2.78, HTTP request metrics were added including request count and latency histogram for enhanced monitoring capabilities.

Agent Server LG API version and request ID in metadata

In v0.2.51, LG API version and request ID were added to metadata and logs for better tracking and traceability.

Distributed tracing headers propagation

Distributed tracing links runs across services using context propagation headers. The client infers the trace context from the current run and sends it as HTTP headers. The server reads the headers and adds them to the run's config and metadata as langsmith-trace and langsmith-project configurable values. The headers used are: langsmith-trace (contains the trace's dotted order) and baggage (specifies the LangSmith project and other optional tags and metadata).

Server-side distributed tracing configuration

To accept distributed trace context, your graph must read the trace headers from the config and set the tracing context. The headers are passed through the configurable field as langsmith-trace and langsmith-project. These values, along with optional langsmith-metadata and langsmith-tags, should be passed to ls.tracing_context(parent=parent_trace, project_name=parent_project, metadata=metadata, tags=tags).

RemoteGraph distributed tracing setup

Set distributed_tracing=True when initializing RemoteGraph to automatically propagate trace headers on all requests. When RemoteGraph is called in the context of ongoing work (a parent LangGraph agent, code traced with @ls.traceable, or other instrumented code), the remote graph's execution will appear as a child of that trace.

SDK distributed tracing with manual header propagation

When using the LangGraph SDK directly, propagate trace headers manually by calling run_tree.to_headers() and passing the result as the headers parameter to client.runs.stream().

Server-side distributed tracing code example

```python import contextlib import langsmith as ls from langgraph.graph import StateGraph, MessagesState # Define your graph builder = StateGraph(MessagesState) # ... add nodes and edges ... my_graph = builder.compile() @contextlib.contextmanager async def graph(config): configurable = config.get("configurable", {}) parent_trace = configurable.get("langsmith-trace") parent_project = configurable.get("langsmith-project") # If you want to also include metadata and tags from the client metadata = configurable.get("langsmith-metadata") tags = configurable.get("langsmith-tags") with ls.tracing_context(parent=parent_trace, project_name=parent_project, metadata=metadata, tags=tags): yield my_graph ``` This example shows how to configure a graph to accept and use distributed trace context from headers passed through the config object.

RemoteGraph distributed tracing code example

```python from langgraph.graph import StateGraph from langgraph.pregel.remote import RemoteGraph remote_graph = RemoteGraph( "agent", url="<DEPLOYMENT_URL>", distributed_tracing=True, # Enable trace propagation ) def subgraph_node(query: str): # Trace context is automatically propagated return remote_graph.invoke({ "messages": [{"role": "user", "content": query}] })['messages'][-1]['content'] # The RemoteGraph is called in the context of some on going work. # This could be a parent LangGraph agent, code traced with `@ls.traceable`, # or any other instrumented code. graph = ( StateGraph(str) .add_node(subgraph_node) .add_edge("__start__", "subgraph_node") .compile() ) # The remote graph's execution will appear as a child of this trace result = graph.invoke("What's the weather in SF?") ``` This example shows how to create a RemoteGraph with distributed tracing enabled and use it as a subgraph node.

SDK distributed tracing code example

```python from langgraph_sdk import get_client import langsmith as ls client = get_client(url="<DEPLOYMENT_URL>") with ls.trace("call_remote_agent", inputs={"query": query}) as rt: headers = rt.to_headers() async for chunk in client.runs.stream( thread_id=None, assistant_id="agent", input={"messages": [{"role": "user", "content": query}]}, stream_mode="values", headers=headers, # Pass trace headers ): pass return chunk result = await call_remote_agent("What's the weather in SF?") ``` This example shows how to manually propagate trace headers when using the LangGraph SDK to call a deployed Agent Server.

Agent Server feedback event in stream

The streaming response emits a `feedback` event with the structure: event: feedback, data: {feedback_key: pre_signed_url}. Each key in the data object matches one of the values passed in feedback_keys, with each value being a URL the client can call to submit feedback.

Required fields for tool runs

Tool runs require: ls_tool_name (name of tool invoked such as 'bash' or 'computer') with tier 'always'.

Required fields for subagent runs

Subagent runs require: ls_subagent_id (stable identifier for subagent) with tier 'always' and ls_subagent_type (type or role of subagent such as 'researcher') with tier 'always'.

Global identity block metadata fields required on every run

Every run must include these identity fields: ls_agent_type (should be 'root', 'subagent', 'middleware', or 'compaction'), ls_agent_purpose (high-level purpose such as 'coding'), ls_integration (identifier of the integration), ls_agent_runtime (human-readable runtime name), thread_id (stable identifier for conversation thread), and ls_trace_schema_version (currently 'coding-agent-v1').

Availability tiers for metadata fields

Fields in the coding agent metadata schema are marked with three availability tiers: 'always' (must be present on every run), 'where_known' (required whenever the runtime can expose the value, omit only when runtime cannot provide it), and 'contextual' (optional metadata, omit when not applicable).

Required fields for all run types

In addition to global identity block fields, all run types require: ls_agent_version (version string for agent runtime) with tier 'where_known', git_branch (active Git branch) with tier 'where_known', git_commit_sha (full SHA of current commit) with tier 'where_known', git_repo_url (remote URL of repository) with tier 'where_known', and working_directory (absolute path of working directory) with tier 'where_known'.

Required fields for llm runs

LLM runs require: ls_model_name (model identifier) with tier 'where_known' and ls_provider (model provider) with tier 'where_known'.

LangSmith Engine for production agent monitoring

LangSmith Engine is used for production agents to detect recurring failures in their traces, diagnose root causes, and resolve them.

Give your agent this brain