AIMessageChunk for streaming
During streaming, AIMessageChunk objects are received. Chunks can be combined into full message by concatenating: full_message = chunk if full_message is None else full_message + chunk.
LangChain · Agents · all subjects
streaming
10 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.
stream() method for chat models
stream() invokes the model and streams the output as it is generated in real-time. It returns an iterator that yields AIMessageChunk objects, each containing a portion of the output text. Multiple chunks can be accumulated via summation to construct the full AIMessage.
astream_events() method for streaming semantic events in Python
LangChain chat models can stream semantic events using astream_events(). This simplifies filtering based on event types and other metadata, and aggregates the full message in the background. Event types include 'on_chat_model_start', 'on_chat_model_stream', and 'on_chat_model_end'.
streamEvents() method for streaming semantic events in JavaScript
LangChain chat models can stream semantic events using streamEvents(). This simplifies filtering based on event types and other metadata, and aggregates the full message in the background. Event types include 'on_chat_model_start', 'on_chat_model_stream', and 'on_chat_model_end'.
Auto-streaming for chat models
LangChain simplifies streaming by automatically enabling streaming mode in certain cases, even when not explicitly calling streaming methods. For example, in LangGraph agents, calling model.invoke() within nodes will automatically delegate to streaming if running in streaming mode. This triggers on_llm_new_token events in LangChain's callback system.
Stream reasoning output in Python
When streaming from a model with reasoning capabilities, reasoning steps can be accessed by checking content blocks for type 'reasoning'. The stream() method yields chunks with content_blocks containing reasoning elements.
AIMessageChunk for streaming responses
When using stream(), the model returns multiple AIMessageChunk objects rather than a single AIMessage. Each chunk contains a portion of output text. Chunks are designed to be gathered into a full message via summation (concatenation).
Streaming accumulation in Python
In Python, AIMessageChunk objects can be accumulated by summing them: full = chunk if full is None else full + chunk. This progressively builds the complete message as chunks arrive.
Streaming accumulation in JavaScript
In JavaScript, AIMessageChunk objects can be accumulated using concat(): full = full ? full.concat(chunk) : chunk. This progressively builds the complete message as chunks arrive.
Streaming requires compatible processing
Streaming only works if all steps in the program know how to process a stream of chunks. An application that isn't streaming-capable would be one that needs to store the entire output in memory before processing.