memory/short-term

18 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.

Short-term memory in entrypoint with checkpointer

When an entrypoint is defined with a checkpointer, it stores information between successive invocations on the same thread id in checkpoints. This allows accessing the state from the previous invocation using the 'previous' parameter (Python) or getPreviousState function (JavaScript). By default, the previous parameter is the return value of the previous invocation.

entrypoint.final for decoupling saved state from return value

@entrypoint.final is a special primitive that can be returned from an entrypoint and allows decoupling the value that is saved in the checkpoint from the return value of the entrypoint. The type annotation is entrypoint.final[return_type, save_type]. The first value is the return value of the entrypoint, and the second value is the value that will be saved in the checkpoint. Example: return entrypoint.final(value=previous, save=2 * number) returns previous to caller but saves 2 * number to checkpoint for use in next invocation.

Short-term memory example with previous parameter

Example showing how to access the previous invocation state using the previous parameter: ```python @entrypoint(checkpointer=checkpointer) def my_workflow(number: int, *, previous: Any = None) -> int: previous = previous or 0 return number + previous config = { "configurable": { "thread_id": "some_thread_id" } } my_workflow.invoke(1, config) # 1 (previous was None) my_workflow.invoke(2, config) # 3 (previous was 1 from the previous invocation) ```

entrypoint.final example for decoupling return and save values

Example of using entrypoint.final to decouple the return value from the saved checkpoint value: ```python @entrypoint(checkpointer=checkpointer) def my_workflow(number: int, *, previous: Any = None) -> entrypoint.final[int, int]: previous = previous or 0 # This will return the previous value to the caller, saving # 2 * number to the checkpoint, which will be used in the next invocation # for the `previous` parameter. return entrypoint.final(value=previous, save=2 * number) config = { "configurable": { "thread_id": "1" } } my_workflow.invoke(3, config) # 0 (previous was None) my_workflow.invoke(1, config) # 6 (previous was 3 * 2 from the previous invocation) ```

Short-term memory: persisting data across invocations

Short-term memory in Functional API allows storing information across different invocations of the same thread_id using a checkpointer. The 'previous' parameter receives the saved value from the last invocation, enabling state to be carried forward.

add_messages utility for conversation management

Use add_messages() to merge message lists across invocations. When building conversational workflows, combine previous messages with new inputs using add_messages(previous, inputs) to maintain full conversation history.

Example: chatbot with conversation history

Workflow takes list[BaseMessage] inputs, previous parameter has prior messages. Uses add_messages(previous, inputs) to build full history, calls call_model task, returns entrypoint.final(value=response, save=add_messages(inputs, response)). Demonstrates multi-turn conversation with persistent memory.

Trim messages to manage LLM context window

Use the trimMessages (JavaScript) or trim_messages (Python) utility to truncate message history before calling an LLM. Specify strategy ('last' to keep most recent), maxTokens (token limit to maintain), startOn ('human' to begin trimming at first human message), and endOn (array of message types marking trim boundary, e.g., ['human', 'tool']). This prevents exceeding the LLM's maximum context window when managing long conversations.

Short-term memory enables multi-turn conversations

Short-term memory (thread-level persistence) allows agents to track and maintain context across multiple turns in a conversation. To add short-term memory, compile a StateGraph with a checkpointer and pass a thread_id in the configurable parameters when invoking the graph.

Trim messages example with token counter

Example: trimMessages(state.messages, { strategy: 'last', maxTokens: 128, startOn: 'human', endOn: ['human', 'tool'], tokenCounter: model }). This keeps the last messages up to 128 tokens, starting the trimming boundary at the first human message, and ending at human or tool messages. In Python, use trim_messages(state['messages'], strategy='last', token_counter=count_tokens_approximately, max_tokens=128, start_on='human', end_on=('human', 'tool')).

Delete specific messages example

Example in Python: Remove earliest two messages with return {'messages': [RemoveMessage(id=m.id) for m in messages[:2]]}. Example in JavaScript: Remove earliest two messages with return { messages: messages.slice(0, 2).map(m => new RemoveMessage({ id: m.id })) }.

Summarize conversation node implementation

In summarizeConversation node: (1) Get existing summary from state or empty string; (2) Create summary prompt - if summary exists, prompt to extend it; if not, create initial summary; (3) Add prompt as HumanMessage to message history; (4) Invoke model on all messages; (5) Delete all messages except last 2 using RemoveMessage; (6) Return updated summary and modified messages list.

RemoveMessage and REMOVE_ALL_MESSAGES for deleting messages from state

Delete messages from graph state using RemoveMessage class. Pass the message id to RemoveMessage(id=m.id) to remove specific messages. To remove all messages, use REMOVE_ALL_MESSAGES constant: RemoveMessage(id=REMOVE_ALL_MESSAGES). RemoveMessage only works with state keys that use the add_messages reducer, such as MessagesState. When deleting messages, ensure the resulting history is valid for your LLM provider (e.g., starts with user message, tool calls followed by tool results).

Summarize conversation to manage long message history

Summarize message history using a chat model to preserve information when trimming or deleting messages. Add a summary state key alongside messages. When generating a summary, check if one exists; if yes, extend it with new messages; if no, create initial summary. Delete all but the last 2 messages after summarizing to keep recent context. This approach prevents information loss from culling the message queue in long conversations.

Short-term memory management strategies for long conversations

Common solutions to prevent exceeding LLM context window in long conversations: (1) Trim messages - remove first or last N messages before calling LLM using token counting; (2) Delete messages - permanently remove specific messages from state using RemoveMessage; (3) Summarize messages - summarize earlier message history and replace with summary; (4) Manage checkpoints - store and retrieve message history across multiple invocations; (5) Custom strategies like message filtering. These maintain conversation context without exceeding token limits.

add_messages utility function

Use add_messages() to append messages to a list. This utility is imported from langgraph.graph and handles message accumulation properly in agent workflows.

add_messages reducer for message handling

LangGraph includes a built-in add_messages reducer that handles special considerations for updating lists of messages: it allows updating existing messages in the state and accepts short-hands for message formats such as OpenAI format. MessagesState is a prebuilt state schema in LangGraph that includes add_messages as a reducer for the messages field.

MessagesValue in JavaScript/TypeScript

MessagesValue is a built-in type in LangGraph's JavaScript/TypeScript implementation that handles message updates automatically with a built-in reducer. It allows updating existing messages and accepts short-hands for message formats. MessagesValue replaces the Python add_messages pattern and is the recommended way to handle messages in TypeScript/JavaScript.

Give your agent this brain