agents/middleware/fault-tolerance

6 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.

Fault tolerance middleware for production reliability

Agents in production encounter failures that rarely appear in development: rate limits, model timeouts, transient API errors. Fault tolerance middleware (ModelRetryMiddleware and ToolRetryMiddleware) handles these at the infrastructure level so your tools and business logic don't need try/catch around every call.

ModelRetryMiddleware parameters and defaults

ModelRetryMiddleware supports the following configuration parameters: max_retries (default 2) controls maximum retry attempts; backoff_factor (default 1.0) multiplies delay between retries exponentially; initial_delay (default 1.0 seconds) sets the first delay; jitter (default true) adds random ±25% variation to prevent thundering herd; retry_on accepts either specific exception types or a callable that filters which errors to retry; on_failure accepts 'error' to re-raise, 'continue' to return AIMessage with error, or a callable to format error messages.

ModelRetryMiddleware example - basic setup

from langchain.agents import create_agent from langchain.agents.middleware import ModelRetryMiddleware agent = create_agent( model="gpt-5.5", tools=[search_tool], middleware=[ModelRetryMiddleware()], ) This example shows basic usage with default settings: 2 retries and exponential backoff.

ModelRetryMiddleware example - custom exception filtering

from langchain.agents.middleware import ModelRetryMiddleware def should_retry(error: Exception) -> bool: if isinstance(error, TimeoutError): return True if hasattr(error, "status_code"): return error.status_code in (429, 503) return False retry_with_filter = ModelRetryMiddleware( max_retries=3, retry_on=should_retry, ) This shows filtering retries by specific exception type or HTTP status code.

ModelRetryMiddleware example - error message handling

from langchain.agents.middleware import ModelRetryMiddleware retry_continue = ModelRetryMiddleware( max_retries=4, on_failure="continue", ) def format_error(error: Exception) -> str: return f"Model call failed: {error}. Please try again later." retry_with_formatter = ModelRetryMiddleware( max_retries=4, on_failure=format_error, ) The first example returns AIMessage with error instead of raising. The second customizes error message formatting.

Validation of required state before step execution

In the apply_step_config middleware, after reading STEP_CONFIG[current_step]['requires'], the code validates: for key in requires: if request.state.get(key) is None: raise ValueError(...). This prevents invalid state transitions and makes dependencies explicit in the step configuration.

Give your agent this brain