ai-sdk/middleware

9 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.

Local caching middleware purpose and use cases

A caching middleware stores AI API responses locally to reuse them when identical inputs are provided, reducing costs and improving development speed. It is particularly useful when iterating on UI/UX (where repeated styling changes don't require new AI generations) and when working on evals (where the same prompts need to be tested repeatedly without generating new responses).

Cache key generation for middleware

Cache keys are generated by JSON stringifying an object containing: the cleaned prompt, a _function field indicating 'generate' or 'stream', and the model's modelId. Tool call IDs in the prompt are normalized to 'cached' and timestamps are standardized for consistent cache key generation.

Caching middleware transforms tool calls and timestamps

The middleware normalizes tool calls in cached responses by setting toolCallId to 'cached' and standardizing tool result formatting. It also handles timestamp conversion, converting stored ISO string timestamps back to Date objects for consistency with fresh responses.

Cache file storage location and structure

Cache is stored in a JSON file at `.cache/ai-cache.json` relative to the current working directory. The cache is a flat JSON object where keys are stringified cache keys and values are the cached response data. The cache directory is created automatically if it does not exist.

Using cached() function with models

To use the caching middleware, wrap a model with the `cached()` function before passing it to `streamText()` or other AI SDK methods. For example: `cached(openai('gpt-4o'))` wraps an OpenAI model with caching behavior.

Caching middleware development-only use case

The caching middleware approach is intended for local development only, not for production environments. Cache invalidation requires manually deleting the cache file to force fresh responses.

Tool call caching behavior with stopWhen

When using `stopWhen`, caching occurs at the individual language model response level only, not across entire execution flows. This means the model's generation is cached but tool calls are not and will execute on each generation.

Caching middleware example with streamText

Example integration: import { openai } from '@ai-sdk/openai'; import { streamText } from 'ai'; import { cached } from '../middleware/your-cache-middleware'; const result = streamText({ model: cached(openai('gpt-4o')), maxOutputTokens: 512, temperature: 0.3, maxRetries: 5, prompt: 'Invent a new holiday and describe its traditions.' }); for await (const textPart of result.textStream) { process.stdout.write(textPart); }

wrapImageModel wraps image models with middleware

The wrapImageModel function provides a way to enhance the behavior of image models by wrapping them with middleware. This allows you to apply custom transformations and logic to image generation requests before they reach the underlying model.

Give your agent this brain