ai-sdk/extensibility

92 notes in this subject, read out of this brain and free to use. This is page 2 of 2.

Middleware caching example with wrapGenerate

In wrapGenerate middleware, first generate a cache key from params using JSON.stringify. Check redis.get(cacheKey) for cached results. If cached, return it with timestamp converted back to Date if present. If not cached, call doGenerate(), store the result in redis, and return it.

Middleware caching example with wrapStream

In wrapStream middleware, generate a cache key from params using JSON.stringify. Check redis.get(cacheKey) for cached results. If cached, convert timestamps in response-metadata chunks back to Date objects and return with simulateReadableStream. If not cached, collect chunks using TransformStream, store full array in redis in the flush method, and return the transformed stream.

LanguageModelMiddleware interface has two methods

LanguageModelMiddleware has two methods: wrapGenerate which is called when using generateText, and wrapStream which is called when using streamText.

wrapGenerate middleware caches responses directly

In LanguageModelMiddleware.wrapGenerate, responses from generateText can be cached directly by storing the return value from doGenerate().

Creating a caching middleware with wrapLanguageModel

To create a caching middleware, use the `wrapLanguageModel` function from the 'ai' package, passing it an object with a middleware property (the LanguageModelV4Middleware) and a model property (the language model to wrap). This wraps the model with custom behavior.

Caching middleware implementation imports

The caching middleware uses imports from: '@ai-sdk/provider' (LanguageModelV4Middleware, LanguageModelV4StreamPart, LanguageModelV4CallOptions, LanguageModelV4 types), '@ai-sdk/provider-utils' (safeParseJSON utility), and 'ai' (wrapLanguageModel and simulateReadableStream functions).

LanguageModelV4Middleware interface structure

The LanguageModelV4Middleware interface requires a specificationVersion field set to 'v4', a wrapGenerate method that accepts an object with doGenerate, params, and model properties, and a wrapStream method that accepts an object with doStream, params, and model properties.

jsonrepair fixes for common JSON issues

The jsonrepair library can automatically fix: missing closing brackets (e.g., {"name": "test" to {"name": "test"}), single quotes (e.g., {'name': 'test'} to {"name": "test"}), missing quotes around keys (e.g., {name: "test"} to {"name": "test"}), trailing commas (e.g., {"items": [1, 2, 3,]} to {"items": [1, 2, 3]}), comments in JSON, and unescaped special characters.

jsonrepair is best-effort repair

jsonrepair handles many common JSON issues but cannot fix all malformed JSON. It cannot fix semantically incorrect data that happens to be valid JSON. Even after repair, the object must still match your schema validation, and repair won't help if the model produces structurally wrong data.

Handling truncated JSON responses

For severely truncated JSON responses, consider increasing maxOutputTokens or simplifying your schema as repair approaches may not help with severely truncated content.

safeParseJSON function location

safeParseJSON is imported from '@ai-sdk/provider-utils' and is used to safely parse JSON text with error handling.

jsonrepair library installation

Install jsonrepair with: pnpm add jsonrepair

uploadSkill with registry skills interface

Use uploadSkill with api: registry.skills('anthropic') to upload skills through a provider's skills interface accessed from the registry.

Custom provider with provider options using middleware

You can override default model settings for a provider using customProvider with wrapLanguageModel and defaultSettingsMiddleware. For example, you can set OpenAI's reasoningEffort to 'high' or 'low' by wrapping the model with middleware that includes providerOptions configuration.

customProvider function for pre-configuring model settings

The customProvider function allows you to create custom providers by pre-configuring model settings, providing model name aliases, and limiting available models. It accepts a configuration object with languageModels, embeddingModels, imageModels, and optionally a fallbackProvider.

Model name aliases with customProvider

customProvider allows you to create aliases for models so you can update the model version in one place. For example, you can alias 'opus' to 'anthropic/claude-opus-4.1', 'sonnet' to 'anthropic/claude-sonnet-4.5', and 'haiku' to 'anthropic/claude-haiku-4.5'.

Limit available models with customProvider

customProvider can restrict which models are available in a system. You define only the languageModels and embeddingModels you want to expose, optionally without a fallbackProvider to enforce strict model availability.

Attach files interface to custom provider

You can attach a provider's files interface to a custom provider using customProvider with a files option. This allows you to use uploadFile through the custom provider abstraction. If no files option is set but a fallbackProvider is configured, the custom provider inherits the files interface from the fallback.

Attach skills interface to custom provider

You can attach a provider's skills interface to a custom provider using customProvider with a skills option. This allows you to use uploadSkill through the custom provider abstraction. If no skills option is set but a fallbackProvider is configured, the custom provider inherits the skills interface from the fallback.

Access language models from provider registry

Use registry.languageModel('providerId:modelId') to access language models from a provider registry. The provider ID becomes the prefix of the model ID.

Access embedding models from provider registry

Use registry.embeddingModel('providerId:modelId') to access text embedding models from a provider registry. The provider ID becomes the prefix of the model ID.

Access video models from provider registry

Use registry.videoModel('providerId:modelId') to access video models from a provider registry. The provider ID becomes the prefix of the model ID.

Access files interface from provider registry

Call registry.files('providerId') to access a provider's files interface from the registry. This is useful when you want to upload files through a provider before referencing them in model requests.

Access skills interface from provider registry

Call registry.skills('providerId') to access a provider's skills interface from the registry.

Comprehensive provider management example

A complete example showing provider registry with: gateway passthrough with namespace prefix, full providers with namespace prefixes (xai), OpenAI-compatible provider with custom API key and base URL (custom), model name aliases (anthropic > fast, anthropic > writing, anthropic > reasoning), pre-configured model settings with middleware (anthropic > reasoning with thinking enabled and budgetTokens: 32000), provider-specific option validation using AnthropicLanguageModelOptions, fallback provider (anthropic), limited provider without fallback (groq with specific models), and custom separator (' > ').

Global provider configuration with AI_SDK_DEFAULT_PROVIDER

AI SDK 5 includes a global provider feature that allows specifying a model using just a plain model ID string. By default, the global provider is set to the Vercel AI Gateway. You can customize it by setting globalThis.AI_SDK_DEFAULT_PROVIDER to a provider instance (e.g., openai) during startup, which allows using model: 'gpt-5.1' without a provider prefix in streamText and other functions.

Custom OpenAI-compatible provider setup

Use createOpenAICompatible to set up an OpenAI-compatible provider with custom configuration. It accepts name, apiKey, and baseURL parameters, allowing you to connect to custom API endpoints with different credentials.

Provider management central configuration file

Provider management best practice involves setting up a central file (e.g., registry.ts) that contains all providers and models you want to use, with pre-configured settings, model aliases, provider limitations, and custom separators.

Combine custom providers, registry, and middleware

You can mix and match custom providers, the provider registry, and middleware in your application for flexible provider management and model configuration.

experimental_generateVideo with provider registry video model

Use experimental_generateVideo with model: registry.videoModel('fal:luma-dream-machine/ray-2') to generate videos using a video model accessed from a provider registry.

uploadFile with registry files interface

Use uploadFile with api: registry.files('openai') to upload files through a provider's files interface accessed from the registry, which returns a providerReference that can be used in model requests.

createProviderRegistry custom separator configuration

By default, createProviderRegistry uses ':' as the separator between provider and model IDs (e.g., 'openai:gpt-5.1'). You can customize this separator by passing a second argument with a separator option. For example: const registry = createProviderRegistry({ anthropic, openai }, { separator: ' > ' }); then access models like registry.languageModel('anthropic > claude-3-opus-20240229').

Give your agent this brain