custom-providers/capabilities

30 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.

Language Model Specification V4 overview

The Language Model Specification V4 is a standardized specification that provides a unified abstraction layer across all AI providers. It creates a consistent interface ensuring developers can interact with any provider using the same patterns and methods, enabling support for emerging LLM capabilities while keeping application code provider-agnostic and future-ready.

V4 specification three main interfaces

The V4 specification defines three main interfaces: ProviderV4 (top-level factory for different model types), LanguageModelV4 (primary interface for text generation models), and EmbeddingModelV4 and ImageModelV4 (interfaces for embeddings and image generation).

ProviderV4 interface definition

interface ProviderV4 { languageModel(modelId: string): LanguageModelV4; .embeddingModel(modelId: string): EmbeddingModelV4<string>; imageModel(modelId: string): ImageModelV4; }

LanguageModelV4 interface definition

interface LanguageModelV4 { specificationVersion: 'V4'; provider: string; modelId: string; supportedUrls: Record<string, RegExp[]>; doGenerate(options: LanguageModelV4CallOptions): Promise<GenerateResult>; doStream(options: LanguageModelV4CallOptions): Promise<StreamResult>; }

LanguageModelV4 key aspects

specificationVersion must be 'V4'. supportedUrls declares which URLs (for file parts) the provider can handle natively. doGenerate and doStream are methods for non-streaming and streaming generation respectively.

LanguageModelV4Content vs LanguageModelV4Prompt distinction

LanguageModelV4Content specifies what the models generate. LanguageModelV4Prompt specifies what you send to the model.

LanguageModelV4Text content type

type LanguageModelV4Text = { type: 'text'; text: string; }; Used for standard model responses, system messages, and any plain text output.

LanguageModelV4ToolCall content type

type LanguageModelV4ToolCall = { type: 'tool-call'; toolCallType: 'function'; toolCallId: string; toolName: string; args: string; }; The toolCallId is crucial for correlating tool results back to their calls, especially in streaming scenarios.

LanguageModelV4File content type

type LanguageModelV4File = { type: 'file'; mediaType: string; // IANA media type (e.g., 'image/png', 'audio/mp3') data: string | Uint8Array; // Generated file data as base64 encoded strings or binary data }; Enables models to generate images, audio, documents, and other file types directly.

LanguageModelV4Reasoning content type

type LanguageModelV4Reasoning = { type: 'reasoning'; text: string; providerMetadata?: SharedV4ProviderMetadata; }; Provides dedicated support for chain-of-thought reasoning, essential for models like OpenAI's o1. Reasoning content is tracked separately from regular text, allowing for proper token accounting and UI presentation.

LanguageModelV4Source content type

type LanguageModelV4Source = { type: 'source'; sourceType: 'url'; id: string; url: string; title?: string; providerMetadata?: SharedV4ProviderMetadata; };

System message role specification

System role supports model instructions with text only content. Format: { role: 'system', content: string }

User message role specification

User role supports human inputs with text and files. Format: { role: 'user', content: Array<LanguageModelV4TextPart | LanguageModelV4FilePart> }

Assistant message role specification

Assistant role supports model outputs with full content type support. Format: { role: 'assistant', content: Array<LanguageModelV4TextPart | LanguageModelV4FilePart | LanguageModelV4ReasoningPart | LanguageModelV4ToolCallPart> }

Tool message role specification

Tool role contains results from tool executions. Format: { role: 'tool', content: Array<LanguageModelV4ToolResultPart> }

LanguageModelV4TextPart interface

interface LanguageModelV4TextPart { type: 'text'; text: string; providerOptions?: SharedV4ProviderOptions; } The most basic prompt part, containing plain text content.

LanguageModelV4ReasoningPart interface

interface LanguageModelV4ReasoningPart { type: 'reasoning'; text: string; providerOptions?: SharedV4ProviderOptions; } Used in assistant messages to capture the model's reasoning process.

LanguageModelV4FilePart interface

interface LanguageModelV4FilePart { type: 'file'; filename?: string; data: LanguageModelV4DataContent; mediaType: string; providerOptions?: SharedV4ProviderOptions; } Enables multimodal inputs by including files in prompts. The data field offers flexibility with Uint8Array (direct binary data), string (base64-encoded data), or URL (reference to external content if supported by provider via supportedUrls).

LanguageModelV4ToolCallPart interface

interface LanguageModelV4ToolCallPart { type: 'tool-call'; toolCallId: string; toolName: string; args: unknown; providerOptions?: SharedV4ProviderOptions; } Represents tool calls made by the assistant.

LanguageModelV4ToolResultPart interface

interface LanguageModelV4ToolResultPart { type: 'tool-result'; toolCallId: string; toolName: string; result: unknown; isError?: boolean; content?: Array<{ type: 'text' | 'image'; text?: string; data?: string; // base64 encoded image data mediaType?: string; }>; providerOptions?: SharedV4ProviderOptions; } Contains the results of executed tool calls. The optional content field enables rich tool results including images.

Stream lifecycle events

The streaming system uses typed events for different stages: stream-start (initial event with warnings about unsupported features), response-metadata (model information and response headers), finish (final event with usage statistics and finish reason), and error (can occur at any point).

Stream content events

All content types (text, file, reasoning, source, tool-call) stream directly. Additionally: tool-call-delta for incremental updates for tool call arguments, and reasoning-part-finish as explicit marker for reasoning section completion.

Example stream sequence

{ type: 'stream-start', warnings: [] } { type: 'text', text: 'Hello' } { type: 'text', text: ' world' } { type: 'tool-call', toolCallId: '1', toolName: 'search', args: {...} } { type: 'response-metadata', modelId: 'gpt-4.1', ... } { type: 'finish', usage: { inputTokens: 10, outputTokens: 20 }, finishReason: 'stop' }

LanguageModelV4Usage interface

type LanguageModelV4Usage = { inputTokens: number | undefined; outputTokens: number | undefined; totalTokens: number | undefined; reasoningTokens?: number | undefined; cachedInputTokens?: number | undefined; };

LanguageModelV4FunctionTool type

type LanguageModelV4FunctionTool = { type: 'function'; name: string; description?: string; inputSchema: JSONSchema7; // Full JSON Schema support }; Standard user-defined functions with JSON Schema validation.

LanguageModelV4ProviderClientDefinedTool type

export type LanguageModelV4ProviderClientDefinedTool = { type: 'provider-defined-client'; id: string; // e.g., 'anthropic.computer-use' name: string; // Human-readable name args: Record<string, unknown>; }; Native provider capabilities exposed as tools.

Tool choice control options

toolChoice can be controlled via: 'auto' | 'none' | 'required' | { type: 'tool', toolName: string }

Native URL support declaration

Providers can declare URLs they can access directly via: supportedUrls: { 'image/*': [/^https:\/\/cdn\.example\.com\/.*/], 'application/pdf': [/^https:\/\/docs\.example\.com\/.*/], 'audio/*': [/^https:\/\/media\.example\.com\/.*\/] } The AI SDK checks these patterns before downloading any URL-based content.

Error handling in streaming

Streaming can emit errors at any point via { type: 'error', error: unknown }. Warnings are non-fatal issues reported in stream-start and response objects.

Finish reasons in V4 specification

Clear indication of why generation stopped: 'stop' (natural completion), 'length' (hit max tokens), 'content-filter' (safety filtering), 'tool-calls' (stopped to execute tools), 'error' (generation failed), 'other' (provider-specific reasons).

Give your agent this brain