ai gateway/overview

27 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.

AI Gateway is default global provider for AI SDK

The Vercel AI Gateway is the default global provider for the AI SDK. This means you can access models using simple string references like 'anthropic/claude-sonnet-4.5' without explicitly importing a provider instance.

AI Gateway language models creation

Create language models using a provider instance with model ID in format 'creator/model-name'. AI Gateway language models can be used in generateText and streamText functions and support structured data generation with Output.

AI Gateway reranking models support

You can create reranking models using the rerankingModel method on the provider instance. Example: gateway.rerankingModel('cohere/rerank-v3.5'). Use with the rerank function to improve search results in retrieval-augmented generation (RAG) pipelines.

AI Gateway experimental realtime models

Create realtime models for bidirectional audio/text WebSocket sessions using the experimental_realtime method on the provider instance. Format: gateway.experimental_realtime('openai/gpt-realtime-2'). Realtime support is experimental and the API may change in patch releases. The Gateway normalizes realtime the same way it normalizes every other modality.

AI Gateway realtime authentication with short-lived tokens

Realtime sessions require a short-lived Gateway client secret created on your server using gateway.experimental_realtime.getToken(). This method uses your Gateway credential (apiKey, AI_GATEWAY_API_KEY, or Vercel OIDC token) to mint a vcst_ client secret and returns the WebSocket URL for the model. Do not expose your Gateway API key or OIDC token to browser clients.

AI Gateway realtime getToken method signature

Use gateway.experimental_realtime.getToken() with parameters: model (string, required, format 'creator/model-name'), expiresAfterSeconds (number, how long the token is valid).

AI Gateway realtime WebSocket transport details

The Gateway WebSocket route transports the short-lived auth token via the versioned Sec-WebSocket-Protocol subprotocol and the model id via the ?ai-model-id= query parameter. These mirror the Authorization and ai-model-id headers used in HTTP routes. Subprotocol values must fit the WebSocket token grammar and the complete Sec-WebSocket-Protocol header should stay compact (under 8 KiB safe header budget).

AI Gateway realtime provider options

Gateway provider options (tags, user, byok, compliance flags) are set under providerOptions.gateway in the session configuration for realtime. Provider options are sent in the initial session update after the socket opens. Connect-time options such as byok and quota selection require a Gateway that resolves them from that update. Routing knobs like order/only do not apply to realtime where the Gateway selects the WebSocket-capable provider.

AI Gateway getAvailableModels method

Use gateway.getAvailableModels() to discover available models programmatically. Returns an object with a models array containing model objects with: id (format 'creator/model-name'), name, description (optional), and pricing object with input, output, cachedInputTokens (optional), and cacheCreationInputTokens (optional) fields.

AI Gateway getCredits method

Use gateway.getCredits() to check your team's current credit balance and usage. Returns an object with: balance (number, your team's current available credit balance) and total_used (number, total credits consumed by your team). Requires an authenticated API key or OIDC token.

AI Gateway generation lookup with getGenerationInfo

Use gateway.getGenerationInfo({ id: generationId }) to look up detailed information about a specific generation. Generation IDs are available in providerMetadata.gateway.generationId on both generateText and streamText responses. When streaming, the generation ID is injected on the first content chunk, allowing you to capture it early without waiting for completion.

AI Gateway GatewayGenerationInfo response fields

gateway.getGenerationInfo() returns a GatewayGenerationInfo object with fields: id (string), totalCost (number, USD), upstreamInferenceCost (number, USD for BYOK), usage (number, USD, same as totalCost), createdAt (string, ISO 8601), model (string), isByok (boolean), providerName (string), streamed (boolean), finishReason (string), latency (number, milliseconds to first token), generationTime (number, total generation milliseconds), promptTokens (number), completionTokens (number), reasoningTokens (number if applicable), cachedTokens (number if applicable), cacheCreationTokens (number), billableWebSearchCalls (number).

AI Gateway custom reporting with user and tags

Track usage per end-user and categorize requests with tags by setting providerOptions.gateway.user (string for end-user ID) and providerOptions.gateway.tags (string[] for categorization). This allows viewing usage and costs broken down by end-user and filtering/analyzing spending by feature or use case.

AI Gateway getSpendReport method

Use gateway.getSpendReport() to query usage data programmatically. This method is only available for Vercel Pro and Enterprise plans. Required parameters: startDate (string, 'YYYY-MM-DD' format, inclusive), endDate (string, 'YYYY-MM-DD' format, inclusive). Optional parameters: groupBy (string: 'day' default, 'user', 'model', 'tag', 'provider', or 'credential_type'), datePart (string: 'day' or 'hour' when groupBy is 'day'), userId (string), model (string), provider (string), credentialType (string: 'byok' or 'system'), tags (string[]).

AI Gateway spend report response structure

gateway.getSpendReport() returns an object with results array. Each row contains a grouping field (matching your groupBy choice) and metrics: totalCost (number, USD), marketCost (number, USD), inputTokens (number), outputTokens (number), cachedInputTokens (number), cacheCreationInputTokens (number), reasoningTokens (number), requestCount (number).

AI Gateway example: basic text generation

Example showing basic text generation with AI Gateway: import { generateText } from 'ai'; const { text } = await generateText({ model: 'anthropic/claude-sonnet-4.6', prompt: 'Write a haiku about programming', }); console.log(text);

AI Gateway example: streaming text

Example showing streaming with AI Gateway: import { streamText } from 'ai'; const { textStream } = await streamText({ model: 'openai/gpt-5.4', prompt: 'Explain the benefits of serverless architecture', }); for await (const textPart of textStream) { process.stdout.write(textPart); }

AI Gateway example: tool usage

Example showing tool usage with AI Gateway: import { generateText, tool } from 'ai'; import { z } from 'zod'; const { text } = await generateText({ model: 'xai/grok-4.5', prompt: 'What is the weather like in San Francisco?', tools: { getWeather: tool({ description: 'Get the current weather for a location', inputSchema: z.object({ location: z.string().describe('The location to get weather for'), }), execute: async ({ location }) => { return `It's sunny in ${location}`; }, }), }, });

AI Gateway example: model discovery with pricing

Example showing dynamic model discovery: import { gateway, generateText } from 'ai'; const availableModels = await gateway.getAvailableModels(); availableModels.models.forEach(model => { console.log(`${model.id}: ${model.name}`); if (model.description) console.log(` Description: ${model.description}`); if (model.pricing) { console.log(` Input: $${model.pricing.input}/token`); console.log(` Output: $${model.pricing.output}/token`); if (model.pricing.cachedInputTokens) console.log(` Cached input (read): $${model.pricing.cachedInputTokens}/token`); if (model.pricing.cacheCreationInputTokens) console.log(` Cache creation (write): $${model.pricing.cacheCreationInputTokens}/token`); } }); const { text } = await generateText({ model: availableModels.models[0].id, prompt: 'Hello world', });

AI Gateway example: generation lookup

Example showing generation lookup: import { gateway, generateText } from 'ai'; const result = await generateText({ model: gateway('anthropic/claude-sonnet-4'), prompt: 'Explain quantum entanglement briefly', }); const generationId = result.providerMetadata?.gateway?.generationId; const generation = await gateway.getGenerationInfo({ id: generationId }); console.log(`Model: ${generation.model}`); console.log(`Cost: $${generation.totalCost.toFixed(6)}`); console.log(`Latency: ${generation.latency}ms`); console.log(`Prompt tokens: ${generation.promptTokens}`); console.log(`Completion tokens: ${generation.completionTokens}`);

AI Gateway example: spend reporting by tags

Example showing spend reporting filtered by tags: import type { GatewayProviderOptions } from '@ai-sdk/gateway'; import { gateway, streamText } from 'ai'; // 1. Make requests with tags const result = streamText({ model: gateway('anthropic/claude-haiku-4.5'), prompt: 'Summarize this quarter's results', providerOptions: { gateway: { tags: ['team:finance', 'feature:summaries'], } satisfies GatewayProviderOptions, }, }); // 2. Later, query spend filtered by those tags const report = await gateway.getSpendReport({ startDate: '2026-03-01', endDate: '2026-03-31', groupBy: 'tag', tags: ['team:finance'], }); for (const row of report.results) { console.log(`${row.tag}: $${row.totalCost.toFixed(4)} (${row.requestCount} requests)`); }

AI Gateway features overview

The AI Gateway provides the following features: Access models from multiple providers without installing additional provider modules/dependencies; Use the same code structure across different AI providers; Switch between models and providers easily; Automatic authentication when deployed on Vercel; View pricing information across providers; Observability for AI model usage through the Vercel dashboard.

AI Gateway example: capture generation ID during streaming

Example showing generation ID capture from stream: import { gateway, streamText } from 'ai'; const result = streamText({ model: gateway('anthropic/claude-sonnet-4'), prompt: 'Explain quantum entanglement briefly', }); let generationId: string | undefined; for await (const part of result.stream) { if (!generationId && part.providerMetadata?.gateway?.generationId) { generationId = part.providerMetadata.gateway.generationId as string; console.log(`Generation ID (early): ${generationId}`); } } if (generationId) { const generation = await gateway.getGenerationInfo({ id: generationId }); console.log(`Cost: $${generation.totalCost.toFixed(6)}`); console.log(`Finish reason: ${generation.finishReason}`); }

AI Gateway example: capture generation ID with onLanguageModelCallEnd

Example capturing generation ID with callback: import { gateway, generateText } from 'ai'; await generateText({ model: gateway('anthropic/claude-sonnet-4'), prompt: 'Explain quantum entanglement briefly', onLanguageModelCallEnd({ providerMetadata }) { const generationId = providerMetadata?.gateway?.generationId as string | undefined; if (generationId) { console.log(`Completed Gateway generation: ${generationId}`); } }, });

AI Gateway realtime example: server-side token generation

Example showing server-side realtime token generation: import { gateway } from 'ai'; export async function POST() { const token = await gateway.experimental_realtime.getToken({ model: 'openai/gpt-realtime-2', expiresAfterSeconds: 60 * 10, }); return Response.json(token); }

AI Gateway realtime example: client-side usage

Example showing client-side realtime usage: 'use client'; import { experimental_useRealtime } from '@ai-sdk/react'; import { gateway } from 'ai'; export default function RealtimePage() { const realtime = experimental_useRealtime({ model: gateway.experimental_realtime('openai/gpt-realtime-2'), api: { token: '/api/realtime/setup', }, }); // ... }

AI Gateway realtime provider options example

Example showing realtime provider options: import type { GatewayProviderOptions } from '@ai-sdk/gateway'; const gatewayOptions: GatewayProviderOptions = { tags: ['cooking-coach', 'v2'], user: 'user-123', }; const sessionConfig = { instructions: 'You are a concise voice assistant.', providerOptions: { gateway: gatewayOptions }, };

Give your agent this brain