ai gateway/capabilities

17 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.

AI Gateway text batch processing

AI Gateway supports experimental durable text batches through experimental_startTextBatch, experimental_getBatchStatus, and experimental_getBatchResults. Text batch support is experimental and the API may change in patch releases. See AI Gateway batch processing guide for supported models, limits, and complete lifecycle.

AI Gateway batch webhook notifications

Pass a publicly reachable HTTPS webhookUrl when starting a batch to receive terminal notification instead of polling. AI Gateway sends batch.completed, batch.failed, or batch.cancelled events. The event contains terminal status information but not batch results. Completion webhooks are an AI Gateway capability; direct Anthropic and OpenAI batch providers return an unsupported warning when this option is provided.

AI Gateway batch webhook signature verification

Verify the x-ai-gateway-signature header against raw request body before trusting webhook events. Signature header format: t=<unix seconds>,v1=<hex digest> where v1 is HMAC-SHA256 digest of "<t>.<raw body>". Use timing-safe comparison, reject stale timestamps, and verify data.jobId matches persisted batch.id. AI Gateway retries failed deliveries and expects 2xx response within 10 seconds.

AI Gateway reranking models

Create reranking models using gateway.rerankingModel('cohere/rerank-v3.5'). Use with rerank function: const { ranking } = await rerank({ model: gateway.rerankingModel('cohere/rerank-v3.5'), query: 'What is the capital of France?', documents: [...], topN: 2 });

AI Gateway realtime models support

Realtime support is experimental and the API may change in patch releases. Create realtime models using gateway.experimental_realtime('openai/gpt-realtime-2'). Gateway normalizes realtime the same way it normalizes every modality, allowing client code to work regardless of which provider backs the model.

AI Gateway realtime token generation

Realtime sessions require a short-lived Gateway client secret created on server: const token = await gateway.experimental_realtime.getToken({ model: 'openai/gpt-realtime-2', expiresAfterSeconds: 60 * 10 }); Returns WebSocket URL for the model. Do not expose Gateway API key or OIDC token to browser clients, only return the short-lived setup response.

AI Gateway realtime provider options

Gateway provider options (tags, user, byok, compliance flags) are set under providerOptions.gateway in the session configuration. Include them in the session configuration sent to Gateway, and Gateway applies them server-side. Provider options are sent in initial session update after socket opens, so connect-time options like byok and quota selection require Gateway that resolves them from that update.

AI Gateway getAvailableModels discovery

Discover available models programmatically: const availableModels = await gateway.getAvailableModels(); Returns object with models array containing id, name, description, and pricing fields (input, output, cachedInputTokens, cacheCreationInputTokens).

AI Gateway getCredits method

Check team's current credit balance: const credits = await gateway.getCredits(); Returns object with balance (number, current available credits) and total_used (number, total credits consumed by team).

AI Gateway getGenerationInfo method

Look up detailed information about a specific generation: const generation = await gateway.getGenerationInfo({ id: generationId }); Returns GatewayGenerationInfo with: id, totalCost, upstreamInferenceCost, usage, createdAt, model, isByok, providerName, streamed, finishReason, latency, generationTime, promptTokens, completionTokens, reasoningTokens, cachedTokens, cacheCreationTokens, billableWebSearchCalls.

AI Gateway generation ID capture from streamText

When streaming with AI Gateway, generation ID is injected on first content chunk: let generationId: string | undefined; for await (const part of result.stream) { if (!generationId && part.providerMetadata?.gateway?.generationId) { generationId = part.providerMetadata.gateway.generationId as string; } }

AI Gateway generation ID with onLanguageModelCallEnd

Capture generation ID for every completed model call: await generateText({ model: gateway('anthropic/claude-sonnet-4'), prompt: 'Explain quantum entanglement briefly', onLanguageModelCallEnd({ providerMetadata }) { const generationId = providerMetadata?.gateway?.generationId as string | undefined; if (generationId) { console.log(`Completed Gateway generation: ${generationId}`); } } });

AI Gateway tool usage example

const { text } = await generateText({ model: 'xai/grok-4.6', prompt: 'What is the weather like in San Francisco?', tools: { getWeather: tool({ description: 'Get the current weather for a location', inputSchema: z.object({ location: z.string().describe('The location to get weather for') }), execute: async ({ location }) => `It's sunny in ${location}` }) } });

AI Gateway provider-executed web search tool example

const result = await generateText({ model: 'openai/gpt-5.4-mini', prompt: 'What is the Vercel AI Gateway?', stopWhen: isStepCount(10), tools: { web_search: openai.tools.webSearch({}) } });

AI Gateway realtime WebSocket authentication

Gateway WebSocket route transports short-lived auth token via versioned Sec-WebSocket-Protocol subprotocol and model id via ?ai-model-id= query parameter (transporting Authorization and ai-model-id headers). Subprotocol values must fit WebSocket token grammar, and complete Sec-WebSocket-Protocol header should stay compact (under 8 KiB safe header budget).

AI Gateway batch webhook idempotency

Retries carry same x-ai-gateway-idempotency-key value (<jobId>-<status>), which receivers can use for deduplication. Reserve callback token before submitting batch, persist secret, model, and minimal batch reference before process exits. Reuse reserved token when retrying; do not generate new one. Delete reservation after definitive start failure.

AI Gateway batch webhook early delivery handling

Reserve token before starting batch so early delivery finds retryable pending record rather than unknown callback. While reservation pending, receiver should return retryable response so early delivery is sent again after setup completes.

Give your agent this brain