createUIMessageStreamResponse returns streaming chat response
The createUIMessageStreamResponse function wraps a UI message stream and returns it as an HTTP response suitable for client-side consumption via useChat.
AI SDK · Cookbook · all subjects
streaming/backend
184 notes in this subject, read out of this brain and free to use. This is page 2 of 4.
isStepCount tool call limit constraint
The isStepCount function can be used with the stopWhen parameter in streamText to limit the number of tool calls before the model stops. For example, isStepCount(5) stops after 5 tool calls.
Complete server-side weather chat API implementation
This example shows a server-side chat API route that defines three tools: getWeatherInformation (server-side with execute method returning weather data), askForConfirmation (client-side with user interaction), and getLocation (client-side auto-executed). It uses streamText with convertToModelMessages to prepare messages, toUIMessageStream to convert the stream, and createUIMessageStreamResponse to return the response.
convertToModelMessages transforms UI messages for the language model
The convertToModelMessages function converts UIMessage objects to the format expected by language models. It is used before passing messages to streamText.
Custom stream format with StreamText
Use the stream property on StreamTextResult to access raw model events and transform them into a custom streaming format. This gives you control over stream chunk format, how steps and tool calls are structured, and manual parsing on the client, without prescribing how you consume the stream.
StreamEvent type definition for custom format
Define a custom StreamEvent union type with three event types: text (contains type and text fields), tool-call (contains type, toolName, and input fields), and tool-result (contains type, toolName, and result fields).
Transform stream events to Server-Sent Events
Use TransformStream to convert model events to Server-Sent Event format. Encode each custom event as a JSON string with 'data: ' prefix and double newlines using TextEncoder, which allows the client to parse newline-delimited JSON.
streamText configuration for custom streaming
Call streamText with parameters: prompt (the user input), model (the AI model), tools (array of available tools), and stopWhen (optional condition like isStepCount(5) to limit execution). The result.stream property provides the underlying stream to pipe through custom transformations.
Server response headers for Server-Sent Events
Return the Response with header 'Content-Type': 'text/event-stream' to signal to the client that the response is a stream of Server-Sent Events.
Combine custom stream format with manual agent loop
For complete control over both the streaming format and the execution loop, combine the custom stream format pattern with a manual agent loop pattern, which allows you to handle agent decision-making and tool invocation manually instead of relying on streamText's built-in loop.
Custom streaming example: server route
```tsx import { tools } from '@/ai/tools'; import { isStepCount, streamText } from 'ai'; export type StreamEvent = | { type: 'text'; text: string } | { type: 'tool-call'; toolName: string; input: unknown } | { type: 'tool-result'; toolName: string; result: unknown }; const encoder = new TextEncoder(); function formatEvent(event: StreamEvent): Uint8Array { return encoder.encode('data: ' + JSON.stringify(event) + '\n\n'); } export async function POST(request: Request) { const { prompt } = await request.json(); const result = streamText({ prompt, model: __MODEL__, tools, stopWhen: isStepCount(5), }); const transformStream = new TransformStream({ transform(chunk, controller) { switch (chunk.type) { case 'text-delta': controller.enqueue(formatEvent({ type: 'text', text: chunk.text })); break; case 'tool-call': controller.enqueue( formatEvent({ type: 'tool-call', toolName: chunk.toolName, input: chunk.input, }), ); break; case 'tool-result': controller.enqueue( formatEvent({ type: 'tool-result', toolName: chunk.toolName, result: chunk.output, }), ); break; } }, }); return new Response(result.stream.pipeThrough(transformStream), { headers: { 'Content-Type': 'text/event-stream' }, }); } ``` This example shows a Next.js route handler that streams custom-formatted events from the AI model to the client.
Use textStream.getReader() for manual control
For more control over streaming, call result.textStream.getReader() to get a reader object. Then repeatedly call await reader.read() which returns an object with done (boolean) and value (string chunk) properties. Continue reading until done is true.
Why stream text instead of waiting
Streaming text generation is useful when generation takes a long time because it displays generated text to clients in real-time as it is produced, rather than requiring users to wait for the entire result before showing anything.
Write streamed text to stdout
When reading from a stream manually using getReader(), write chunks to stdout with process.stdout.write(value) to display the streamed text in real-time.
streamText for text streaming
The streamText function from the 'ai' SDK streams text generation from language models in real-time. It accepts parameters including model (e.g., 'openai/gpt-4o'), maxOutputTokens (e.g., 512), and prompt. The function returns an object with a textStream property.
Iterate textStream with for-await loop
The result.textStream from streamText can be iterated directly using a for-await loop: for await (const textPart of result.textStream) { ... }. Each iteration yields a chunk of text.
streamText function signature for chat
The streamText function takes these parameters: model (string, required, e.g. 'openai/gpt-4o'), maxOutputTokens (number, optional, e.g. 1024), system (string, optional, system prompt), and messages (array of objects, required, containing role and content fields). The messages array contains objects with role ('user' or 'assistant') and content (array of content objects with type and text properties).
Message format for chat completion
Messages in streamText must be an array where each element has a role ('user' or 'assistant') and content (an array of content blocks). Each content block has a type property (e.g. 'text') and corresponding data (e.g. text property for text blocks). This allows mixing different content types in a single message.
streamText with chat prompt example
The streamText function streams text generation from a model in real-time using a chat conversation history. The function accepts model, maxOutputTokens, system prompt, and messages array. Messages follow role-based structure with user and assistant roles, where content is an array containing text objects. The result.textStream is an async iterable that yields individual text chunks which can be consumed with a for-await loop. This allows displaying generated text to the client as it is produced rather than waiting for completion.
Streaming text output with async iteration
After calling streamText, access the textStream property on the result object. The textStream is an async iterable that yields text chunks one at a time. Use a for-await loop to iterate through the stream and write each chunk to output using process.stdout.write() or similar methods.
Stream text with image prompt using streamText
The streamText function from the AI SDK accepts a messages array where each message can contain multiple content items. To include an image alongside text, add a content array with both a text object and a file object. The file object requires type: 'file', mediaType: 'image', and a data property containing the binary image data read from the filesystem. The text stream is then iterated over with for await...of to write output as it arrives.
Stream text with image example
```ts import { streamText } from 'ai'; __PROVIDER_IMPORT__; import 'dotenv/config'; import fs from 'node:fs'; async function main() { const result = streamText({ model: __MODEL__, messages: [ { role: 'user', content: [ { type: 'text', text: 'Describe the image in detail.' }, { type: 'file', mediaType: 'image', data: fs.readFileSync('./data/comic-cat.png'), }, ], }, ], }); for await (const textPart of result.textStream) { process.stdout.write(textPart); } } main().catch(console.error); ``` This example shows how to stream text responses from a vision-language model when providing both a text prompt and an image file.
Streaming text output from vision-language models
After calling streamText with a model and multimodal messages, access result.textStream to iterate over the streamed response chunks. Use for await...of to consume the async iterable and write each text chunk to output as it arrives.
Stream text with file content in Node.js
Use streamText() with a message containing both text and file content. In the messages array, provide an object with role 'user' and content as an array. The content array includes a text object with type 'text' and the prompt, followed by a file object with type 'file', data as the file buffer read with fs.readFileSync(), and mediaType specifying the file format (e.g., 'application/pdf'). Iterate over result.textStream with a for-await loop to stream the response text.
Stream text with file prompt example
This example demonstrates streaming text analysis of a PDF file: ```ts import { streamText } from 'ai'; __PROVIDER_IMPORT__; import 'dotenv/config'; import fs from 'node:fs'; async function main() { const result = streamText({ model: __MODEL__, messages: [ { role: 'user', content: [ { type: 'text', text: 'What is an embedding model according to this document?', }, { type: 'file', data: fs.readFileSync('./data/ai.pdf'), mediaType: 'application/pdf', }, ], }, ], }); for await (const textPart of result.textStream) { process.stdout.write(textPart); } } main().catch(console.error); ``` This shows how to ask the model a question about a PDF file and stream the response text to stdout.
Stream Object for generative UI with partial data
Object generation can take a long time, especially with large schemas. For Generative UI use cases, you can stream the object to the client in real-time to render UIs as the object is being generated. Use `streamText` with `Output.object` to generate partial object streams that arrive incrementally.
partialOutputStream property from streamText
The `streamText` function returns a result object that contains a `partialOutputStream` property. This is an async iterable that yields partial object data as it is generated. You can iterate over it with `for await` to process each partial object as it arrives.
Use onEnd callback with streamText for token recording example
import { streamText, Output } from 'ai'; import { z } from 'zod'; const result = streamText({ model: 'openai/gpt-4.1', output: Output.object({ schema: z.object({ recipe: z.object({ name: z.string(), ingredients: z.array(z.string()), steps: z.array(z.string()), }), }), }), prompt: 'Generate a lasagna recipe.', onEnd({ usage }) { console.log('Token usage:', usage); }, });
Use onEnd callback to record token usage after streaming object
When streaming structured data with streamText and Output, you can use the onEnd callback to record token usage after the stream finishes. The onEnd callback receives an object containing a usage property with token usage information. This is useful for billing purposes.
streamText result contains a usage promise for token counts
The streamText result includes a usage property that is a Promise. When awaited, this promise resolves to a LanguageModelUsage object containing inputTokens, outputTokens, and totalTokens. This allows you to record token usage after the stream has completed by calling result.usage.then(callback) or awaiting result.usage.
LanguageModelUsage object structure
The LanguageModelUsage object has three properties: inputTokens (number of tokens in the prompt), outputTokens (number of tokens in the completion), and totalTokens (sum of input and output tokens).
Record token usage after streaming structured data example
import { streamText, Output, LanguageModelUsage } from 'ai'; import { z } from 'zod'; const result = streamText({ model: 'openai/gpt-4.1', output: Output.object({ schema: z.object({ recipe: z.object({ name: z.string(), ingredients: z.array(z.string()), steps: z.array(z.string()), }), }), }), prompt: 'Generate a lasagna recipe.', }); function recordUsage({ inputTokens, outputTokens, totalTokens, }: LanguageModelUsage) { console.log('Prompt tokens:', inputTokens); console.log('Completion tokens:', outputTokens); console.log('Total tokens:', totalTokens); } result.usage.then(recordUsage); for await (const partialObject of result.partialOutputStream) { }
streamText with structured output for vision tasks
The streamText function from the AI SDK supports streaming structured data (using Output.object with a Zod schema) when processing image prompts. The function returns a partialOutputStream that yields partial objects as they are generated, allowing you to iterate through results in real-time.
Streaming structured data with image prompts - File buffer example
This example demonstrates streaming structured object output from an image-based prompt using streamText with a Zod schema, where the image is provided as a base64-encoded file buffer: ```ts import { streamText, Output } from 'ai'; import dotenv from 'dotenv'; import { z } from 'zod'; import fs from 'fs'; dotenv.config(); async function main() { const { partialOutputStream } = streamText({ model: 'openai/gpt-4.1', maxOutputTokens: 512, output: Output.object({ schema: z.object({ stamps: z.array( z.object({ country: z.string(), date: z.string(), }), ), }), }), messages: [ { role: 'user', content: [ { type: 'text', text: 'list all the stamps in these passport pages?', }, { type: 'file', mediaType: 'image', data: fs.readFileSync('./data/passport.png', { encoding: 'base64', }), }, ], }, ], }); for await (const partialObject of partialOutputStream) { console.clear(); console.log(partialObject); } } main(); ```
Streaming structured data with image prompts - URL example
This example demonstrates streaming structured object output from an image-based prompt using streamText with a Zod schema that extracts stamp information from a passport image: ```ts import { streamText, Output } from 'ai'; import dotenv from 'dotenv'; import { z } from 'zod'; dotenv.config(); async function main() { const { partialOutputStream } = streamText({ model: 'openai/gpt-4.1', maxOutputTokens: 512, output: Output.object({ schema: z.object({ stamps: z.array( z.object({ country: z.string(), date: z.string(), }), ), }), }), messages: [ { role: 'user', content: [ { type: 'text', text: 'list all the stamps in these passport pages?', }, { type: 'file', mediaType: 'image', data: new URL( 'https://upload.wikimedia.org/wikipedia/commons/thumb/c/c5/WW2_Spanish_official_passport.jpg/1498px-WW2_Spanish_official_passport.jpg', ), }, ], }, ], }); for await (const partialObject of partialOutputStream) { console.clear(); console.log(partialObject); } } main(); ```
Streaming structured data with Output.object schema
Use Output.object() with a Zod schema to stream structured data. The schema defines the shape of the final object that will be returned and validated.
Example: Record final object after streaming with onEnd and result.output
import { streamText, Output } from 'ai'; import { z } from 'zod'; const result = streamText({ model: 'openai/gpt-4.1', output: Output.object({ schema: z.object({ recipe: z.object({ name: z.string(), ingredients: z.array(z.string()), steps: z.array(z.string()), }), }), }), prompt: 'Generate a lasagna recipe.', onEnd({ usage }) { console.log('Token usage:', usage); }, }); for await (const _ of result.partialOutputStream) { } try { const output = await result.output; console.log('Final object:', JSON.stringify(output, null, 2)); } catch (error) { console.error('Failed to parse output:', error); }
Example: Extract typed fields from result.output
import { streamText, Output } from 'ai'; import { z } from 'zod'; const result = streamText({ model: 'openai/gpt-4.1', output: Output.object({ schema: z.object({ recipe: z.object({ name: z.string(), ingredients: z.array(z.string()), steps: z.array(z.string()), }), }), }), prompt: 'Generate a lasagna recipe.', }); for await (const partialObject of result.partialOutputStream) { } try { const { recipe } = await result.output; console.log('Recipe:', JSON.stringify(recipe, null, 2)); } catch (error) { console.error(error); }
streamText result.output promise for structured data
The streamText result object contains an output promise that resolves to the final parsed object. This promise returns fully typed output according to your Zod schema. If type validation fails according to the schema, the promise rejects with a TypeValidationError. Await result.output after iterating through the partial stream to get the complete final object.
streamText onEnd callback for token usage
The streamText function accepts an onEnd callback that receives metadata including token usage. Call this callback to access usage information after the stream completes.
Streaming response chunks in manual agent loop
When streaming a response in a manual agent loop, iterate through result.stream and handle different chunk types: 'text-delta' chunks contain streamed text output that can be written to stdout; 'tool-call' chunks indicate that a tool is being called and contain the toolName.
pipeUIMessageStreamToResponse helper function
The pipeUIMessageStreamToResponse function pipes UI message stream data directly to a Node.js HTTP server response object. It accepts parameters: response (the HTTP response object) and stream (a UI message stream created via toUIMessageStream()).
pipeTextStreamToResponse for plain text streaming
For streaming plain text without UI message formatting, use pipeTextStreamToResponse with a toTextStream() converted stream. This sends raw text data to the HTTP response without message protocol wrapping.
Node.js HTTP server streaming setup requires AI_GATEWAY_API_KEY
When using the Vercel AI Gateway with a Node.js HTTP server, set the AI_GATEWAY_API_KEY environment variable. This is required for API authentication.
UI message stream example with custom data
Example showing how to send custom data alongside streamed text in a Node.js HTTP server: ```ts const stream = createUIMessageStream({ execute: ({ writer }) => { writer.write({ type: 'start' }); writer.write({ type: 'data-custom', data: { custom: 'Hello, world!' }, }); const result = streamText({ model: 'openai/gpt-4o', prompt: 'Invent a new holiday and describe its traditions.', }); writer.merge( toUIMessageStream({ stream: result.stream, sendStart: false, onError: error => { return error instanceof Error ? error.message : String(error); }, }), ); }, }); pipeUIMessageStreamToResponse({ stream, response: res }); ```
Basic text stream example for Node.js HTTP server
Example of streaming plain text from a Node.js HTTP server: ```ts import { pipeTextStreamToResponse, streamText, toTextStream } from 'ai'; import { createServer } from 'http'; createServer(async (req, res) => { const result = streamText({ model: 'openai/gpt-4o', prompt: 'Invent a new holiday and describe its traditions.', }); pipeTextStreamToResponse({ response: res, stream: toTextStream({ stream: result.stream }), }); }).listen(8080); ```
Basic Node.js HTTP server with text streaming
Create a simple HTTP server that listens on port 8080 and streams text responses using the AI SDK. The server accepts POST requests and uses streamText() to generate content, which is piped to the response using pipeUIMessageStreamToResponse().
Basic UI message stream example for Node.js HTTP server
Example of streaming UI messages from a Node.js HTTP server: ```ts import { pipeUIMessageStreamToResponse, streamText, toUIMessageStream, } from 'ai'; import { createServer } from 'http'; createServer(async (req, res) => { const result = streamText({ model: 'openai/gpt-4o', prompt: 'Invent a new holiday and describe its traditions.', }); pipeUIMessageStreamToResponse({ response: res, stream: toUIMessageStream({ stream: result.stream }), }); }).listen(8080); ```
toUIMessageStream converts text stream to UI message stream
The toUIMessageStream function converts a text stream from streamText() into a UI message stream format. It accepts a stream parameter and optional sendStart and onError parameters to control stream behavior and error handling.
Using cached middleware with streamText
import { openai } from '@ai-sdk/openai'; import { streamText } from 'ai'; import 'dotenv/config'; import { cached } from '../middleware/your-cache-middleware'; async function main() { const result = streamText({ model: cached(openai('gpt-4o')), maxOutputTokens: 512, temperature: 0.3, maxRetries: 5, prompt: 'Invent a new holiday and describe its traditions.', }); for await (const textPart of result.textStream) { process.stdout.write(textPart); } console.log(); console.log('Token usage:', await result.usage); console.log('Finish reason:', await result.finishReason); } main().catch(console.error); This example demonstrates wrapping a model with the cached middleware and using it with streamText to get both caching benefits and streaming output.
Cache file location and gitignore
The cache middleware stores responses in a JSON file at .cache/ai-cache.json (relative to the current working directory). You must add this path to your .gitignore to avoid committing cached responses to version control.
Local caching middleware is development-only
The local caching middleware approach is intended only for local development, not for production environments.
Cache invalidation for local caching middleware
To get fresh responses when using the local caching middleware, you need to clear the cache by deleting the cache file (typically .cache/ai-cache.json). There is no automatic cache invalidation mechanism.
Caching behavior with tool calls in multi-step flows
When using the caching middleware with stopWhen in multi-step flows, be aware that caching occurs at the individual language model response level, not across the entire execution flow. This means the model's generation is cached, but the tool call itself will run on each generation.
Local caching middleware purpose and use cases
A caching middleware stores AI model responses locally to avoid repeated API calls during development. It is particularly useful when iterating on UI/UX (where you don't want to regenerate responses for every code change) and when working on evals (where you need to test the same prompts repeatedly without generating new responses each time).
simulateReadableStream default chunk delay
The simulateReadableStream utility used in the caching middleware defaults to a 10ms delay between chunks when replaying cached streaming responses.
How local caching middleware works for streaming
For streaming implementations, the middleware captures each token as it arrives and stores the full sequence. On cache hits, it uses the SDK's simulateReadableStream utility to recreate the token-by-token streaming experience at a controlled speed (defaults to 10ms between chunks). This preserves streaming behavior for UI development while providing instant responses for repeated queries.
Local caching middleware implementation example
import { type LanguageModelV4Middleware, type LanguageModelV4StreamPart, type LanguageModelV4CallOptions, type LanguageModelV4, } from '@ai-sdk/provider'; import { safeParseJSON } from '@ai-sdk/provider-utils'; import 'dotenv/config'; import fs from 'fs'; import path from 'path'; import { wrapLanguageModel, simulateReadableStream } from 'ai'; const CACHE_FILE = path.join(process.cwd(), '.cache/ai-cache.json'); export const cached = (model: LanguageModelV4) => wrapLanguageModel({ middleware: cacheMiddleware, model, }); const ensureCacheFile = () => { const cacheDir = path.dirname(CACHE_FILE); if (!fs.existsSync(cacheDir)) { fs.mkdirSync(cacheDir, { recursive: true }); } if (!fs.existsSync(CACHE_FILE)) { fs.writeFileSync(CACHE_FILE, '{}'); } }; const getCachedResult = (key: string | object) => { ensureCacheFile(); const cacheKey = typeof key === 'object' ? JSON.stringify(key) : key; try { const cacheContent = fs.readFileSync(CACHE_FILE, 'utf-8'); const parseResult = safeParseJSON({ text: cacheContent }); if (!parseResult.success) { console.error('Failed to parse cache:', parseResult.error); return null; } const cache = parseResult.value as Record<string, unknown>; const result = cache[cacheKey]; return result ?? null; } catch (error) { console.error('Cache error:', error); return null; } }; const updateCache = (key: string, value: any) => { ensureCacheFile(); try { const parseResult = safeParseJSON({ text: fs.readFileSync(CACHE_FILE, 'utf-8'), }); const cache = parseResult.success ? (parseResult.value as Record<string, unknown>) : {}; const updatedCache = { ...cache, [key]: value }; fs.writeFileSync(CACHE_FILE, JSON.stringify(updatedCache, null, 2)); } catch (error) { console.error('Failed to update cache:', error); } }; const cleanPrompt = (prompt: LanguageModelV4CallOptions['prompt']) => { return prompt.map(m => { if (m.role === 'assistant') { return { ...m, content: m.content.map(part => part.type === 'tool-call' ? { ...part, toolCallId: 'cached' } : part, ), }; } if (m.role === 'tool') { return { ...m, content: m.content.map(tc => ({ ...tc, toolCallId: 'cached', result: {}, })), }; } return m; }); }; export const cacheMiddleware: LanguageModelV4Middleware = { specificationVersion: 'v4', wrapGenerate: async ({ doGenerate, params, model }) => { const cacheKey = JSON.stringify({ prompt: cleanPrompt(params.prompt), _function: 'generate', model: model.modelId, }); const cached = getCachedResult(cacheKey); if (cached && cached !== null) { return { ...cached, response: { ...cached.response, timestamp: cached?.response?.timestamp ? new Date(cached?.response?.timestamp) : undefined, }, }; } const result = await doGenerate(); updateCache(cacheKey, result); return result; }, wrapStream: async ({ doStream, params, model }) => { const cacheKey = JSON.stringify({ prompt: cleanPrompt(params.prompt), _function: 'stream', model: model.modelId, }); const cached = getCachedResult(cacheKey); if (cached && cached !== null) { const { chunks, ...rest } = cached; const formattedChunks = (chunks as LanguageModelV4StreamPart[]).map(p => { if (p.type === 'response-metadata' && p.timestamp) { return { ...p, timestamp: new Date(p.timestamp) }; } return p; }); return { stream: simulateReadableStream({ initialDelayInMs: 0, chunkDelayInMs: 10, chunks: formattedChunks, }), ...rest, }; } const { stream, ...rest } = await doStream(); const fullResponse: LanguageModelV4StreamPart[] = []; const transformStream = new TransformStream< LanguageModelV4StreamPart, LanguageModelV4StreamPart >({ transform(chunk, controller) { fullResponse.push(chunk); controller.enqueue(chunk); }, flush() { updateCache(cacheKey, { chunks: fullResponse, ...rest }); }, }); return { stream: stream.pipeThrough(transformStream), ...rest, }; }, }; This example shows a complete local caching middleware that intercepts both generate and stream calls, stores responses in a JSON cache file, and replays cached responses with simulated streaming on subsequent calls.
UI Message Stream example code
```ts import { pipeUIMessageStreamToResponse, streamText, toUIMessageStream, } from 'ai'; import express, { Request, Response } from 'express'; const app = express(); app.post('/', async (req: Request, res: Response) => { const result = streamText({ model: 'openai/gpt-4o', prompt: 'Invent a new holiday and describe its traditions.', }); pipeUIMessageStreamToResponse({ response: res, stream: toUIMessageStream({ stream: result.stream }), }); }); app.listen(8080, () => { console.log(`Example app listening on port ${8080}`); }); ``` This example shows how to use pipeUIMessageStreamToResponse to pipe stream data to the server response in Express.
Express server basic setup with streamText
Use the AI SDK in an Express server by importing streamText, toUIMessageStream, and pipeUIMessageStreamToResponse. Create an Express app, define a POST route, call streamText with model and prompt parameters, and pipe the stream to the response using pipeUIMessageStreamToResponse. The server listens on a specified port (e.g., 8080).