ai-sdk/streaming

67 notes in this subject, read out of this brain and free to use. This is page 1 of 2.

Server route for streaming text with image using Next.js

On the server side in app/api/chat/route.ts, import convertToModelMessages, createUIMessageStreamResponse, streamText, and toUIMessageStream from 'ai'. Call streamText with model 'openai/gpt-4.1' and messages converted via convertToModelMessages. Return the stream using createUIMessageStreamResponse with toUIMessageStream wrapper.

Stream text with image using GPT-4o via useChat

To stream text with an image prompt using the AI SDK and Next.js, use the streamText function on the server with convertToModelMessages to handle multimodal content. On the client, use the useChat hook and send messages with a parts array containing file objects for images and text objects for prompts. The file object should have type 'file', mediaType 'image/png', and url properties.

Client-side useChat implementation for multimodal messages

On the client side in app/page.tsx, import useChat from '@ai-sdk/react'. In the form submission handler, call sendMessage with a message object containing role 'user' and parts array. Conditionally include file objects for images with type 'file', mediaType 'image/png', and url properties, followed by a text object with the user's prompt.

generateText with image prompt example for Node.js

Example showing how to use generateText() with an image URL prompt: import { generateText } from 'ai'; const result = await generateText({ model: 'openai/gpt-4.1', maxOutputTokens: 512, messages: [ { role: 'user', content: [ { type: 'text', text: 'what are the red things in this image?', }, { type: 'file', mediaType: 'image', data: new URL( 'https://upload.wikimedia.org/wikipedia/commons/thumb/3/3e/2024_Solar_Eclipse_Prominences.jpg/720px-2024_Solar_Eclipse_Prominences.jpg', ), }, ], }, ], }); console.log(result);

generateText with base64 image buffer example for Node.js

Example showing how to use generateText() with a base64-encoded image buffer: import { generateText } from 'ai'; import fs from 'fs'; const result = await generateText({ model: 'openai/gpt-4.1', maxOutputTokens: 512, messages: [ { role: 'user', content: [ { type: 'text', text: 'what are the red things in this image?', }, { type: 'file', mediaType: 'image', data: fs.readFileSync('./node/attachments/eclipse.jpg', { encoding: 'base64', }), }, ], }, ], }); console.log(result);

Stream text with image using streamText and vision models

Vision-language models can analyze images alongside text prompts to generate responses about visual content. Use streamText with a vision model, passing messages with both text and image content. The example shows how to structure a multimodal message with text and an image file to perform tasks like visual question answering and image analysis.

Message content structure for images in AI SDK

When using vision models with streamText, message content is an array that can include multiple content objects. Text content uses {type: 'text', text: 'string'}. Image/file content uses {type: 'file', mediaType: 'image', data: Buffer} where mediaType is 'image' and data is a Buffer containing the image file bytes.

Iterate over textStream in Node.js for streamed responses

The streamText function returns a result object with a textStream property. In Node.js, use 'for await...of' to iterate over result.textStream and process streamed text chunks using process.stdout.write() or similar.

Load image files with fs.readFileSync for prompts

To include an image file in a prompt, read it as a Buffer using fs.readFileSync() from Node's fs module, then pass the Buffer as the data property in a file-type content object.

generateText function

generateText is an AI function located at packages/ai/src/generate-text/generate-text.ts that generates a complete text result from a language model in a single call.

streamText function

streamText is an AI function located at packages/ai/src/generate-text/stream-text.ts that streams language model output incrementally as it is produced.

Stream text loop control - step execution flow

The stream text loop executes in a do-while loop where each iteration represents a step. Each step: (1) prepares input and converts messages to language model v4 format, (2) calls doStream with those messages, (3) transforms the stream for user-friendly format, (4) executes tool transformations that run tools and inject tool results, (5) pipes through further transforms (add start-step, filter empty text, tool input start, filter raw chunks, add finish-step, add finish), performs bookkeeping of tool calls/outputs/errors, manages timeouts, and emits telemetry/tool input delta events, (6) adds the transformed stream to a stitchable stream, (7) converts assembled step output to additional response model messages. The loop continues unless: a stop condition is met, finish reason is not tool-calls, a tool without execute is called, a tool needing approval is called, or there are deferred tool calls.

Stream pipeline - stitchable stream structure

Multiple step streams are sequentially funneled into a stitchable stream. Each step produces: tool callbacks, tool execution results, step metadata with start/finish markers. These are added to the stitchable stream via addStream() calls. The stitchable stream is a sequential queue that consumes one stream at a time, with the next step added on recursion from flush.

Stream pipeline - middle pipeline transforms

After the stitchable stream, there is a single linear transform chain in the middle pipeline that includes: (1) resilient stream for abort handling and start event emission, (2) stop gate for stopStream() support, (3) user transforms from experimental_transform[] array, (4) output transform that enriches with partialOutput, (5) event processor that handles onChunk, onStepFinish callbacks, accumulates content, and resolves delayed promises.

Stream pipeline - output funneling via .tee()

The base stream is output on-demand using .tee() operations. Each .tee() call splits the stream into two: one for the consumer and one that remains as the baseStream for the next tee. This can be called multiple times. The final outputs are: textStream (text deltas only), fullStream (all parts), partialOutput (json parse), elementStream (output spec), uiMessageStream (maps to UI), and consumeStream (drains stream and resolves promises).

Stream pipeline - step stream processing

Each step produces a stream from model.doStream() that goes through sequential processing: tool callbacks are executed, tool execution results are generated, step metadata with start/finish markers is added. Each completed step is added to the stitchable stream via addStream() call.

useChat hook with Gemini in Next.js

The useChat hook from @ai-sdk/react provides messages array and sendMessage function. Messages have id, role, and parts properties where parts are arrays containing objects with type property (e.g., 'text') and corresponding data (e.g., text property for text type).

useChat hook location

The useChat hook is imported from '@ai-sdk/react'.

streamText with tools integration

The streamText function accepts a model parameter, messages converted to model format, a stopWhen parameter for flow control, and a tools object containing tool definitions. In this example, tools includes the generateImage tool and the model is 'openai/gpt-4o'.

UIMessage structure with tool invocations

UIMessage contains an id, role, and parts array. Each part can be of type 'text' or 'tool-generateImage'. Tool parts contain a state property ('input-available' or 'output-available'), toolCallId, input object (with prompt), and output object (with image base64 data).

Sending messages with useChat

The sendMessage function accepts a message object with a parts array. Each part should have a type property set to 'text' and a text property containing the message content.

Tool state transitions in chat

Tool invocations in chat messages transition through states: 'input-available' indicates the tool is being executed and should display a loading message; 'output-available' indicates the tool execution is complete and results are available for display.

createUIMessageStreamResponse for tool-enabled endpoints

The createUIMessageStreamResponse function from the ai module wraps a UIMessageStream to create an appropriate HTTP response for tool-enabled chat endpoints. It accepts a stream parameter that is created by wrapping streamText's result stream with toUIMessageStream.

streamText with URL-based image input

When streaming structured data with an image prompt, you can include images via URL using the file content type. The image is referenced as a URL object in the messages content array with type 'file', mediaType 'image', and data set to a new URL instance pointing to the image location.

streamText with base64-encoded file buffer image

When streaming structured data with an image prompt, you can include images via file buffer by reading the file with fs.readFileSync in base64 encoding. The image is referenced in the messages content array with type 'file', mediaType 'image', and data set to the base64-encoded file string.

streamText returns partialOutputStream for structured data

The streamText function with Output.object() returns a partialOutputStream property that yields partial objects as they are generated. You can iterate over this stream using a for-await loop to receive incremental updates of the structured data.

streamText Output.object schema definition

The Output.object() configuration accepts a schema property that defines the structure of the output using Zod validation. The schema constrains the streaming output to match the specified object structure with typed fields.

Client-side UI rendering from tool output data

When tools return JSON objects instead of text, the client can conditionally render different React components based on the tool type using switch statements or if/else logic on part.type. For example, part.type 'tool-api-search-course' could render a <Courses/> component, while part.type 'tool-api-meetings' could render a <Meetings/> component.

Access tool results from message parts with part.output

When iterating over message.parts in an assistant message, tool results are available with a part.type like 'tool-weather' and part.state of 'output-available'. The actual data returned by the tool is accessed via part.output.

createStreamableUI function from @ai-sdk/rsc

The createStreamableUI() function belongs to the @ai-sdk/rsc module and creates a stream that can send React components to the client. It has a done() method that accepts a React component to stream to the client.

Server-side component rendering with RSC simplifies UI flow

Using React Server Components with @ai-sdk/rsc, you can render React components on the server during tool execution and stream them to the client. This eliminates the need for conditional rendering logic on the client side. The client receives pre-rendered components via message.display.

Tool result rendering example with weather data

Example showing how to access and render tool results from message parts: ```tsx if (part.type === 'tool-weather' && part.state === 'output-available') { const { temperature, unit, description, forecast } = part.output; return ( <WeatherCard weather={{ temperature, unit, description, forecast, }} /> ); } ``` This accesses the JSON object returned by the tool's execute function and passes it as props to a React component.

Server-side component streaming example with createStreamableUI

Example showing how to render a component on the server and stream it to the client: ```tsx import { createStreamableUI } from '@ai-sdk/rsc' const uiStream = createStreamableUI(); const text = generateText({ model: __MODEL__, instructions: 'you are a friendly assistant', prompt: 'what is the weather in SF?', tools: { getWeather: { description: 'Get the weather for a location', inputSchema: z.object({ city: z.string().describe('The city to get the weather for'), unit: z.enum(['C', 'F']).describe('The unit to display the temperature in') }), execute: async ({ city, unit }) => { const weather = getWeather({ city, unit }) const { temperature, unit, description, forecast } = weather uiStream.done( <WeatherCard weather={{ temperature: 47, unit: 'F', description: 'sunny', forecast, }} /> ) } } } }) return { display: uiStream.value } ``` On the client, render with: ```tsx return ( <div> {messages.map(message => ( <div>{message.display}</div> ))} </div> ); ```

Multiple tool rendering pattern with switch statement

When an application has multiple tools that each return different user interfaces, use a switch statement on part.type to conditionally render the appropriate component: ```tsx message.parts.map(part => { if (part.state !== 'output-available') return null; switch (part.type) { case 'tool-api-search-course': return <Courses courses={part.output} />; case 'tool-api-search-profile': return <People people={part.output} />; case 'tool-api-meetings': return <Meetings meetings={part.output} />; case 'tool-api-search-building': return <Buildings buildings={part.output} />; case 'tool-api-events': return <Events events={part.output} />; case 'tool-api-meals': return <Meals meals={part.output} />; case 'text': return <div>{part.text}</div>; default: return null; } }); ```

streamText response streaming pattern

To stream text responses to the client, use createUIMessageStreamResponse with toUIMessageStream. Pass the stream from streamText's result to toUIMessageStream, then pass the transformed stream to createUIMessageStreamResponse. This converts the stream into a UI-compatible message stream for consumption by the useChat hook.

Chat completion streaming example with Next.js

Example: Client component uses useChat hook with DefaultChatTransport pointing to '/api/chat' endpoint. User types a message and presses Enter, triggering sendMessage with { parts: [{ type: 'text', text: input }] }. Server endpoint receives UIMessage[] in request body, calls streamText with 'openai/gpt-4o' model and user messages, returns response using createUIMessageStreamResponse.

createUIMessageStreamResponse and toUIMessageStream for API responses

Use createUIMessageStreamResponse with toUIMessageStream to convert the result.stream from streamText into a proper response stream for API routes. toUIMessageStream accepts an object with a stream property containing the result.stream.

simulateReadableStream parameters control chunk timing

The simulateReadableStream function accepts initialDelayInMs and chunkDelayInMs parameters to control the initial delay and delay between chunks when replaying cached stream data.

React memo component for custom comparison

React's memo function can accept a custom comparison function as a second parameter. The comparison function receives prevProps and nextProps and should return true if props are equal (skip re-render) or false if props differ (perform re-render). Example: memo(Component, (prevProps, nextProps) => { if (prevProps.content !== nextProps.content) return false; return true; }).

createUIMessageStreamResponse function

The createUIMessageStreamResponse function creates a streaming response for the UI by accepting a stream object. It is used to wrap the UI message stream: createUIMessageStreamResponse({ stream: toUIMessageStream({ stream: result.stream }) }).

toUIMessageStream function

The toUIMessageStream function converts a streaming text result stream into a UI message stream format. It accepts an object with a stream property: toUIMessageStream({ stream: result.stream }).

Memoization for Markdown rendering performance

When rendering streaming Markdown responses in a chatbot, using memoization prevents re-parsing and re-rendering of already-processed Markdown blocks on each new token. This technique significantly improves rendering performance for long conversations by caching parsed Markdown blocks and reusing them rather than regenerating them with each token update.

streamText function with system prompt and model

The streamText function accepts a system prompt, model identifier, and messages array. Example usage: streamText({ system: 'You are a helpful assistant. Respond to the user in Markdown format.', model: 'openai/gpt-4o', messages: await convertToModelMessages(messages) }). The function returns a result object with a stream property.

createUIMessageStreamResponse for streaming chat

createUIMessageStreamResponse accepts a stream parameter that is created from toUIMessageStream, which wraps the result.stream from streamText.

streamText function signature and parameters

The streamText function accepts an object with the following parameters: model (string, required), maxOutputTokens (number), system (string for system prompt), and messages (array of message objects). Each message object has a role ('user' or 'assistant') and content (array with objects containing type and text properties).

streamText example with chat messages

Example showing how to use streamText with OpenAI's gpt-4o model: ```ts import { streamText } from 'ai'; const result = streamText({ model: 'openai/gpt-4o', maxOutputTokens: 1024, system: 'You are a helpful chatbot.', messages: [ { role: 'user', content: [{ type: 'text', text: 'Hello!' }], }, { role: 'assistant', content: [{ type: 'text', text: 'Hello! How can I help you today?' }], }, { role: 'user', content: [{ type: 'text', text: 'I need help with my computer.' }], }, ], }); for await (const textPart of result.textStream) { process.stdout.write(textPart); } ``` This example demonstrates streaming chat completion responses in real-time using the AI SDK.

maxOutputTokens parameter for controlling response length

The streamText function accepts a maxOutputTokens parameter to limit the maximum number of tokens in the generated response. In the example, this is set to 1024.

streamText returns textStream for iteration

The streamText function returns an object with a textStream property that is an async iterable. Text chunks can be consumed using for-await-of loops to process streamed responses incrementally.

streamText file prompt syntax with PDF

The streamText function accepts file content in the messages array using a content object with type 'file'. The file object requires three properties: data (the file content as a buffer from fs.readFileSync), mediaType (the MIME type, e.g. 'application/pdf'), and the text prompt as a separate content item with type 'text'. The textStream property of the result can be iterated with for-await-of to stream text chunks.

How the caching middleware handles streaming responses

The streaming implementation captures each token as it arrives and stores the full sequence. On cache hits, it uses the SDK's `simulateReadableStream` utility to recreate the token-by-token streaming experience at a controlled speed (defaults to 10ms between chunks). This preserves the streaming behavior for UI development while providing instant responses for repeated queries.

simulateReadableStream utility for cached streaming

The SDK provides a `simulateReadableStream` utility that recreates a token-by-token streaming experience from cached chunks. It accepts parameters: initialDelayInMs (delay before first chunk, defaults to 0), chunkDelayInMs (delay between chunks, defaults to 10ms), and chunks (array of LanguageModelV4StreamPart).

maxDuration export sets streaming response timeout

In a Next.js server component or client component, export maxDuration to set the maximum time allowed for streaming responses. In this example, maxDuration is set to 30 seconds.

maxDuration for streaming responses

The maxDuration export set to 30 seconds allows streaming responses to continue for up to 30 seconds in React Server Components.

Message interface for chat

The Message interface has a role field that is either 'user' or 'assistant', and a content field that is a string containing the message text.

streamText parameter: model

The streamText function accepts a model parameter that takes a provider/model identifier string, such as 'openai/gpt-5.4'.

streamText parameter: system

The streamText function accepts a system parameter that specifies the system prompt as a string to set the behavior of the AI model.

RSC streaming with readStreamableValue

On the client side, readStreamableValue is imported from '@ai-sdk/rsc' and used to read a streamable value returned from a server action. It is an async iterable that yields text deltas as they arrive from the server.

streamText function with chat history

The streamText function accepts a model identifier, a system prompt, and a messages array containing the conversation history. It returns an object containing textStream which is an async iterable of text deltas. The example uses model 'openai/gpt-5.4', a system prompt, and messages array to stream text responses.

Streaming text response with createStreamableValue

createStreamableValue is used to create a streamable value that can be updated with text deltas from streamText. The pattern involves calling stream.update(text) for each delta from the textStream async iterable, then calling stream.done() when streaming completes. The stream.value property is returned to the client.

streamText parameter: messages

The streamText function accepts a messages parameter that is an array of Message objects containing the conversation history with role and content fields.

Give your agent this brain