ai-sdk/image-generation

51 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.

generateImage function basic usage

The generateImage function from the ai package generates images based on a prompt using an image model. Basic usage: `const { image } = await generateImage({ model: __IMAGE_MODEL__, prompt: 'Santa Claus driving a Cadillac' });`

generateImage image data access

Image data from generateImage can be accessed via two properties: `image.base64` for base64 image data and `image.uint8Array` for Uint8Array binary image data.

generateImage size parameter format

The size parameter for generateImage is specified as a string in the format '{width}x{height}', for example '1024x1024'. Each model and provider supports different sizes.

generateImage aspectRatio parameter format

The aspectRatio parameter for generateImage is specified as a string in the format '{width}:{height}', for example '16:9'. Each model and provider supports different aspect ratios.

generateImage generating multiple images

The generateImage function supports generating multiple images at once using the `n` parameter: `const { images } = await generateImage({ model, prompt, n: 4 });`

generateImage automatic batching for multiple images

generateImage automatically batches requests when generating multiple images using the `n` parameter. Each image model has an internal limit on how many images it can generate in a single API call. For example, DALL-E 3 can only generate 1 image per call, while DALL-E 2 supports up to 10. The AI SDK manages this automatically by calling the model as often as needed in parallel.

generateImage maxImagesPerCall parameter

The `maxImagesPerCall` parameter allows overriding the default batch size for image generation: `const { images } = await generateImage({ model, prompt, maxImagesPerCall: 5, n: 10 });` This will make 2 calls of 5 images each. This is useful when working with new or custom models where the default batch size might not be optimal.

generateImage seed parameter

The seed parameter can be provided to generateImage to control the output of the image generation process. If supported by the model, the same seed will always produce the same image: `const { image } = await generateImage({ model, prompt, seed: 1234567890 });`

generateImage providerOptions parameter

Image models often have provider-specific settings that can be passed to generateImage using the `providerOptions` parameter. The options for the provider become request body properties. Example: `providerOptions: { openai: { style: 'vivid', quality: 'hd' } satisfies OpenAIImageModelGenerationOptions }`

generateImage abortSignal parameter

generateImage accepts an optional `abortSignal` parameter of type AbortSignal that can be used to abort the image generation process or set a timeout: `const { image } = await generateImage({ model, prompt, abortSignal: AbortSignal.timeout(1000) });`

generateImage headers parameter

generateImage accepts an optional `headers` parameter of type `Record<string, string>` that can be used to add custom headers to the image generation request: `const { image } = await generateImage({ model, prompt, headers: { 'X-Custom-Header': 'custom-value' } });`

generateImage warnings response property

If the model returns warnings (for example for unsupported parameters), they will be available in the `warnings` property of the generateImage response.

generateImage providerMetadata response property

Some providers expose additional metadata in the `providerMetadata` property of the generateImage response. The outer key is the provider name, the inner values are the metadata. An `images` key is always present in the metadata and is an array with the same length as the top-level `images` key. Example: `const { image, providerMetadata } = await generateImage({ model: openai.image('dall-e-3'), prompt }); const revisedPrompt = providerMetadata.openai.images[0]?.revisedPrompt;`

NoImageGeneratedError exception handling

When generateImage cannot generate a valid image, it throws an AI_NoImageGeneratedError. This error occurs when the AI provider fails to generate an image or when the model generated a response that could not be parsed. The error preserves `responses` (metadata about the image model responses including timestamp, model, and headers) and `cause` (the cause of the error for more detailed error handling). Check error type with: `if (NoImageGeneratedError.isInstance(error)) { ... }`

Image model wrapping with wrapImageModel

Image models can be enhanced using `wrapImageModel` and `ImageModelV4Middleware` to set default values or implement logging. Example: `const model = wrapImageModel({ model: __IMAGE_MODEL__, middleware: { specificationVersion: 'v3', transformParams: async ({ params }) => ({ ...params, size: params.size ?? '1024x1024' }) } });`

generateText image generation output

Some language models such as Google `gemini-2.5-flash-image` support multi-modal outputs including images. With such models, generated images can be accessed using the `files` property of the generateText response. Each file object provides: `file.base64` (string in data URL format), `file.uint8Array` (Uint8Array binary data), and `file.mediaType` (string e.g. 'image/png').

generateImage function

generateImage is an AI function located at packages/ai/src/generate-image/generate-image.ts that generates one or more images from prompt input.

ImageModelV4 interface

ImageModelV4 is the model specification interface located at packages/provider/src/image-model/v4/image-model-v4.ts that defines how image models generate image outputs from text prompts.

OpenAI image model implementation

OpenAIImageModel, located at packages/openai/src/image/openai-image-model.ts, is an example of a provider-specific image model implementation of ImageModelV4.

Google image model implementation

GoogleImageModel, located at packages/google/src/google-image-model.ts, is an example of a provider-specific image model implementation of ImageModelV4.

generateImage tool integration with chat

To generate images in a chat interface, create a tool using the generateImage function from the AI SDK. The tool accepts a prompt parameter and returns an object containing the image in base64 format and the original prompt. In production, save the generated image to blob storage and return a URL instead of base64 data to avoid generation failures.

generateImage tool schema definition

The generateImageTool is defined with a Zod schema that includes a required string parameter named 'prompt' with the description 'The prompt to generate the image from'. The execute function receives the prompt and calls generateImage with the model and prompt parameters.

generateImage function with OpenAI DALL-E-3

The generateImage function accepts a model parameter set to openai.imageModel('dall-e-3') and a prompt parameter. It returns an object with an image property containing base64-encoded image data.

Display generated image in Next.js

Generated images returned as base64 data can be displayed using the Next.js Image component with a data URL in the format 'data:image/png;base64,{base64_string}'. The alt text should use the prompt that generated the image.

generateImage example code

const { image } = await generateImage({ model: openai.imageModel('dall-e-3'), prompt, }); return { image: image.base64, prompt };

wrapImageModel function signature and parameters

The wrapImageModel function accepts the following parameters: model (type ImageModelV4, required) - the original ImageModelV4 instance to be wrapped; middleware (type ImageModelV4Middleware | ImageModelV4Middleware[], required) - the middleware to be applied to the image model, where when multiple middlewares are provided, the first middleware transforms the input first and the last middleware is wrapped directly around the model; modelId (type string, optional) - a custom model ID to override the original model's ID; providerId (type string, optional) - a custom provider ID to override the original model's provider. The function returns a new ImageModelV4 instance with middleware applied.

wrapImageModel import statement

The wrapImageModel function can be imported from the 'ai' package using: import { wrapImageModel } from 'ai'

wrapImageModel usage example with OpenAI

Example: import { generateImage, wrapImageModel } from 'ai'; import { openai } from '@ai-sdk/openai'; const model = wrapImageModel({ model: openai.image('gpt-image-2'), middleware: yourImageModelMiddleware, }); const { image } = await generateImage({ model, prompt: 'Santa Claus driving a Cadillac', });

generateImage with provider registry image model

Use generateImage with model: registry.imageModel('openai:dall-e-3') to generate images using an image model accessed from a provider registry.

experimental_generateVideo function signature and basic usage

The AI SDK provides the experimental_generateVideo function to generate videos based on a prompt using a video model. Basic usage: const { video } = await generateVideo({ model: __VIDEO_MODEL__, prompt: 'A cat walking on a treadmill' }). Video data is accessed via video.base64 or video.uint8Array properties.

experimental_generateVideo aspectRatio parameter format

The aspectRatio parameter is specified as a string in the format {width}:{height}. Models only support specific aspect ratios that vary per model and provider. Some models also accept 'adaptive', which lets the provider derive the output ratio from input media instead of a fixed value. The 'adaptive' option is typically required for image-to-video, video editing, and video extension.

experimental_generateVideo resolution parameter format

The resolution parameter is specified as a string in the format {width}x{height}. Models only support specific resolutions, and the supported resolutions are different for each model and provider.

experimental_generateVideo duration parameter

Some video models support specifying the duration of the generated video in seconds using the duration parameter.

experimental_generateVideo fps parameter

Some video models allow specifying the frames per second for the generated video using the fps parameter.

experimental_generateVideo generateAudio option

Some video models can generate audio alongside the video. Use the generateAudio boolean option to control this: { generateAudio: true }.

experimental_generateVideo multiple videos with n parameter

The experimental_generateVideo function supports generating multiple videos at once using the n parameter: { n: 3 } generates 3 videos. The AI SDK automatically calls the model as often as needed in parallel to generate the requested number of videos.

experimental_generateVideo maxVideosPerCall parameter

The maxVideosPerCall setting allows overriding the default batch size for video generation. Each video model has an internal limit on how many videos it can generate in a single API call. The AI SDK manages this automatically by batching requests appropriately. Most video models only support generating 1 video per call due to computational cost.

experimental_generateVideo image-to-video generation

Some video models support generating videos from an input image. Provide an image using the prompt object: { image: 'https://example.com/my-image.png', text: 'Animate this image with gentle motion' }. The image can be provided as a URL, base64-encoded string, or Uint8Array.

experimental_generateVideo frameImages parameter

Some video models support first-last-frame generation. Use the frameImages option to pass role-tagged images: frameImages: [{ image: 'https://example.com/first-frame.png', frameType: 'first_frame' }, { image: 'https://example.com/last-frame.png', frameType: 'last_frame' }].

experimental_generateVideo inputReferences parameter

Some video models support reference-to-video generation where you provide reference images or videos that the model incorporates into the generated video. Use the inputReferences option as an array of URLs or objects. For URL-based video references, use the object form with explicit mediaType: { data: 'https://example.com/reference.mp4', mediaType: 'video/mp4' }. Providers route each reference by its media type and emit a warning when a reference kind is unsupported.

experimental_generateVideo seed parameter

You can provide a seed parameter to control the output of the video generation process. If supported by the model, the same seed will always produce the same video.

experimental_generateVideo providerOptions parameter

Video models often have provider- or model-specific settings. Pass such settings using the providerOptions parameter. The options for the provider become request body properties. Example: { providerOptions: { fal: { loop: true, motionStrength: 0.8 } } }.

experimental_generateVideo abortSignal parameter

The experimental_generateVideo function accepts an optional abortSignal parameter of type AbortSignal that can be used to abort the video generation process or set a timeout. Example: { abortSignal: AbortSignal.timeout(60000) } aborts after 60 seconds. Video generation typically takes longer than image generation; consider using longer timeouts (60 seconds or more) depending on the model and video length.

experimental_generateVideo polling configuration

Video generation is an asynchronous process that can take several minutes to complete. The SDK automatically polls the provider to check if the video is ready. Configure polling behavior with the poll option: { poll: { intervalMs: 5000, timeoutMs: 600000 } }. Default intervalMs is 5 seconds, default timeoutMs is 600000 milliseconds (10 minutes). For durable workflows with custom sleep primitives, pass poll.delay which is used for both polling intervals and webhook timeouts.

experimental_generateVideo webhook support

For models with native webhook support, pass a webhook factory that returns a public URL and a promise that resolves when the application receives the webhook request. Example: { webhook: async () => { const { url, received } = await createWebhook(); return { url, received }; } }. The createWebhook helper registers a webhook listener and its received promise must resolve with request headers and body. The SDK sends url to the provider, waits for received, and then retrieves the completed video. You can provide poll together with webhook; if the model supports webhooks, poll.timeoutMs limits how long the SDK waits for notification; if not supported, the SDK falls back to polling.

experimental_generateVideo headers parameter

The experimental_generateVideo function accepts an optional headers parameter of type Record<string, string> that can be used to add custom headers to the video generation request.

experimental_generateVideo warnings in response

If the model returns warnings (e.g., for unsupported parameters), they will be available in the warnings property of the response.

experimental_generateVideo providerMetadata response

Some providers expose additional metadata for the result overall or per video in the providerMetadata property. The outer key of the returned providerMetadata is the provider name. A videos key is typically present in the metadata as an array with the same length as the top level videos key. Each video metadata may include: duration, fps, width, height. When generating multiple videos with n > 1, access per-call metadata through the responses array, which contains objects with: timestamp, modelId, and providerMetadata.

experimental_generateVideo error handling with AI_NoVideoGeneratedError

When experimental_generateVideo cannot generate a valid video, it throws an AI_NoVideoGeneratedError. This error occurs when the AI provider fails to generate a video due to: the model failed to generate a response, or the model generated a response that could not be parsed. The error preserves: responses (metadata about video model responses including timestamp, model, and headers) and cause (the cause of the error for detailed error handling). Use NoVideoGeneratedError.isInstance(error) to check for this error type.

Video models available in AI SDK providers

Supported video models: FAL luma-dream-machine/ray-2 (text-to-video, image-to-video), FAL minimax-video (text-to-video), Google veo-2.0-generate-001 (text-to-video, up to 4 videos per call), Google Vertex veo-3.1-generate-001 (text-to-video, audio generation), Google Vertex veo-3.1-fast-generate-001 (text-to-video, audio generation), Google Vertex veo-3.0-generate-001 (text-to-video, audio generation), Google Vertex veo-3.0-fast-generate-001 (text-to-video, audio generation), Google Vertex veo-2.0-generate-001 (text-to-video, up to 4 videos per call), Kling AI kling-v2.6-t2v (text-to-video), Kling AI kling-v2.6-i2v (image-to-video), Kling AI kling-v2.6-motion-control (motion control), Replicate minimax/video-01 (text-to-video), xAI grok-imagine-video (text-to-video, image-to-video, editing, extension, R2V).

experimental_generateVideo is an experimental feature

Video generation is an experimental feature in the AI SDK. The API may change in future versions.

Give your agent this brain