xai/capabilities

40 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.

xAI text generation example

```ts import { xai } from '@ai-sdk/xai'; import { generateText } from 'ai'; const { text } = await generateText({ model: xai('grok-4.6'), prompt: 'Write a vegetarian lasagna recipe for 4 people.', }); ``` This example shows how to use xAI language models with the `generateText` function.

xAI Realtime API models

Create models that call the xAI Realtime API using the `.experimental_realtime()` factory method: `xai.experimental_realtime('grok-voice-latest')`. Realtime sessions run in the browser and require a short-lived token created on the server with `xai.experimental_realtime.getToken({model: 'grok-voice-latest'})`.

xAI Responses API (Agentic Tools)

The xAI Responses API is the default when using `xai(modelId)` since AI SDK 7, or explicitly use `xai.responses(modelId)`. This enables the model to autonomously orchestrate tool calls and research on xAI's servers. Server-side tools available: web_search, x_search, code_execution, view_image, view_x_video, mcp_server, file_search.

xAI vision support with Responses API

The Responses API supports image input with vision models. Pass images via message content with type 'file' and mediaType 'image'. Control image resolution with the `imageDetail` provider option on the image part: 'low' (reduced resolution, fewer tokens), 'high' (full resolution), 'auto' (xAI decides). When not set, the image is processed at full resolution.

xAI vision example

```ts import { xai } from '@ai-sdk/xai'; import { generateText } from 'ai'; const { text } = await generateText({ model: xai.responses('grok-4.6'), messages: [ { role: 'user', content: [ { type: 'text', text: 'What do you see in this image?' }, { type: 'file', mediaType: 'image', data: fs.readFileSync('./image.png'), }, ], }, ], }); ``` This example shows how to use vision with xAI Responses API.

xAI imageDetail provider option example

```ts import { xai } from '@ai-sdk/xai'; import { generateText } from 'ai'; const { text } = await generateText({ model: xai('grok-4.6'), messages: [ { role: 'user', content: [ { type: 'text', text: 'What do you see in this image?' }, { type: 'file', mediaType: 'image/png', data: fs.readFileSync('./image.png'), providerOptions: { xai: { imageDetail: 'low' }, }, }, ], }, ], }); ``` This example shows how to control image resolution with imageDetail.

xAI web search tool example

```ts import { xai } from '@ai-sdk/xai'; import { generateText } from 'ai'; const { text, sources } = await generateText({ model: xai.responses('grok-4.6'), prompt: 'What are the latest developments in AI?', tools: { web_search: xai.tools.webSearch({ allowedDomains: ['arxiv.org', 'openai.com'], enableImageUnderstanding: true, }), }, }); console.log(text); console.log('Citations:', sources); ``` This example shows how to use xAI's web search tool with domain filtering.

xAI transcription example

```ts import { xai } from '@ai-sdk/xai'; import { transcribe } from 'ai'; import { readFile } from 'fs/promises'; const result = await transcribe({ model: xai.transcription(), audio: await readFile('meeting.mp3'), }); ``` This example shows how to use xAI's transcription model.

xAI X search tool example

```ts const { text, sources } = await generateText({ model: xai.responses('grok-4.6'), prompt: 'What are people saying about AI on X this week?', tools: { x_search: xai.tools.xSearch({ allowedXHandles: ['elonmusk', 'xai'], fromDate: '2025-10-23', toDate: '2025-10-30', enableImageUnderstanding: true, enableVideoUnderstanding: true, }), }, }); ``` This example shows how to use xAI's X search tool with handle and date filtering.

xAI code execution tool example

```ts const { text } = await generateText({ model: xai.responses('grok-4.6'), prompt: 'Calculate the compound interest for $10,000 at 5% annually for 10 years', tools: { code_execution: xai.tools.codeExecution(), }, }); ``` This example shows how to use xAI's code execution tool for Python calculations.

xAI view image tool example

```ts const { text } = await generateText({ model: xai.responses('grok-4.6'), prompt: 'Describe what you see in the image', tools: { view_image: xai.tools.viewImage(), }, }); ``` This example shows how to use xAI's view image tool.

xAI view X video tool example

```ts const { text } = await generateText({ model: xai.responses('grok-4.6'), prompt: 'Summarize the content of this X video', tools: { view_x_video: xai.tools.viewXVideo(), }, }); ``` This example shows how to use xAI's view X video tool.

xAI image generation tool

The image generation tool lets the model create and edit images with Grok Imagine. The model decides when to call the tool, writes the image prompt, and returns the finished image alongside text response. Access generated images via `result.staticToolResults`.

xAI image generation tool example

```ts import { xai } from '@ai-sdk/xai'; import { generateText } from 'ai'; const result = await generateText({ model: xai.responses('grok-4.6'), prompt: 'Generate an image of a corgi surfing a big wave, in the style of a Japanese woodblock print', tools: { image_generation: xai.tools.imageGeneration(), }, }); for (const toolResult of result.staticToolResults) { if (toolResult.toolName === 'image_generation') { const base64Image = toolResult.output.result; } } ``` This example shows how to use xAI's image generation tool.

xAI MCP server tool example

```ts const { text } = await generateText({ model: xai.responses('grok-4.6'), prompt: 'Use the weather tool to check conditions in San Francisco', tools: { weather_server: xai.tools.mcpServer({ serverUrl: 'https://example.com/mcp', serverLabel: 'weather-service', serverDescription: 'Weather data provider', allowedTools: ['get_weather', 'get_forecast'], }), }, }); ``` This example shows how to use xAI's MCP server tool.

xAI file search tool example

```ts import { xai, type XaiLanguageModelResponsesOptions } from '@ai-sdk/xai'; import { streamText } from 'ai'; const result = streamText({ model: xai.responses('grok-4.6'), prompt: 'What documents do you have access to?', tools: { file_search: xai.tools.fileSearch({ vectorStoreIds: ['collection_your-collection-id'], maxNumResults: 10, }), }, providerOptions: { xai: { include: ['file_search_call.results'], } satisfies XaiLanguageModelResponsesOptions, }, }); ``` This example shows how to use xAI's file search tool.

xAI multiple tools example

```ts import { xai } from '@ai-sdk/xai'; import { streamText } from 'ai'; const { stream } = streamText({ model: xai.responses('grok-4.6'), prompt: 'Research AI safety developments and calculate risk metrics', tools: { web_search: xai.tools.webSearch(), x_search: xai.tools.xSearch(), code_execution: xai.tools.codeExecution(), file_search: xai.tools.fileSearch({ vectorStoreIds: ['collection_your-documents'], }), data_service: xai.tools.mcpServer({ serverUrl: 'https://data.example.com/mcp', serverLabel: 'data-service', }), }, }); for await (const part of stream) { if (part.type === 'text-delta') { process.stdout.write(part.text); } else if (part.type === 'source' && part.sourceType === 'url') { console.log('\nSource:', part.url); } } ``` This example shows how to combine multiple server-side tools.

xAI speech models available

xAI provides text-to-speech via `.speech()` factory method. Create models using `xai.speech()` with no model identifier required. Use with `generateSpeech` function from AI SDK.

xAI text-to-speech example

```ts import { xai } from '@ai-sdk/xai'; import { generateSpeech } from 'ai'; const result = await generateSpeech({ model: xai.speech(), text: 'Hello from the AI SDK!', voice: 'ara', language: 'en', outputFormat: 'mp3', speed: 1.1, }); ``` This example shows how to use xAI's text-to-speech model.

xAI transcription models available

xAI provides speech-to-text via `.transcription()` factory method. Create models using `xai.transcription()` with no model identifier required. Use with `transcribe` function from AI SDK.

xAI image generation models

xAI provides image generation via `.image()` factory method. Create models with `xai.image('grok-imagine-image')`. Use with `generateImage()` function. xAI does not support `size` parameter; use `aspectRatio` instead. Supported aspect ratios: 1:1, 16:9, 9:16, 4:3, 3:4, 3:2, 2:3, 2:1, 1:2, 19.5:9, 9:19.5, 20:9, 9:20, auto.

xAI image generation example

```ts import { xai } from '@ai-sdk/xai'; import { generateImage } from 'ai'; const { image } = await generateImage({ model: xai.image('grok-imagine-image'), prompt: 'A futuristic cityscape at sunset', }); ``` This example shows how to use xAI's image generation model.

xAI image editing support

xAI supports image editing via `grok-imagine-image` model. Pass input images via `prompt.images` to transform or edit. xAI image editing does not support masks; editing is prompt-driven - describe what to change in text prompt.

xAI basic image editing example

```ts import { xai } from '@ai-sdk/xai'; import { generateImage } from 'ai'; import { readFileSync } from 'fs'; const imageBuffer = readFileSync('./input-image.png'); const { images } = await generateImage({ model: xai.image('grok-imagine-image'), prompt: { text: 'Turn the cat into a golden retriever dog', images: [imageBuffer], }, }); ``` This example shows how to edit an image with xAI.

xAI multi-image editing example

```ts import { xai } from '@ai-sdk/xai'; import { generateImage } from 'ai'; import { readFileSync } from 'fs'; const cat = readFileSync('./cat.png'); const dog = readFileSync('./dog.png'); const { images } = await generateImage({ model: xai.image('grok-imagine-image'), prompt: { text: 'Combine these two animals into a group photo', images: [cat, dog], }, }); ``` This example shows how to edit multiple images together.

xAI style transfer example

```ts const imageBuffer = readFileSync('./input-image.png'); const { images } = await generateImage({ model: xai.image('grok-imagine-image'), prompt: { text: 'Transform this into a watercolor painting style', images: [imageBuffer], }, aspectRatio: '1:1', }); ``` This example shows how to apply style transfer with xAI.

xAI video generation models

xAI provides video generation via `.video()` factory method. Create models with `xai.video('grok-imagine-video')`. Use with `experimental_generateVideo()` function. Supports text-to-video, image-to-video, video editing, video extension, and reference-to-video (R2V) operations.

xAI text-to-video example

```ts import { xai, type XaiVideoModelOptions } from '@ai-sdk/xai'; import { experimental_generateVideo as generateVideo } from 'ai'; const { video } = await generateVideo({ model: xai.video('grok-imagine-video'), prompt: 'A chicken flying into the sunset in the style of 90s anime.', aspectRatio: '16:9', duration: 5, providerOptions: { xai: { user: 'user-123', pollTimeoutMs: 600000, // 10 minutes } satisfies XaiVideoModelOptions, }, }); ``` This example shows how to generate videos from text prompts.

xAI image-to-video example

```ts import { xai, type XaiVideoModelOptions } from '@ai-sdk/xai'; import { experimental_generateVideo as generateVideo } from 'ai'; const { video } = await generateVideo({ model: xai.video('grok-imagine-video'), prompt: { image: 'https://example.com/start-frame.png', text: 'The cat slowly turns its head and blinks', }, duration: 5, providerOptions: { xai: { pollTimeoutMs: 600000, } satisfies XaiVideoModelOptions, }, }); ``` This example shows how to generate videos from image input.

xAI video with frameImages example

```ts import { xai, type XaiVideoModelOptions } from '@ai-sdk/xai'; import { experimental_generateVideo as generateVideo } from 'ai'; import fs from 'node:fs'; const { video } = await generateVideo({ model: xai.video('grok-imagine-video'), prompt: 'The cat slowly turns its head and blinks', frameImages: [ { image: fs.readFileSync('./start-frame.png'), frameType: 'first_frame', }, ], duration: 5, providerOptions: { xai: { pollTimeoutMs: 600000, } satisfies XaiVideoModelOptions, }, }); ``` This example shows how to provide starting frame via frameImages.

xAI video first frame support

xAI does not support first-last-frame interpolation. A `last_frame` entry in `frameImages` is ignored with a warning. Use the `extend-video` mode to continue from a video's last frame instead.

xAI video editing example

```ts import { xai, type XaiVideoModelOptions } from '@ai-sdk/xai'; import { experimental_generateVideo as generateVideo } from 'ai'; const { video } = await generateVideo({ model: xai.video('grok-imagine-video'), prompt: 'Give the person sunglasses and a hat', providerOptions: { xai: { mode: 'edit-video', videoUrl: 'https://example.com/source-video.mp4', pollTimeoutMs: 600000, } satisfies XaiVideoModelOptions, }, }); ``` This example shows how to edit an existing video.

xAI video editing constraints

Video editing accepts input videos up to 8.7 seconds long. The `duration`, `aspectRatio`, and `resolution` parameters are not supported for editing - the output matches the input video's properties (capped at 720p).

xAI sequential video editing example

```ts import { xai, type XaiVideoModelOptions } from '@ai-sdk/xai'; import { experimental_generateVideo as generateVideo } from 'ai'; const providerOptions = { xai: { mode: 'edit-video', videoUrl: 'https://example.com/source-video.mp4', pollTimeoutMs: 600000, } satisfies XaiVideoModelOptions, }; const step1 = await generateVideo({ model: xai.video('grok-imagine-video'), prompt: 'Add a party hat to the person', providerOptions, }); const step1VideoUrl = step1.providerMetadata?.xai?.videoUrl as string; const [withSunglasses, withScarf] = await Promise.all([ generateVideo({ model: xai.video('grok-imagine-video'), prompt: 'Add sunglasses', providerOptions: { xai: { mode: 'edit-video', videoUrl: step1VideoUrl, pollTimeoutMs: 600000, }, }, }), generateVideo({ model: xai.video('grok-imagine-video'), prompt: 'Add a scarf', providerOptions: { xai: { mode: 'edit-video', videoUrl: step1VideoUrl, pollTimeoutMs: 600000, }, }, }), ]); ``` This example shows how to chain sequential edits or branch into concurrent edits using the xAI-hosted video URL from providerMetadata.

xAI video extension example

```ts import { xai, type XaiVideoModelOptions } from '@ai-sdk/xai'; import { experimental_generateVideo as generateVideo } from 'ai'; const source = await generateVideo({ model: xai.video('grok-imagine-video'), prompt: 'A cat sitting on a sunlit windowsill, tail gently swishing.', duration: 5, aspectRatio: '16:9', providerOptions: { xai: { pollTimeoutMs: 600000, } satisfies XaiVideoModelOptions, }, }); const sourceUrl = source.providerMetadata?.xai?.videoUrl as string; const extended = await generateVideo({ model: xai.video('grok-imagine-video'), prompt: 'The cat turns its head, notices a butterfly, and leaps off.', duration: 6, providerOptions: { xai: { mode: 'extend-video', videoUrl: sourceUrl, pollTimeoutMs: 600000, } satisfies XaiVideoModelOptions, }, }); ``` This example shows how to extend an existing video from its last frame.

xAI video extension constraints

Video extension does not support custom `aspectRatio` or `resolution` — the output inherits those from the source video. `duration` is supported and controls how long the extension is (not the total video length).

xAI reference-to-video (R2V) example

```ts import { xai, type XaiVideoModelOptions } from '@ai-sdk/xai'; import { experimental_generateVideo as generateVideo } from 'ai'; const { video } = await generateVideo({ model: xai.video('grok-imagine-video'), prompt: 'The comic cat from <IMAGE_1> and the comic dog from <IMAGE_2> are having a playful chase through a sunlit park. Cinematic slow-motion, warm afternoon light.', duration: 8, aspectRatio: '16:9', providerOptions: { xai: { mode: 'reference-to-video', referenceImageUrls: [ 'https://example.com/comic-cat.png', 'https://example.com/comic-dog.png', ], pollTimeoutMs: 600000, } satisfies XaiVideoModelOptions, }, }); ``` This example shows how to use reference-to-video with image URLs.

xAI reference-to-video with inputReferences example

```ts import { xai, type XaiVideoModelOptions } from '@ai-sdk/xai'; import { experimental_generateVideo as generateVideo } from 'ai'; import fs from 'node:fs'; const { video } = await generateVideo({ model: xai.video('grok-imagine-video'), prompt: 'The comic cat and the comic dog are having a playful chase through a sunlit park. Cinematic slow-motion, warm afternoon light.', inputReferences: [ fs.readFileSync('./comic-cat.png'), fs.readFileSync('./comic-dog.png'), ], duration: 8, aspectRatio: '16:9', providerOptions: { xai: { pollTimeoutMs: 600000, } satisfies XaiVideoModelOptions, }, }); ``` This example shows how to use reference-to-video with file data.

xAI reference audio in R2V example

```ts import { xai, type XaiVideoModelOptions } from '@ai-sdk/xai'; import { experimental_generateVideo as generateVideo } from 'ai'; const { video } = await generateVideo({ model: xai.video('grok-imagine-video-1.5'), prompt: 'The person from <IMAGE_0> stands in the room from <IMAGE_1> and speaks to the camera with the voice from <AUDIO_0>.', aspectRatio: '9:16', duration: 10, providerOptions: { xai: { mode: 'reference-to-video', referenceImageUrls: [ 'https://example.com/person.png', 'https://example.com/room.png', ], referenceVoiceIds: ['eve'], resolution: '720p', pollTimeoutMs: 600000, } satisfies XaiVideoModelOptions, }, }); ``` This example shows how to use reference audio with reference-to-video.

xAI video generation async nature

Video generation is an asynchronous process that can take several minutes. Consider setting `pollTimeoutMs` to at least 10 minutes (600000ms) for reliable operation. Generated video URLs are ephemeral and should be downloaded promptly.

Give your agent this brain