google/capabilities

47 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.

Google safety ratings in response

Safety ratings provide insight into the safety of the model's response. Each rating includes: category (string), probability (string), probabilityScore (number), severity (string), severityScore (number), and optionally blocked (boolean).

Google Interactions API streamText support

`streamText` is supported with Interactions API. The stream's `finish` part exposes `interactionId` on `providerMetadata.google` for chaining.

Google language model with generateText

Google language models can be used with the `generateText` function from the `ai` package. Example: `const { text } = await generateText({ model: google('gemini-2.5-flash'), prompt: 'Write a vegetarian lasagna recipe for 4 people.' });`

Google language model streaming and structured output

Google language models can be used with the `streamText` function and support structured data generation with `Output` from the AI SDK Core.

Gemini 3+ models with thinkingLevel example

Example using thinkingLevel with Gemini 3.1 Pro: `const { text, reasoning } = await generateText({ model: google('gemini-3.1-pro-preview'), prompt: 'What is the sum of the first 10 prime numbers?', providerOptions: { google: { thinkingConfig: { thinkingLevel: 'high', includeThoughts: true } } satisfies GoogleLanguageModelOptions } }); console.log(text); console.log(reasoning);`

Gemini 2.5 models with thinkingBudget example

Example using thinkingBudget with Gemini 2.5 Flash: `const { text, reasoning } = await generateText({ model: google('gemini-2.5-flash'), prompt: 'What is the sum of the first 10 prime numbers?', providerOptions: { google: { thinkingConfig: { thinkingBudget: 8192, includeThoughts: true } } satisfies GoogleLanguageModelOptions } }); console.log(text); console.log(reasoning);`

Google file input support

Google provider supports file inputs including PDF files. Example: `const result = await generateText({ model: google('gemini-2.5-flash'), messages: [{ role: 'user', content: [{ type: 'text', text: 'What is an embedding model according to this document?' }, { type: 'file', data: fs.readFileSync('./data/ai.pdf'), mediaType: 'application/pdf' }] }] });`

Google YouTube URL support

Google provider supports YouTube URLs directly in file inputs. Example: `const result = await generateText({ model: google('gemini-2.5-flash'), messages: [{ role: 'user', content: [{ type: 'text', text: 'Summarize this video' }, { type: 'file', data: 'https://www.youtube.com/watch?v=dQw4w9WgXcQ', mediaType: 'video/mp4' }] }] });`

Google automatic URL downloading behavior

The AI SDK automatically downloads URLs if passed as data, except for `https://generativelanguage.googleapis.com/v1beta/files/` and YouTube URLs. YouTube URLs (public or unlisted videos) are supported directly—you can specify one YouTube video URL per request.

Google implicit caching for Gemini 2.5

Gemini 2.5 models automatically provide cache cost savings without explicit cache creation. When requests share common prefixes with previous requests, a 75% token discount is provided on cached content. Minimum token requirements: Gemini 2.5 Flash requires 1024 tokens minimum, Gemini 2.5 Pro requires 2048 tokens minimum.

Google implicit caching strategy

To maximize implicit cache hits: keep content at the beginning of requests consistent, add variable content (like user questions) at the end of prompts, and ensure requests meet minimum token requirements.

Google implicit caching example

Example of implicit caching: `const baseContext = 'You are a cooking assistant with expertise in Italian cuisine. Here are 1000 lasagna recipes for reference...'; const { text: veggieLasagna } = await generateText({ model: google('gemini-2.5-pro'), prompt: baseContext + '\n\nWrite a vegetarian lasagna recipe for 4 people.' }); const { text: meatLasagna, providerMetadata } = await generateText({ model: google('gemini-2.5-pro'), prompt: baseContext + '\n\nWrite a meat lasagna recipe for 12 people.' }); console.log('Cached tokens:', providerMetadata.google);`

Google explicit caching with Gemini 2.5

For guaranteed cost savings, explicit caching can be used with Gemini 2.5 and 2.0 models by creating a cache with `ai.caches.create()` and passing the cache name via the `cachedContent` provider option. The cache name format is `cachedContents/{cachedContent}`.

Google explicit caching example

Example of explicit caching: `import { GoogleGenAI } from '@google/genai'; const ai = new GoogleGenAI({ apiKey: process.env.GOOGLE_GENERATIVE_AI_API_KEY }); const cache = await ai.caches.create({ model: 'gemini-2.5-pro', config: { contents: [{ role: 'user', parts: [{ text: '1000 Lasagna Recipes...' }] }], ttl: '300s' } }); const { text } = await generateText({ model: google('gemini-2.5-pro'), prompt: 'Write a vegetarian lasagna recipe for 4 people.', providerOptions: { google: { cachedContent: cache.name } satisfies GoogleLanguageModelOptions } });`

Google usage metadata in provider metadata

Usage metadata including cachedContentTokenCount is available in `providerMetadata.google.usageMetadata` as of @ai-sdk/google@1.2.23. The usageMetadata object includes: cachedContentTokenCount, thoughtsTokenCount, promptTokenCount, candidatesTokenCount, totalTokenCount.

Google image output support

Gemini models with image generation capabilities (e.g. gemini-2.5-flash-image) support generating images as part of multimodal responses. Images are exposed as files in the response. Example: `const result = await generateText({ model: google('gemini-2.5-flash-image'), prompt: 'Create a picture of a nano banana dish in a fancy restaurant with a Gemini theme' }); for (const file of result.files) { if (file.mediaType.startsWith('image/')) { console.log('Generated image:', file); } }`

Google schema limitations for structured output

Google Generative AI API uses a subset of OpenAPI 3.0 schema which does not support features like unions. Errors appear as: 'GenerateContentRequest.generation_config.response_schema.properties[occupation].type: must be specified'. Known unsupported Zod features: z.union, z.record.

Google structuredOutputs false workaround

To handle schema limitations with unions or unsupported features, disable structured outputs: `await generateText({ model: google('gemini-2.5-flash'), providerOptions: { google: { structuredOutputs: false } satisfies GoogleLanguageModelOptions }, output: Output.object({ schema: z.object({ ... }) }) })` This allows using features like z.union as a workaround.

Google realtime custom events

Gemini lifecycle signals without provider-neutral equivalents are emitted as custom events. Inspect `rawType` for `goAway`, `sessionResumptionUpdate`, and `generationComplete` and use the event's `raw` payload for provider-specific details.

Google Interactions API endpoint

The Gemini Interactions API (`POST /v1beta/interactions`) is a separate Google endpoint with server-side state, unified content blocks, first-class built-in tools, agent presets, managed agents in sandboxed Linux environment, and native multimodal image output. It is reached via `google.interactions(...)` factory.

Google Interactions API basic usage

Example using Interactions API: `import { google } from '@ai-sdk/google'; const { text } = await generateText({ model: google.interactions('gemini-2.5-flash'), prompt: 'Hello, how are you?' });`

Google Interactions API factory shapes

`google.interactions(...)` accepts a model ID string (e.g. 'gemini-2.5-flash'), `{ agent: <name> }` to use a Gemini agent preset, or `{ managedAgent: <name> }` to invoke a managed agent created on Google's side.

Google Interactions API vs standard provider

Use `google(...)` for the standard `:generateContent` / `:streamGenerateContent` endpoints and `google.interactions(...)` for the new Interactions endpoint. Pick one per model instance as they target different request bodies and SSE event vocabularies.

Google Interactions API provider metadata fields

`result.providerMetadata.google` (typed via GoogleInteractionsProviderMetadata) exposes: interactionId (string) - server-side interaction id to pass back as previousInteractionId on next turn, serviceTier (string) - service tier the request actually ran on, signature (string) - per-block signature hash set by SDK on output reasoning and tool-call parts.

Google Interactions API stateful chaining example

Example of stateful chaining with default store: true: `const turn1 = await generateText({ model: google.interactions('gemini-2.5-flash'), prompt: 'What are the three largest cities in Spain?' }); const interactionId = turn1.providerMetadata?.google?.interactionId; const turn2 = await generateText({ model: google.interactions('gemini-2.5-flash'), prompt: 'What is the most famous landmark in the second one?', providerOptions: { google: { previousInteractionId: interactionId } satisfies GoogleLanguageModelInteractionsOptions } });`

Google Interactions API stateless multi-turn example

Example of stateless multi-turn with store: false: `const messages = [{ role: 'user', content: 'What are the three largest cities in Spain?' }]; const turn1 = await generateText({ model: google.interactions('gemini-2.5-flash'), messages, providerOptions: { google: { store: false } satisfies GoogleLanguageModelInteractionsOptions } }); messages.push(...turn1.responseMessages); messages.push({ role: 'user', content: 'What is the most famous landmark in the second one?' }); const turn2 = await generateText({ model: google.interactions('gemini-2.5-flash'), messages, providerOptions: { google: { store: false } satisfies GoogleLanguageModelInteractionsOptions } });`

Google Interactions API custom managed agents

For user-defined agents created via `/v1beta/agents` endpoint, pass the agent's name through `managedAgent` factory shape instead of `agent`. Example: `google.interactions({ managedAgent: 'my-custom-agent' })`

Google Interactions API image output example

Example generating images via Interactions: `const result = await generateText({ model: google.interactions('gemini-3-pro-image-preview'), prompt: 'Generate an image of a comic cat in a spaceship.', providerOptions: { google: { responseFormat: [{ type: 'image' }] } satisfies GoogleLanguageModelInteractionsOptions } }); for (const file of result.files) { if (file.mediaType.startsWith('image/')) { // file.uint8Array | file.base64 | file.mediaType } }`

Google Interactions API image with aspect ratio example

Example generating images with aspect ratio and size: `const result = await generateText({ model: google.interactions('gemini-3-pro-image-preview'), prompt: 'Generate a high-quality landscape photo of mountains at sunset.', providerOptions: { google: { responseFormat: [{ type: 'image', aspectRatio: '16:9', imageSize: '4K' }] } satisfies GoogleLanguageModelInteractionsOptions } });`

Google Interactions API multimodal output example

Example of multimodal output (text + image): `const result = await generateText({ model: google.interactions('gemini-2.5-flash-image'), prompt: 'Tell me a three sentence bedtime story about a unicorn, accompanied by a suitable illustration.', providerOptions: { google: { responseFormat: [{ type: 'text' }, { type: 'image', aspectRatio: '16:9' }] } satisfies GoogleLanguageModelInteractionsOptions } }); console.log(result.text); const images = result.files.filter(file => file.mediaType.startsWith('image/'));`

Google Interactions API image editing with stateful chaining example

Example of iterative image editing with stateful chaining: `const model = google.interactions('gemini-3-pro-image-preview'); const turn1 = await generateText({ model, prompt: 'Generate an image of a comic cat in a spaceship.', providerOptions: { google: { responseFormat: [{ type: 'image' }] } satisfies GoogleLanguageModelInteractionsOptions } }); const interactionId = turn1.providerMetadata?.google?.interactionId; const turn2 = await generateText({ model, prompt: 'now make the cat red', providerOptions: { google: { responseFormat: [{ type: 'image' }], previousInteractionId: interactionId } satisfies GoogleLanguageModelInteractionsOptions } });`

Google Interactions API agent presets

Pass `{ agent: <name> }` to target a Gemini agent preset. The factory type-checks the agent name. Example: `google.interactions({ agent: 'deep-research-pro-preview-12-2025' })`

Google Interactions API agent synchronous vs background

Agent execution type depends on the agent. Long-running presets (like deep-research-* family) require `background: true` — the POST returns non-terminal status and SDK polls internally. Other agents accept synchronous calls only and reject `background: true`. Set the flag explicitly via `providerOptions.google.background`.

Google Interactions API polling timeout for agents example

Example with increased polling timeout for slower agents: `await generateText({ model: google.interactions({ agent: 'deep-research-max-preview-04-2026' }), prompt: 'Produce a long-form research brief on ...', providerOptions: { google: { background: true, pollingTimeoutMs: 60 * 60 * 1000 // 1 hour } satisfies GoogleLanguageModelInteractionsOptions } });`

Google Interactions API managed agents

Managed agents run in a sandboxed Linux environment provisioned per interaction. Pass the `environment` provider option to control sandbox setup. The option is only accepted on agent calls.

Google Interactions API managed agent basic example

Example provisioning a fresh sandbox: `const result = await generateText({ model: google.interactions({ agent: 'antigravity-preview-05-2026' }), prompt: 'What is 2 + 2?', providerOptions: { google: { environment: 'remote' } satisfies GoogleLanguageModelInteractionsOptions } });`

Google Interactions API managed agent with sources example

Example provisioning sandbox with preloaded file: `await generateText({ model: google.interactions({ agent: 'antigravity-preview-05-2026' }), prompt: 'Read the file at /data/note.txt and tell me exactly what it contains.', providerOptions: { google: { environment: { type: 'remote', sources: [{ type: 'inline', content: 'hello from the AI SDK example\n', target: '/data/note.txt' }] } } satisfies GoogleLanguageModelInteractionsOptions } });`

Google Interactions API streamText example

Example using streamText with Interactions: `const result = streamText({ model: google.interactions('gemini-2.5-flash'), prompt: 'Hello, how are you?' }); for await (const textPart of result.textStream) { process.stdout.write(textPart); } const googleMetadata = (await result.providerMetadata)?.google; console.log('Interaction id:', googleMetadata?.interactionId);`

Google embedding API endpoint routing

The Google provider routes API calls based on embedding type. Single embeddings using `embed()` use the `:embedContent` endpoint which has higher rate limits. Multiple values using `embedMany()` or multiple values in `embed()` use the `:batchEmbedContents` endpoint.

Imagen model aspect ratio support

Imagen models support the following aspect ratios: 1:1, 3:4, 4:3, 9:16, 16:9. This applies to: imagen-4.0-generate-001, imagen-4.0-ultra-generate-001, imagen-4.0-fast-generate-001.

Gemini image models image editing support

Gemini image models (e.g. gemini-2.5-flash-image) support image editing by providing input images through the prompt object with text and images array. Images can be provided as file buffers or URLs.

Gemini 3 Flash supported thinking levels

Gemini 3 Flash supports the following thinking levels: 'minimal', 'low', 'medium', and 'high'.

Gemini 3 Pro supported thinking levels

Gemini 3 Pro supports the following thinking levels: 'low' and 'high'.

Google Search grounding tool with Gemini

Gemini can access the latest information using Google Search grounding via the google.tools.googleSearch() tool. This returns grounding metadata and safety ratings accessible through providerMetadata.google.groundingMetadata and providerMetadata.google.safetyRatings.

Tool calling with Gemini using generateText

Gemini 3 models support tool calling with improved reliability. Tools are defined using the tool() function with description and inputSchema properties, passed in a tools object to generateText. Multi-step tool calling is enabled via the stopWhen parameter with isStepCount(). The result object contains both text and steps properties.

Google Search grounding code example

Google Search grounding is enabled by including google.tools.googleSearch({}) in the tools object of generateText. The returned object contains text, sources, and providerMetadata properties. Grounding metadata is accessed via providerMetadata?.google?.groundingMetadata and safety ratings via providerMetadata?.google?.safetyRatings.

Gemini 3 capabilities overview

Gemini 3 delivers state-of-the-art reasoning with unprecedented depth and nuance, PhD-level performance on complex benchmarks (Humanity's Last Exam 37.5%, GPQA Diamond 91.9%), leading multimodal understanding (MMMU-Pro 81%, Video-MMMU 87.6%), best-in-class agentic capabilities, and superior long-horizon planning for multi-step workflows.

Give your agent this brain