openai/tools

37 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.

OpenAI programmatic tool calling authorization

Treat generated programs and tool arguments as untrusted. Enforce authorization and approval inside every tool, avoid programmatic access to side-effecting tools unless the operation is idempotent, and set explicit step limits.

OpenAI Responses API namespaced function calls

OpenAI supports grouping related function tools into namespaces. When the Responses API returns a function_call with a namespace, the OpenAI provider exposes this value on the generated tool-call part as providerMetadata.openai.namespace. When using streamText, the namespace is available on tool-input-end and tool-call events. The SDK uses providerMetadata.openai.namespace or providerOptions.openai.namespace to round-trip the namespace back to OpenAI on subsequent requests.

OpenAI programmatic tool calling support

OpenAI Programmatic Tool Calling allows supported Responses models to generate and run hosted JavaScript that coordinates client-side function tools. Add the hosted tool with openai.tools.programmaticToolCalling(), then opt each eligible function into programmatic calls with allowedCallers in providerOptions. The allowedCallers supports ['direct'] (default, only direct model tool calls), ['programmatic'] (only hosted JavaScript calls), or ['direct', 'programmatic'] (both modes). Use outputSchema to describe the JSON value returned by the function, sent to OpenAI as output_schema for generated JavaScript.

OpenAI web search tool configuration

The openai.tools.webSearch() tool supports configuration: externalWebAccess (boolean, defaults to true), searchContextSize ('low' | 'medium' | 'high'), userLocation (object with type 'approximate', country, city, region, timezone), filters (object with allowedDomains and blockedDomains arrays, up to 100 each). Domain filters should omit HTTP/HTTPS prefix, and subdomains are automatically included/excluded.

OpenAI file search tool filtering

The openai.tools.fileSearch() tool supports comparison filters (eq, ne, gt, gte, lt, lte, in, nin) and compound filters (and, or). Comparison filters have key, type, and value. Array values use the 'in' or 'nin' types. Compound filters nest multiple filters with 'and' or 'or' type.

OpenAI image generation tool configuration

The openai.tools.imageGeneration() tool supports outputFormat (e.g., 'webp') and quality ('low', 'high') parameters. When store is false, previously generated images will not be accessible by the model. It is recommended to use the image generation tool without setting store: false.

OpenAI code interpreter tool configuration

The openai.tools.codeInterpreter() tool can be configured with an optional container parameter that accepts either a container ID string or an object with fileIds array to specify uploaded files available to the code interpreter.

OpenAI MCP tool configuration

The openai.tools.mcp() tool requires serverLabel (string) and either serverUrl (string) or connectorId (string). Optional parameters include serverDescription (string), allowedTools (string[] or object with readOnly boolean and toolNames array), authorization (string for OAuth token), headers (Record<string, string>), and requireApproval ('always' | 'never' | object with never.toolNames array). When requireApproval is not set, tool calls are approved by default.

OpenAI MCP tool security warning

Connect to only trusted MCP servers and share data only with servers you trust. The OpenAI MCP tool is different from the general MCP client approach; it is a built-in provider-defined tool for OpenAI models to connect directly to MCP servers.

OpenAI computer tool configuration

The openai.tools.computer() tool accepts needsApproval callback and execute async callback. The execute callback receives actions (ordered array of click, double_click, drag, keypress, move, screenshot, scroll, type, wait actions), pendingSafetyChecks (array), and status. The callback returns object with output (computer_screenshot with imageUrl or fileId, and detail 'auto'|'low'|'high'|'original') and acknowledgedSafetyChecks array.

OpenAI computer tool action types and fields

Computer tool action types: click (button, x, y, optional keys), double_click (x, y, optional keys), drag (path array of {x, y} points, optional keys), keypress (keys), move (x, y, optional keys), screenshot (no fields), scroll (x, y, scrollX, scrollY, optional keys), type (text), wait (no fields).

OpenAI computer tool security warning

Run computer use in an isolated browser or VM. Restrict reachable domains, accounts, credentials, environment variables, and file-system access. Treat on-screen content as untrusted prompt-injection input, require confirmation immediately before consequential actions, and avoid exposing sensitive information in screenshots.

OpenAI local shell tool

The openai.tools.localShell() tool allows agents to run shell commands on a machine. It requires an execute async callback that receives action and returns output.

OpenAI shell tool environment modes

The openai.tools.shell() tool supports three environment modes: local (default, execute via callback), containerAuto (type 'containerAuto', OpenAI-hosted, no callback needed), and containerReference (type 'containerReference', existing container by ID).

OpenAI shell tool containerAuto configuration

The containerAuto environment supports fileIds (string[]), memoryLimit ('1g' | '4g' | '16g' | '64g'), and networkPolicy (object with type 'disabled' or 'allowlist' with allowedDomains string[] and optional domainSecrets array).

OpenAI shell tool execute callback return format

The execute callback for local shell execution must return an output array with results for each command: stdout (string), stderr (string), and outcome (object with type 'timeout' or {type: 'exit', exitCode: number}).

OpenAI shell tool skills support

Skills are versioned bundles of files with a SKILL.md manifest that extend shell tool capabilities. Container skills support skillReference format (skillId) or inline format (base64-encoded zip). Local skills point to a directory on disk containing a SKILL.md file.

OpenAI apply patch tool

The openai.tools.applyPatch() tool allows models to create, update, and delete files using structured diffs. The execute callback receives callId and operation, and must return status ('completed' | 'failed') and optional output (string).

OpenAI tool search hosted mode

Hosted tool search uses openai.tools.toolSearch() with no arguments. Mark tools with providerOptions.openai.deferLoading: true to make them searchable. OpenAI searches deferred tools and returns loaded subset in same response. Tool search items appear with execution: 'server' and call_id: null.

OpenAI tool search namespaces

Use providerOptions.openai.namespace (object with name and description) to group related function tools. Tools in the same namespace must use the same namespace name and description. Keep namespace descriptions concise and put detailed usage guidance on individual tools.

OpenAI tool search client-executed mode

Client-executed tool search passes execution: 'client', description, parameters schema, and execute callback to openai.tools.toolSearch(). The model emits tool_search_call with execution 'client' and non-null call_id. The execute callback receives arguments and returns tools array. The SDK sends tool_search_output back to model.

OpenAI custom tool configuration

The openai.tools.customTool() returns raw string output optionally constrained to grammar. Configuration includes description (string), format (object with type 'grammar'|'text', syntax 'regex'|'lark', definition string), and execute (async function receiving raw string input, returning string result).

OpenAI code interpreter file retrieval

Files generated by Code Interpreter can be retrieved using container_id and file_id available in providerMetadata via the Retrieve container file content API.

OpenAI programmatic tool calling example

Example using openai.tools.programmaticToolCalling() with generateText and stopWhen(isStepCount(10)). Define function tools with providerOptions.openai.allowedCallers: ['programmatic'] and outputSchema. The provider preserves program code, replay fingerprints, and nested function caller metadata.

OpenAI web search tool example

Example: generateText({ model: openai('gpt-5'), prompt: 'What happened in San Francisco last week?', tools: { web_search: openai.tools.webSearch({ externalWebAccess: true, searchContextSize: 'high', userLocation: { type: 'approximate', city: 'San Francisco', region: 'California' }, filters: { allowedDomains: ['sfchronicle.com', 'sfgate.com'] } }) } }). Access sources via result.sources or result.toolResults.

OpenAI file search tool example

Example: generateText({ model: openai('gpt-5'), prompt: 'What does the document say about user authentication?', tools: { file_search: openai.tools.fileSearch({ vectorStoreIds: ['vs_123'], maxNumResults: 5, filters: { key: 'author', type: 'eq', value: 'Jane Smith' }, ranking: { ranker: 'auto', scoreThreshold: 0.5 } }), providerOptions: { openai: { include: ['file_search_call.results'] } } }).

OpenAI image generation tool example

Example: generateText({ model: openai('gpt-5'), prompt: 'Generate an image of an echidna swimming across the Mozambique channel.', tools: { image_generation: openai.tools.imageGeneration({ outputFormat: 'webp' }) } }). Access base64Image from result.staticToolResults or stream part with type 'tool-result'.

OpenAI code interpreter tool example

Example: generateText({ model: openai('gpt-5'), prompt: 'Write and run Python code to calculate the factorial of 10', tools: { code_interpreter: openai.tools.codeInterpreter({ container: { fileIds: ['file-123', 'file-456'] } }) } }).

OpenAI MCP tool example

Example: generateText({ model: openai('gpt-5'), prompt: 'Search the web for the latest news about AI developments', tools: { mcp: openai.tools.mcp({ serverLabel: 'web-search', serverUrl: 'https://mcp.exa.ai/mcp', serverDescription: 'A web-search API for AI agents' }) } }).

OpenAI computer tool example

Example: generateText({ model: openai.responses('gpt-5.4'), tools: { computer: openai.tools.computer({ needsApproval: ({ pendingSafetyChecks }) => pendingSafetyChecks.length > 0, execute: async ({ actions, pendingSafetyChecks }) => { for (const action of actions) { await executeComputerAction(action); } return { output: { type: 'computer_screenshot', imageUrl: `data:image/png;base64,${await captureScreenshot()}` }, acknowledgedSafetyChecks: pendingSafetyChecks } } }) }, prompt: 'Open the settings page and enable dark mode.', stopWhen: isStepCount(10) }).

OpenAI shell tool local execution example

Example: generateText({ model: openai('gpt-5.2'), tools: { shell: openai.tools.shell({ execute: async ({ action }) => { return { output: results } } }) }, prompt: 'List the files in the current directory and show disk usage.' }).

OpenAI shell tool containerAuto example

Example: generateText({ model: openai('gpt-5.2'), tools: { shell: openai.tools.shell({ environment: { type: 'containerAuto', memoryLimit: '4g', fileIds: ['file-abc123'], networkPolicy: { type: 'allowlist', allowedDomains: ['example.com'] } } }) }, prompt: 'Install numpy and compute the eigenvalues of a 3x3 matrix.' }).

OpenAI apply patch tool example

Example: generateText({ model: openai('gpt-5.1'), tools: { apply_patch: openai.tools.applyPatch({ execute: async ({ callId, operation }) => { /* implementation for applying diffs */ } }) }, prompt: 'Create a python file that calculates the factorial of a number', stopWhen: isStepCount(5) }).

OpenAI tool search server-executed example

Example: generateText({ model: openai.responses('gpt-5.4'), prompt: 'What is the weather in San Francisco?', stopWhen: isStepCount(10), tools: { toolSearch: openai.tools.toolSearch(), get_weather: tool({ description: 'Get the current weather at a specific location', inputSchema: z.object({ location: z.string(), unit: z.enum(['celsius', 'fahrenheit']) }), execute: async ({ location, unit }) => ({ location, temperature: unit === 'celsius' ? 18 : 64 }), providerOptions: { openai: { deferLoading: true } } }) } }).

OpenAI tool search with namespace example

Example using providerOptions.openai.namespace with name 'crm' and description to group related tools. Multiple tools in same namespace must use identical namespace object. Mark tools with providerOptions.openai.deferLoading: true.

OpenAI tool search client-executed example

Example: generateText({ model: openai.responses('gpt-5.4'), prompt: 'What is the weather in San Francisco?', stopWhen: isStepCount(10), tools: { toolSearch: openai.tools.toolSearch({ execution: 'client', description: 'Search for available tools...', parameters: { type: 'object', properties: { goal: { type: 'string' } }, required: ['goal'] }, execute: async ({ arguments: args }) => { return { tools: [...] } } }) } }).

OpenAI custom tool example

Example: generateText({ model: openai.responses('gpt-5.2-codex'), tools: { write_sql: openai.tools.customTool({ description: 'Write a SQL SELECT query...', format: { type: 'grammar', syntax: 'regex', definition: 'SELECT .+' }, execute: async input => { const rows = await db.query(input); return JSON.stringify(rows); } }) }, toolChoice: 'required', prompt: 'Write a SQL query to get all users older than 25.', stopWhen: isStepCount(3) }).

Give your agent this brain