openai-compatible/setup

17 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.

OpenAI-compatible model class imports

Import these classes from '@ai-sdk/openai-compatible': OpenAICompatibleChatLanguageModel for chat models, OpenAICompatibleCompletionLanguageModel for completion models, OpenAICompatibleEmbeddingModel for embedding models, and OpenAICompatibleImageModel for image models.

Custom OpenAI-compatible provider file structure

A custom OpenAI-compatible provider package should have the following file structure: packages/example/ with src/ containing example-chat-settings.ts, example-completion-settings.ts, example-embedding-settings.ts, example-image-settings.ts, example-provider.ts, example-provider.test.ts, and index.ts; plus package.json, tsconfig.json, tsup.config.ts for build configuration, and README.md.

Chat model settings type definition in custom provider

Define chat model IDs in example-chat-settings.ts using a union type. The type should include all supported model ID strings as literals, plus (string & {}) to allow custom model IDs. Example: export type ExampleChatModelId = | 'example/chat-model-1' | 'example/chat-model-2' | (string & {});

Custom provider main function signature

The main export function (e.g., createExample) should accept ExampleProviderSettings as an optional parameter with empty object default, and return an ExampleProvider object that provides methods for creating chat, completion, embedding, and image models.

Provider interface methods for model creation

The ExampleProvider interface must include: a default call function that creates a chat model (modelId: ExampleChatModelId, settings?: ExampleChatSettings), chatModel method, completionModel method, embeddingModel method, and imageModel method. Each method takes a model ID and optional settings specific to that model type.

Common model configuration in custom provider

Create a CommonModelConfig interface with provider (string identifying the model type like 'example.chat'), url (function taking {path} parameter and returning full URL string, including baseURL and query parameters), headers (function returning Record<string, string> with authorization and custom headers), and optional fetch property.

Creating chat model instance in custom provider

Instantiate OpenAICompatibleChatLanguageModel with three arguments: modelId (the chat model identifier), settings (chat-specific settings object), and getCommonModelConfig('chat') for the common configuration.

API key loading from environment

Use loadApiKey function from '@ai-sdk/provider-utils' with parameters: apiKey (optional from settings), environmentVariableName (name of the environment variable like 'EXAMPLE_API_KEY'), and description (human-readable description of the API key).

Default base URL handling

Use withoutTrailingSlash function from '@ai-sdk/provider-utils' to process the baseURL. Set default to 'https://api.example.com/v1' if not provided in options, ensuring no trailing slash.

Query parameters in custom provider URL building

In the url function within CommonModelConfig, check if options.queryParams exists and append them using new URLSearchParams(options.queryParams).toString() to the URL search property.

Package.json dependencies for custom provider

A custom OpenAI-compatible provider should include these dependencies: @ai-sdk/openai-compatible (version ^0.0.7 or later), @ai-sdk/provider (version ^1.0.2 or later), and @ai-sdk/provider-utils (version ^2.0.4 or later).

Custom provider npm package naming convention

The npm package for a custom provider should follow the naming pattern @company-name/provider-name, for example @company-name/example.

Public exports from custom provider index

The index.ts file should export: createExample function, example default instance, and type definitions for ExampleProvider and ExampleProviderSettings.

Using a custom OpenAI-compatible provider

Users can use a published custom provider by importing the default instance or creation function and passing a model ID to it. Example: import { example } from '@company-name/example'; const { text } = await generateText({ model: example('example/chat-model-1'), prompt: 'Hello' });

Accessing internal API from openai-compatible package

Some internal utilities from the OpenAI Compatible package can be imported from '@ai-sdk/openai-compatible/internal', such as convertToOpenAICompatibleChatMessages. See the latest available exports in the AI SDK GitHub repository at https://github.com/vercel/ai/blob/main/packages/openai-compatible/src/internal/index.ts.

OpenAI Compatible Provider setup example

const provider = createOpenAICompatible({ name: 'providerName', apiKey: process.env.PROVIDER_API_KEY, baseURL: 'https://api.provider.com/v1', includeUsage: true, });

OpenAI Compatible Provider supported implementations

Detailed documentation is provided for the following OpenAI compatible providers: LM Studio, NIM, Heroku, Clarifai, and NEAR AI Cloud.

Give your agent this brain