Skip to content
Docs
AI Core (Experimental)
Custom Providers

Custom Providers

💡

The AI SDK Language Model Specification is experimental. It may change in the future without a major version bump.

The AI SDK provides a language model specification. You can write your own providers that adhere to the AI SDK language model specification and they will be compatible with the AI Core functions.

You can find the Language Model Specification in the AI SDK repository (opens in a new tab). It can be imported from '@ai-sdk/provider'.

We also provide utilities that make it easier to implement a custom provider. You can find them in the @ai-sdk/provider-utils package (source code (opens in a new tab)).

There are several reference implementations, e.g. a OpenAI reference implementation (opens in a new tab) and a Mistral reference implementation (opens in a new tab).

Provider Facade

A custom provider should follow the pattern of using a provider facade with factory methods for the specific providers. An instance of the custom provider class with default settings can be exported for convenience.

custom-provider-facade.ts
import { generateId, loadApiKey, withoutTrailingSlash } from ''@ai-sdk/provider-utils'';
import { CustomChatLanguageModel } from './custom-chat-language-model';
import { CustomChatModelId, CustomChatSettings } from './mistral-chat-settings';
 
/**
 * Custom provider facade.
 */
export class CustomProvider {
  readonly baseURL: string;
  readonly apiKey?: string;
 
  constructor(
    options: {
      baseURL?: string;
      apiKey?: string;
    } = {},
  ) {
    this.baseURL = withoutTrailingSlash(options.baseURL) ??
      'https://api.custom.ai/v1';
    this.apiKey = options.apiKey;
  }
 
  private get baseConfig() {
    return {
      baseURL: this.baseURL,
      headers: () => ({
        Authorization: `Bearer ${loadApiKey({
          apiKey: this.apiKey,
          environmentVariableName: 'CUSTOM_API_KEY',
          description: 'Custom Provider',
        })}`,
      }),
    };
  }
 
  chat(modelId: CustomChatModelId, settings: CustomChatSettings = {}) {
    return new CustomChatLanguageModel(modelId, settings, {
      provider: 'custom.chat',
      ...this.baseConfig,
    });
  }
}
 
/**
 * Default custom provider instance.
 */
export const customprovider = new CustomProvider();

Language Model Implementation

Please refer to the Language Model Specification in the AI SDK repository (opens in a new tab).

We also provide utilities that make it easier to implement a custom provider. You can find them in the @ai-sdk/provider-utils package (source code (opens in a new tab)).

There are several reference implementations, e.g. a OpenAI reference implementation (opens in a new tab) and a Mistral reference implementation (opens in a new tab).

Errors

The AI SDK provides standardized errors (opens in a new tab) that should be used by providers where possible. This will make it easy for user to debug them.

Retries, timeouts, and abort signals

The AI SDK will handle retries, timeouts, and aborting requests in a unified way. The model classes should not implement retries or timeouts themselves. Instead, they should use the abortSignal parameter to determine when the call should be aborted, and they should throw ApiCallErrors (or similar) with a correct isRetryable flag when errors such as network errors occur.


© 2023 Vercel Inc.