Community ProvidersWriting a Custom Provider

Writing a Custom Provider

The Vercel AI SDK provides a Language Model Specification. You can write your own provider that adheres to the specification and it will be compatible with the AI SDK.

Language Model Specification

You can find the Language Model Specification in the AI SDK repository. It can be imported from '@ai-sdk/provider'. We also provide utilities that make it easier to implement a custom provider. You can find them in the @ai-sdk/provider-utils package (source code). There are several reference implementations, e.g. a Mistral reference implementation.

Provider Facade

A custom provider should follow the pattern of using a provider facade with factory methods for the specific providers. An instance of the custom provider class with default settings can be exported for convenience.

import { generateId, loadApiKey, withoutTrailingSlash } from ''@ai-sdk/provider-utils'';
import { CustomChatLanguageModel } from './custom-chat-language-model';
import { CustomChatModelId, CustomChatSettings } from './mistral-chat-settings';
/**
* Custom provider facade.
*/
export class CustomProvider {
readonly baseURL: string;
readonly apiKey?: string;
constructor(
options: {
baseURL?: string;
apiKey?: string;
} = {},
) {
this.baseURL = withoutTrailingSlash(options.baseURL) ??
'https://api.custom.ai/v1';
this.apiKey = options.apiKey;
}
private get baseConfig() {
return {
baseURL: this.baseURL,
headers: () => ({
Authorization: `Bearer ${loadApiKey({
apiKey: this.apiKey,
environmentVariableName: 'CUSTOM_API_KEY',
description: 'Custom Provider',
})}`,
}),
};
}
chat(modelId: CustomChatModelId, settings: CustomChatSettings = {}) {
return new CustomChatLanguageModel(modelId, settings, {
provider: 'custom.chat',
...this.baseConfig,
});
}
}
/**
* Default custom provider instance.
*/
export const customprovider = new CustomProvider();

Errors

The AI SDK provides standardized errors that should be used by providers where possible. This will make it easy for user to debug them.

Retries, timeouts, and abort signals

The AI SDK will handle retries, timeouts, and aborting requests in a unified way. The model classes should not implement retries or timeouts themselves. Instead, they should use the abortSignal parameter to determine when the call should be aborted, and they should throw ApiCallErrors (or similar) with a correct isRetryable flag when errors such as network errors occur.