DeepInfra Provider

The DeepInfra provider contains support for state-of-the-art models through the DeepInfra API, including Llama 3, Mixtral, Qwen, and many other popular open-source models.

Setup

The DeepInfra provider is available via the @ai-sdk/deepinfra module. You can install it with:

pnpm add @ai-sdk/deepinfra

Provider Instance

You can import the default provider instance deepinfra from @ai-sdk/deepinfra:

import { deepinfra } from '@ai-sdk/deepinfra';

If you need a customized setup, you can import createDeepInfra from @ai-sdk/deepinfra and create a provider instance with your settings:

import { createDeepInfra } from '@ai-sdk/deepinfra';

const deepinfra = createDeepInfra({
  apiKey: process.env.DEEPINFRA_API_KEY ?? '',
});

You can use the following optional settings to customize the DeepInfra provider instance:

• baseURL string

  Use a different URL prefix for API calls, e.g. to use proxy servers. The default prefix is https://api.deepinfra.com/v1/openai.

• apiKey string

  API key that is being sent using the Authorization header. It defaults to the DEEPINFRA_API_KEY environment variable.

• headers Record<string,string>

  Custom headers to include in the requests.

• fetch (input: RequestInfo, init?: RequestInit) => Promise<Response>

  Custom fetch implementation. Defaults to the global fetch function. You can use it as a middleware to intercept requests, or to provide a custom fetch implementation for e.g. testing.

Language Models

You can create language models using a provider instance. The first argument is the model ID, for example:

import { deepinfra } from '@ai-sdk/deepinfra';
import { generateText } from 'ai';

const { text } = await generateText({
  model: deepinfra('meta-llama/Meta-Llama-3.1-70B-Instruct'),
  prompt: 'Write a vegetarian lasagna recipe for 4 people.',
});

DeepInfra language models can also be used in the streamText and streamUI functions (see AI SDK Core and AI SDK RSC).

Model Capabilities