Azure OpenAI Provider
The Azure OpenAI provider contains language model support for the Azure OpenAI chat API.
Setup
The Azure OpenAI provider is available in the @ai-sdk/azure
module. You can install it with
pnpm add @ai-sdk/azure
Provider Instance
You can import the default provider instance azure
from @ai-sdk/azure
:
import { azure } from '@ai-sdk/azure';
If you need a customized setup, you can import createAzure
from @ai-sdk/azure
and create a provider instance with your settings:
import { createAzure } from '@ai-sdk/azure';
const azure = createAzure({ resourceName: 'your-resource-name', // Azure resource name apiKey: 'your-api-key',});
You can use the following optional settings to customize the OpenAI provider instance:
-
resourceName string
Azure resource name. It defaults to the
AZURE_RESOURCE_NAME
environment variable.The resource name is used in the assembled URL:
https://{resourceName}.openai.azure.com/openai/deployments/{modelId}{path}
. You can usebaseURL
instead to specify the URL prefix. -
apiKey string
API key that is being send using the
api-key
header. It defaults to theAZURE_API_KEY
environment variable. -
baseURL string
Use a different URL prefix for API calls, e.g. to use proxy servers.
Either this or
resourceName
can be used. When a baseURL is provided, the resourceName is ignored.With a baseURL, the resolved URL is
{baseURL}/{modelId}{path}
. -
headers Record<string,string>
Custom headers to include in the requests.
-
fetch (input: RequestInfo, init?: RequestInit) => Promise<Response>
Custom fetch implementation. Defaults to the global
fetch
function. You can use it as a middleware to intercept requests, or to provide a custom fetch implementation for e.g. testing.
Language Models
The Azure OpenAI provider instance is a function that you can invoke to create a language model:
const model = azure('your-deployment-name');
You need to pass your deployment name as the first argument.
Example
You can use OpenAI language models to generate text with the generateText
function:
import { azure } from '@ai-sdk/azure';import { generateText } from 'ai';
const { text } = await generateText({ model: azure('your-deployment-name'), prompt: 'Write a vegetarian lasagna recipe for 4 people.',});
OpenAI language models can also be used in the streamText
, generateObject
, streamObject
, and streamUI
functions
(see AI SDK Core and AI SDK RSC).
Azure OpenAI sends larger chunks than OpenAI. This can lead to the perception that the response is slower. See Troubleshooting: Azure OpenAI Slow To Stream
Chat Models
The URL for calling Azure chat models will be constructed as follows:
https://RESOURCE_NAME.openai.azure.com/openai/deployments/DEPLOYMENT_NAME/chat/completions?api-version=2024-08-01-preview
Azure OpenAI chat models support also some model specific settings that are not part of the standard call settings. You can pass them as an options argument:
const model = azure('your-deployment-name', { logitBias: { // optional likelihood for specific tokens '50256': -100, }, user: 'test-user', // optional unique user identifier});
The following optional settings are available for OpenAI chat models:
-
logitBias Record<number, number>
Modifies the likelihood of specified tokens appearing in the completion.
Accepts a JSON object that maps tokens (specified by their token ID in the GPT tokenizer) to an associated bias value from -100 to 100. You can use this tokenizer tool to convert text to token IDs. Mathematically, the bias is added to the logits generated by the model prior to sampling. The exact effect will vary per model, but values between -1 and 1 should decrease or increase likelihood of selection; values like -100 or 100 should result in a ban or exclusive selection of the relevant token.
As an example, you can pass
{"50256": -100}
to prevent the token from being generated. -
logProbs boolean | number
Return the log probabilities of the tokens. Including logprobs will increase the response size and can slow down response times. However, it can be useful to better understand how the model is behaving.
Setting to true will return the log probabilities of the tokens that were generated.
Setting to a number will return the log probabilities of the top n tokens that were generated.
-
parallelToolCalls boolean
Whether to enable parallel function calling during tool use. Default to true.
-
user string
A unique identifier representing your end-user, which can help OpenAI to monitor and detect abuse. Learn more.
Completion Models
You can create models that call the completions API using the .completion()
factory method.
The first argument is the model id.
Currently only gpt-35-turbo-instruct
is supported.
const model = azure.completion('your-gpt-35-turbo-instruct-deployment');
OpenAI completion models support also some model specific settings that are not part of the standard call settings. You can pass them as an options argument:
const model = azure.completion('your-gpt-35-turbo-instruct-deployment', { echo: true, // optional, echo the prompt in addition to the completion logitBias: { // optional likelihood for specific tokens '50256': -100, }, suffix: 'some text', // optional suffix that comes after a completion of inserted text user: 'test-user', // optional unique user identifier});
The following optional settings are available for Azure OpenAI completion models:
-
echo: boolean
Echo back the prompt in addition to the completion.
-
logitBias Record<number, number>
Modifies the likelihood of specified tokens appearing in the completion.
Accepts a JSON object that maps tokens (specified by their token ID in the GPT tokenizer) to an associated bias value from -100 to 100. You can use this tokenizer tool to convert text to token IDs. Mathematically, the bias is added to the logits generated by the model prior to sampling. The exact effect will vary per model, but values between -1 and 1 should decrease or increase likelihood of selection; values like -100 or 100 should result in a ban or exclusive selection of the relevant token.
As an example, you can pass
{"50256": -100}
to prevent the <|endoftext|> token from being generated. -
logProbs boolean | number
Return the log probabilities of the tokens. Including logprobs will increase the response size and can slow down response times. However, it can be useful to better understand how the model is behaving.
Setting to true will return the log probabilities of the tokens that were generated.
Setting to a number will return the log probabilities of the top n tokens that were generated.
-
suffix string
The suffix that comes after a completion of inserted text.
-
user string
A unique identifier representing your end-user, which can help OpenAI to monitor and detect abuse. Learn more.
Embedding Models
You can create models that call the Azure OpenAI embeddings API
using the .embedding()
factory method.
const model = azure.embedding('your-embedding-deployment');
Azure OpenAI embedding models support several aditional settings. You can pass them as an options argument:
const model = azure.embedding('your-embedding-deployment', { dimensions: 512 // optional, number of dimensions for the embedding user: 'test-user' // optional unique user identifier})
The following optional settings are available for Azure OpenAI embedding models:
-
dimensions: number
The number of dimensions the resulting output embeddings should have. Only supported in text-embedding-3 and later models.
-
user string
A unique identifier representing your end-user, which can help OpenAI to monitor and detect abuse. Learn more.