Google Vertex Provider

The Google Vertex provider for the AI SDK contains language model support for the Google Vertex AI APIs.

Setup

The Google provider is available in the @ai-sdk/google-vertex module. You can install it with

pnpm add @ai-sdk/google-vertex

Provider Instance

You can import the default provider instance vertex from @ai-sdk/google-vertex:

import { vertex } from '@ai-sdk/google-vertex';

If you need a customized setup, you can import createVertex from @ai-sdk/google-vertex and create a provider instance with your settings:

import { createVertex } from '@ai-sdk/google-vertex';

const vertex = createVertex({
  project: 'my-project', // optional
  location: 'us-central1', // optional
});

You can use the following optional settings to customize the Google Generative AI provider instance:

• project string

  The Google Cloud project ID that you want to use for the API calls. It uses the GOOGLE_VERTEX_PROJECT environment variable by default.

• location string

  The Google Cloud location that you want to use for the API calls, e.g. us-central1. It uses the GOOGLE_VERTEX_LOCATION environment variable by default.

• googleAuthOptions object

  Optional. The Authentication options used by the Google Auth Library:

  • authClient object An AuthClient to use.

  • keyFilename string Path to a .json, .pem, or .p12 key file.

  • keyFile string Path to a .json, .pem, or .p12 key file.

  • credentials object Object containing client_email and private_key properties, or the external account client options.

  • clientOptions object Options object passed to the constructor of the client.

  • scopes string | string[] Required scopes for the desired API request.

  • projectId string Your project ID.

  • universeDomain string The default service domain for a given Cloud universe.

Language Models

You can create models that call the Vertex API using the provider instance. The first argument is the model id, e.g. gemini-1.5-pro.

const model = vertex('gemini-1.5-pro');

Google Vertex models support also some model specific settings that are not part of the standard call settings. You can pass them as an options argument:

const model = vertex('gemini-1.5-pro', {
  safetySettings: [
    { category: 'HARM_CATEGORY_UNSPECIFIED', threshold: 'BLOCK_LOW_AND_ABOVE' },
  ],
});

The following optional settings are available for Google Vertex models:

• structuredOutputs boolean

  Optional. Enable structured output. Default is true.

  This is useful when the JSON Schema contains elements that are not supported by the OpenAPI schema version that Google Vertex uses. You can use this to disable structured outputs if you need to.

  See Troubleshooting: Schema Limitations for more details.

• safetySettings Array<{ category: string; threshold: string }>

  Optional. Safety settings for the model.

  • category string

    The category of the safety setting. Can be one of the following:

    • HARM_CATEGORY_UNSPECIFIED
    • HARM_CATEGORY_HATE_SPEECH
    • HARM_CATEGORY_DANGEROUS_CONTENT
    • HARM_CATEGORY_HARASSMENT
    • HARM_CATEGORY_SEXUALLY_EXPLICIT

  • threshold string

    The threshold of the safety setting. Can be one of the following:

    • HARM_BLOCK_THRESHOLD_UNSPECIFIED
    • BLOCK_LOW_AND_ABOVE
    • BLOCK_MEDIUM_AND_ABOVE
    • BLOCK_ONLY_HIGH
    • BLOCK_NONE

• useSearchGrounding boolean

  Optional. When enabled, the model will use Google search to ground the response.

You can use Google Vertex language models to generate text with the generateText function:

import { vertex } from '@ai-sdk/google-vertex';
import { generateText } from 'ai';

const { text } = await generateText({
  model: vertex('gemini-1.5-pro'),
  prompt: 'Write a vegetarian lasagna recipe for 4 people.',
});

Google Vertex language models can also be used in the streamText and streamUI functions (see AI SDK Core and AI SDK RSC).

File Inputs

The Google Vertex provider supports file inputs, e.g. PDF files.

import { vertex } from '@ai-sdk/google-vertex';
import { generateText } from 'ai';

const { text } = await generateText({
  model: vertex('gemini-1.5-pro'),
  messages: [
    {
      role: 'user',
      content: [
        {
          type: 'text',
          text: 'What is an embedding model according to this document?',
        },
        {
          type: 'file',
          data: fs.readFileSync('./data/ai.pdf'),
          mimeType: 'application/pdf',
        },
      ],
    },
  ],
});

See File Parts for details on how to use files in prompts.

Search Grounding

With search grounding, the model has access to the latest information using Google search. Search grounding can e.g. be used to provide answers around current events:

import { vertex } from '@ai-sdk/google-vertex';
import { generateText } from 'ai';

const { text, experimental_providerMetadata } = await generateText({
  model: vertex('gemini-1.5-pro', {
    useSearchGrounding: true,
  }),
  prompt:
    'List the top 5 San Francisco news from the past week.' +
    'You must include the date of each article.',
});

// access the grounding metadata
const groundingMetadata =
  experimental_providerMetadata?.vertex.groundingMetadata;

Troubleshooting

Schema Limitations

The Google Vertex API uses a subset of the OpenAPI 3.0 schema, which does not support features such as unions. The errors that you get in this case look like this:

GenerateContentRequest.generation_config.response_schema.properties[occupation].type: must be specified

By default, structured outputs are enabled (and for tool calling they are required). You can disable structured outputs for object generation as a workaround:

const result = await generateObject({
  model: vertex('gemini-1.5-pro', {
    structuredOutputs: false,
  }),
  schema: z.object({
    name: z.string(),
    age: z.number(),
    contact: z.union([
      z.object({
        type: z.literal('email'),
        value: z.string(),
      }),
      z.object({
        type: z.literal('phone'),
        value: z.string(),
      }),
    ]),
  }),
  prompt: 'Generate an example person for testing.',
});

Model Capabilities

Embedding Models

You can create models that call the Google Vertex AI embeddings API using the .textEmbeddingModel() factory method:

const model = vertex.textEmbeddingModel('text-embedding-004');

Google Vertex AI embedding models support additional settings. You can pass them as an options argument:

const model = vertex.textEmbeddingModel('text-embedding-004', {
  outputDimensionality: 512, // optional, number of dimensions for the embedding
});

The following optional settings are available for Google Vertex AI embedding models:

• outputDimensionality: number

  Optional reduced dimension for the output embedding. If set, excessive values in the output embedding are truncated from the end.

Model Capabilities