Tool Calling

As covered under Foundations, tools are objects that can be called by the model to perform a specific task.

When used with AI SDK Core, tools contain three elements:

• description: An optional description of the tool that can influence when the tool is picked.
• parameters: A Zod schema that defines the parameters. It is converted to a JSON schema that is consumed by the LLM, and also used to validate the LLM tool calls.
• execute: An optional async function that is called with the arguments from the tool call. It produces a value of type RESULT (generic type). It is optional because you might want to forward tool calls to the client or to a queue instead of executing them in the same process.

The tools parameter of generateText and streamText is an object that has the tool names as keys and the tools as values:

import { z } from 'zod';
import { generateText, tool } from 'ai';

const result = await generateText({
  model: yourModel,
  tools: {
    weather: tool({
      description: 'Get the weather in a location',
      parameters: z.object({
        location: z.string().describe('The location to get the weather for'),
      }),
      execute: async ({ location }) => ({
        location,
        temperature: 72 + Math.floor(Math.random() * 21) - 10,
      }),
    }),
  },
  prompt:
    'What is the weather in San Francisco and what attractions should I visit?',
});

Tool calling is not restricted to only text generation. You can also use it to render user interfaces with Generative AI.

Tool Choice

You can use the toolChoice setting to influence when a tool is selected. It supports the following settings:

• auto (default): the model can choose whether and which tools to call.
• required: the model must call a tool. It can choose which tool to call.
• none: the model must not call tools
• { type: 'tool', toolName: string (typed) }: the model must call the specified tool

import { z } from 'zod';
import { generateText, tool } from 'ai';

const result = await generateText({
  model: yourModel,
  tools: {
    weather: tool({
      description: 'Get the weather in a location',
      parameters: z.object({
        location: z.string().describe('The location to get the weather for'),
      }),
      execute: async ({ location }) => ({
        location,
        temperature: 72 + Math.floor(Math.random() * 21) - 10,
      }),
    }),
  },
  toolChoice: 'required', // force the model to call a tool
  prompt:
    'What is the weather in San Francisco and what attractions should I visit?',
});

Tool Roundtrips

The large language model needs to know the tool results before it can continue generating text. This requires sending the tool results back to the model. You can enable this feature by setting the maxToolRoundtrips setting to a number greater than 0.

import { z } from 'zod';
import { generateText, tool } from 'ai';

const result = await generateText({
  model: yourModel,
  tools: {
    weather: tool({
      description: 'Get the weather in a location',
      parameters: z.object({
        location: z.string().describe('The location to get the weather for'),
      }),
      execute: async ({ location }) => ({
        location,
        temperature: 72 + Math.floor(Math.random() * 21) - 10,
      }),
    }),
  },
  maxToolRoundtrips: 5, // allow up to 5 tool roundtrips
  prompt:
    'What is the weather in San Francisco and what attractions should I visit?',
});

Pre-Built Tools

When you work with tools, you typically need a mix of application specific tools and general purpose tools. There are several providers that offer pre-built tools that you can use out of the box:

• agentic - A collection of 20+ tools. Most tools connect to access external APIs such as Exa or E2B.
• browserbase - Browser tool that runs a headless browser

Prompt Engineering with Tools

When you create prompts that include tools, getting good results can be tricky as the number and complexity of your tools increases.

Here are a few tips to help you get the best results:

1. Use a model that is strong at tool calling, such as gpt-4 or gpt-4-turbo. Weaker models will often struggle to call tools effectively and flawlessly.
2. Keep the number of tools low, e.g. to 5 or less.
3. Keep the complexity of the tool parameters low. Complex Zod schemas with many nested and optional elements, unions, etc. can be challenging for the model to work with.
4. Use semantically meaningful names for your tools, parameters, parameter properties, etc. The more information you pass to the model, the better it can understand what you want.
5. Add .describe("...") to your Zod schema properties to give the model hints about what a particular property is for.
6. When the output of a tool might be unclear to the model and there are dependencies between tools, use the description field of a tool to provide information about the output of the tool execution.
7. You can include example input/outputs of tool calls in your prompt to help the model understand how to use the tools. Keep in mind that the tools work with JSON objects, so the examples should use JSON.

In general, the goal should be to give the model all information it needs in a clear way.