Skip to content
Docs
AI Core (Experimental)
streamText API

experimental_streamText

Streams text and call tools for a given prompt using a language model.

You can use the streamText function for interactive use cases such as chat bots (with and without tool usage), and text and code diff streaming in UIs. You can also generate UI components with tools (see Generative UI).

import { experimental_streamText } from 'ai';
 
const { textStream } = await experimental_streamText({
  model,
  prompt: 'Invent a new holiday and describe its traditions.',
});

experimental_streamText streams the output. If you do not want to stream the output, use experimental_generateText.

Parameters

The parameters are passed into experimental_streamText as a single options object.

  • model - The language model to use.
  • tools - The tools that the model can call. The model needs to support calling tools.
  • system - A system message that will be apart of the prompt.
  • prompt - A simple text prompt. You can either use prompt or messages but not both.
  • messages - A list of messages. You can either use prompt or messages but not both.
  • maxTokens - Maximum number of tokens to generate.
  • temperature - Temperature setting. This is a number between 0 (almost no randomness) and 1 (very random). It is recommended to set either temperature or topP, but not both.
  • topP - Nucleus sampling. This is a number between 0 and 1. E.g. 0.1 would mean that only tokens with the top 10% probability mass are considered. It is recommended to set either temperature or topP, but not both.
  • presencePenalty - Presence penalty setting. It affects the likelihood of the model to repeat information that is already in the prompt. The presence penalty is a number between -1 (increase repetition) and 1 (maximum penalty, decrease repetition). 0 means no penalty.
  • frequencyPenalty - Frequency penalty setting. It affects the likelihood of the model to repeatedly use the same words or phrases. The frequency penalty is a number between -1 (increase repetition) and 1 (maximum penalty, decrease repetition). 0 means no penalty.
  • seed - The seed (integer) to use for random sampling. If set and supported by the model, calls will generate deterministic results.
  • maxRetries - Maximum number of retries. Set to 0 to disable retries. Default: 2.
  • abortSignal - An optional abort signal that can be used to cancel the call.

Return Type

streamText returns a result object with several properties and methods:

  • textStream: AsyncIterable<string> & ReadableStream<string> - A stream of text deltas.
  • fullStream: AsyncIterable<TextStreamPart> & ReadableStream<TextStreamPart> - A stream of text stream parts:
    • text delta stream part with type = 'text-delta' and a textDelta property.
    • tool call stream part with type = 'tool-call' and toolCallId, toolName, and args properties.
    • tool result stream part with type = 'tool-result' and toolCallId, toolName, args, and result properties.
    • finish stream part with type = 'finish' and finishReason and usage properties. Sent as the last stream part.
    • error stream part with type = 'error' and an error property.
  • toAIStream(callbacks): convert the stream to an AI stream that is compatible with StreamingTextResponse.
  • warnings: Array<Warning> | undefined - Warnings from the model provider (e.g., unsupported settings).

You can only process one of the streams. Once it is processing, the other stream properties will throw errors.

Examples

Basic call

const { textStream } = await experimental_streamText({
  model,
  prompt: 'Invent a new holiday and describe its traditions.',
});

API Route for useCompletion

This example shows an App router (opens in a new tab) API route that is used by the useCompletion hook.

import { StreamingTextResponse, experimental_streamText } from 'ai';
import { openai } from '@ai-sdk/openai';
 
export const runtime = 'edge';
 
export async function POST(req: Request) {
  // Extract the `prompt` text from the body of the request
  const { prompt }: { prompt: string } = await req.json();
 
  const result = await experimental_streamText({
    model: openai.chat('gpt-4'),
    prompt,
  });
 
  return new StreamingTextResponse(result.toAIStream());
}

API Route for useChat

This example shows an App router (opens in a new tab) API route that is used by the useChat hook.

import { StreamingTextResponse, experimental_streamText } from 'ai';
import { openai } from '@ai-sdk/openai';
 
export const runtime = 'edge';
 
export async function POST(req: Request) {
  const { messages } = await req.json();
 
  const result = await experimental_streamText({
    model: openai.chat('gpt-4'),
    system: 'You are a helpful assistant.',
    messages,
  });
 
  return new StreamingTextResponse(result.toAIStream());
}

Terminal chatbot

import { ExperimentalMessage, experimental_streamText } from 'ai';
import { openai } from '@ai-sdk/openai';
import * as readline from 'node:readline/promises';
 
const terminal = readline.createInterface({
  input: process.stdin,
  output: process.stdout,
});
 
const messages: ExperimentalMessage[] = [];
 
async function main() {
  while (true) {
    const userInput = await terminal.question('You: ');
 
    messages.push({ role: 'user', content: userInput });
 
    const result = await experimental_streamText({
      model: openai.chat('gpt-3.5-turbo'),
      system: `You are a helpful, respectful and honest assistant.`,
      messages,
    });
 
    let fullResponse = '';
    process.stdout.write('\nAssistant: ');
    for await (const delta of result.textStream) {
      fullResponse += delta;
      process.stdout.write(delta);
    }
    process.stdout.write('\n\n');
 
    messages.push({ role: 'assistant', content: fullResponse });
  }
}
 
main().catch(console.error);

Terminal chatbot with tools

import {
  ExperimentalMessage,
  ToolCallPart,
  ToolResultPart,
  experimental_streamText,
} from 'ai';
import { openai } from '@ai-sdk/openai';
import * as readline from 'node:readline/promises';
 
const terminal = readline.createInterface({
  input: process.stdin,
  output: process.stdout,
});
 
const messages: ExperimentalMessage[] = [];
 
async function main() {
  let toolResponseAvailable = false;
 
  while (true) {
    if (!toolResponseAvailable) {
      const userInput = await terminal.question('You: ');
      messages.push({ role: 'user', content: userInput });
    }
 
    const result = await experimental_streamText({
      model: openai.chat('gpt-3.5-turbo'),
      tools: {
        weatherTool: {
          description: 'Get the weather in a location',
          parameters: z.object({
            location: z
              .string()
              .describe('The location to get the weather for'),
          }),
          execute: async ({ location }: { location: string }) => ({
            location,
            temperature: 72 + Math.floor(Math.random() * 21) - 10,
          }),
        },
      },
      system: `You are a helpful, respectful and honest assistant.`,
      messages,
    });
 
    toolResponseAvailable = false;
    let fullResponse = '';
    const toolCalls: ToolCallPart[] = [];
    const toolResponses: ToolResultPart[] = [];
 
    for await (const delta of result.fullStream) {
      switch (delta.type) {
        case 'text-delta': {
          if (fullResponse.length === 0) {
            process.stdout.write('\nAssistant: ');
          }
 
          fullResponse += delta.textDelta;
          process.stdout.write(delta.textDelta);
          break;
        }
 
        case 'tool-call': {
          toolCalls.push(delta);
 
          process.stdout.write(
            `\nTool call: '${delta.toolName}' ${JSON.stringify(delta.args)}`,
          );
          break;
        }
 
        case 'tool-result': {
          toolResponses.push(delta);
 
          process.stdout.write(
            `\nTool response: '${delta.toolName}' ${JSON.stringify(
              delta.result,
            )}`,
          );
          break;
        }
      }
    }
    process.stdout.write('\n\n');
 
    messages.push({
      role: 'assistant',
      content: [{ type: 'text', text: fullResponse }, ...toolCalls],
    });
 
    if (toolResponses.length > 0) {
      messages.push({ role: 'tool', content: toolResponses });
    }
 
    toolResponseAvailable = toolCalls.length > 0;
  }
}
 
main().catch(console.error);

© 2023 Vercel Inc.