Skip to content
Docs
StreamData

StreamData

The StreamData class allows you to stream arbitrary data to the client alongside your LLM response. For information on the implementation, see the associated pull request (opens in a new tab).

Usage

On the Server

app/api/chat/route.ts
import { OpenAIStream, StreamingTextResponse, StreamData } from 'ai';
import OpenAI from 'openai';
import type { ChatCompletionCreateParams } from 'openai/resources/chat';
 
// Create an OpenAI API client (that's edge friendly!)
const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY || '',
});
 
export async function POST(req: Request) {
  const { messages } = await req.json();
 
  const response = await openai.chat.completions.create({
    model: 'gpt-3.5-turbo-0613',
    stream: true,
    messages,
    functions,
  });
 
  // Instantiate the StreamData. It works with all API providers.
  const data = new StreamData();
 
  const stream = OpenAIStream(response, {
    experimental_onFunctionCall: async (
      { name, arguments: args },
      createFunctionCallMessages,
    ) => {
      if (name === 'get_current_weather') {
        // Call a weather API here
        const weatherData = {
          temperature: 20,
          unit: args.format === 'celsius' ? 'C' : 'F',
        };
 
        data.append({
          text: 'Some custom data',
        });
 
        const newMessages = createFunctionCallMessages(weatherData);
        return openai.chat.completions.create({
          messages: [...messages, ...newMessages],
          stream: true,
          model: 'gpt-3.5-turbo-0613',
        });
      }
    },
    onCompletion(completion) {
      console.log('completion', completion);
    },
    onFinal(completion) {
      // IMPORTANT! you must close StreamData manually or the response will never finish.
      data.close();
    },
  });
 
  data.append({
    text: 'Hello, how are you?',
  });
 
  // IMPORTANT! If you aren't using StreamingTextResponse, you MUST have the `X-Experimental-Stream-Data: 'true'` header
  // in your response so the client uses the correct parsing logic.
  return new StreamingTextResponse(stream, {}, data);
}

On the client

In future versions, each Message will have a data object attached to it. For the initial implementation, the SDK only supports a global data returned by the useChat hook:

const { data } = useChat({
  api: '/api/chat',
});

And data is of the type JSONValue[]


© 2023 Vercel Inc.