Skip to content
Docs
AssistantResponse

AssistantResponse

The AssistantResponse allows you to send a stream of assistant update to useAssistant.

AssistantResponse(settings: AssistantResponseSettings, process: AssistantResponseCallback): Response

The AssistantResponse is designed to facilitate streaming assistant responses to the useAssistant hook. It receives an assistant thread and a current message, and can send messages and data messages to the client.

Parameters

settings: {threadId: string, messageId: string}

You can pass the thread and the latest message into the AssistantResponse. This establishes the context for the response.

  • threadId: string: The thread ID that the response is associated with.
  • messageId: string: The ID of the latest message that the response is associated with.

process: AssistantResponseCallback

The process parameter is a callback in which you can run the assistant on threads, and send messages and data messages to the client.

It gets invoked with the following functions that you can use to send messages and data messages to the client:

  • forwardStream: (stream: AssistantStream) => Run | undefined: Forwards the assistant response stream to the client. Returns the Run object after it completes, or when it requires an action.
  • sendDataMessage: (message: DataMessage) => void: Send a data message to the client. You can use this to provide information for rendering custom UIs while the assistant is processing the thread.

Example

Server-Side Implementation

This example highlights the usage of AssistantResponse for an OpenAI assistant within a Next.js environment.

Server:

app/api/assistant/route.ts
import { AssistantResponse } from 'ai';
import OpenAI from 'openai';
 
// Create an OpenAI API client (that's edge friendly!)
const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY || '',
});
 
// IMPORTANT! Set the runtime to edge
export const runtime = 'edge';
 
const homeTemperatures = {
  bedroom: 20,
  'home office': 21,
  'living room': 21,
  kitchen: 22,
  bathroom: 23,
};
 
export async function POST(req: Request) {
  // Parse the request body
  const input: {
    threadId: string | null;
    message: string;
  } = await req.json();
 
  // Create a thread if needed
  const threadId = input.threadId ?? (await openai.beta.threads.create({})).id;
 
  // Add a message to the thread
  const createdMessage = await openai.beta.threads.messages.create(threadId, {
    role: 'user',
    content: input.message,
  });
 
  return AssistantResponse(
    { threadId, messageId: createdMessage.id },
    async ({ forwardStream, sendDataMessage }) => {
      // Run the assistant on the thread
      const runStream = openai.beta.threads.runs.createAndStream(threadId, {
        assistant_id:
          process.env.ASSISTANT_ID ??
          (() => {
            throw new Error('ASSISTANT_ID is not set');
          })(),
      });
 
      // forward run status would stream message deltas
      let runResult = await forwardStream(runStream);
 
      // status can be: queued, in_progress, requires_action, cancelling, cancelled, failed, completed, or expired
      while (
        runResult?.status === 'requires_action' &&
        runResult.required_action?.type === 'submit_tool_outputs'
      ) {
        const tool_outputs =
          runResult.required_action.submit_tool_outputs.tool_calls.map(
            (toolCall: any) => {
              const parameters = JSON.parse(toolCall.function.arguments);
 
              switch (toolCall.function.name) {
                case 'getRoomTemperature': {
                  const temperature =
                    homeTemperatures[
                      parameters.room as keyof typeof homeTemperatures
                    ];
 
                  return {
                    tool_call_id: toolCall.id,
                    output: temperature.toString(),
                  };
                }
 
                case 'setRoomTemperature': {
                  const oldTemperature =
                    homeTemperatures[
                      parameters.room as keyof typeof homeTemperatures
                    ];
 
                  homeTemperatures[
                    parameters.room as keyof typeof homeTemperatures
                  ] = parameters.temperature;
 
                  sendDataMessage({
                    role: 'data',
                    data: {
                      oldTemperature,
                      newTemperature: parameters.temperature,
                      description: `Temperature in ${parameters.room} changed from ${oldTemperature} to ${parameters.temperature}`,
                    },
                  });
 
                  return {
                    tool_call_id: toolCall.id,
                    output: `temperature set successfully`,
                  };
                }
 
                default:
                  throw new Error(
                    `Unknown tool call function: ${toolCall.function.name}`,
                  );
              }
            },
          );
 
        runResult = await forwardStream(
          openai.beta.threads.runs.submitToolOutputsStream(
            threadId,
            runResult.id,
            { tool_outputs },
          ),
        );
      }
    },
  );
}

Client-Side Setup

See use-assistant (opens in a new tab) for more details on how to setup the useAssistant hook on the client-side.


© 2023 Vercel Inc.