-
{#each $messages as message}
- {message.role}: {message.content} {/each}
{messages.map(m => (
))}
{m.role}
{m.content}
{messages.map(m => (
))}
{m.role}
{m.content.length > 0 ? ( m.content ) : ( {'calling tool: ' + m?.toolInvocations?.[0].toolName} )}
{messages.map(m => (
);
}
```
{m.role === 'user' ? 'User: ' : 'AI: '}
{m.content}
))}
{messages.map(m => (
);
}
```
In this code, you:
1. Create state to hold the files and create a ref to the file input field.
2. Display the "uploaded" files in the UI.
3. Update the `onSubmit` function, to call the `handleSubmit` function manually, passing the the files as an option using the `experimental_attachments` key.
4. Add a file input field to the form, including an `onChange` handler to handle updating the files state.
## Running Your Application
With that, you have built everything you need for your multi-modal chatbot! To start your application, use the command:
{m.role === 'user' ? 'User: ' : 'AI: '}
{m.content}
))}
{m?.experimental_attachments
?.filter(attachment =>
attachment?.contentType?.startsWith('image/'),
)
.map((attachment, index) => (
{message.role === 'user' ? 'User: ' : 'AI: '}
{message.content}
))}
);
}
```
The useChat hook on your root page (`app/page.tsx`) will make a request to your AI provider endpoint (`app/api/chat/route.ts`) whenever the user submits a message. The messages are then streamed back in real-time and displayed in the chat UI.
This enables a seamless chat experience where the user can see the AI response as soon as it is available, without having to wait for the entire response to be received.
### Going Beyond Text
The AI SDK's React Server Components (RSC) API enables you to create rich, interactive interfaces that go beyond simple text generation. With the [`streamUI`](/docs/reference/ai-sdk-rsc/stream-ui) function, you can dynamically stream React components from the server to the client.
Let's dive into how you can leverage tools with [AI SDK RSC](/docs/ai-sdk-rsc/overview) to build a generative user interface with Next.js (App Router).
First, create a Server Action.
```tsx filename="app/actions.tsx"
'use server';
import { streamUI } from 'ai/rsc';
import { createOpenAI as createGroq } from '@ai-sdk/openai';
import { z } from 'zod';
const groq = createGroq({
baseURL: 'https://api.groq.com/openai/v1',
apiKey: process.env.GROQ_API_KEY,
});
export async function streamComponent() {
const result = await streamUI({
model: groq('llama-3.1-70b-versatile'),
prompt: 'Get the weather for San Francisco',
text: ({ content }) => {content}
,
tools: {
getWeather: {
description: 'Get the weather for a location',
parameters: z.object({ location: z.string() }),
generate: async function* ({ location }) {
yield loading...
;
const weather = '25c'; // await getWeather(location);
return (
the weather in {location} is {weather}.
);
},
},
},
});
return result.value;
}
```
In this example, if the model decides to use the `getWeather` tool, it will first yield a `div` while fetching the weather data, then return a weather component with the fetched data (note: static data in this example). This allows for a more dynamic and responsive UI that can adapt based on the AI's decisions and external data.
On the frontend, you can call this Server Action like any other asynchronous function in your application. In this case, the function returns a regular React component.
```tsx filename="app/page.tsx"
'use client';
import { useState } from 'react';
import { streamComponent } from './actions';
export default function Page() {
const [component, setComponent] = useState{component}
{message.role === 'user' ? 'User: ' : 'AI: '}
{message.content}
))}
);
}
```
The useChat hook on your root page (`app/page.tsx`) will make a request to your AI provider endpoint (`app/api/chat/route.ts`) whenever the user submits a message. The messages are then displayed in the chat UI.
{message.role === 'user' ? 'User: ' : 'AI: '}
{message.content}
))}
);
}
```
```ts filename='app/api/chat/route.ts'
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
// Allow streaming responses up to 30 seconds
export const maxDuration = 30;
export async function POST(req: Request) {
const { messages } = await req.json();
const result = streamText({
model: openai('gpt-4-turbo'),
system: 'You are a helpful assistant.',
messages,
});
return result.toDataStreamResponse();
}
```
In the `Page` component, the `useChat` hook will request to your AI provider endpoint whenever the user submits a message.
The messages are then streamed back in real-time and displayed in the chat UI.
This enables a seamless chat experience where the user can see the AI response as soon as it is available,
without having to wait for the entire response to be received.
## Customized UI
`useChat` also provides ways to manage the chat message and input states via code, show loading and error states, and update messages without being triggered by user interactions.
### Loading State
The `isLoading` state returned by the `useChat` hook can be used for several
purposes
- To show a loading spinner while the chatbot is processing the user's message.
- To show a "Stop" button to abort the current message.
- To disable the submit button.
```tsx filename='app/page.tsx' highlight="6,20-27,34"
'use client';
import { useChat } from 'ai/react';
export default function Page() {
const { messages, input, handleInputChange, handleSubmit, isLoading, stop } =
useChat({});
return (
<>
{messages.map(message => (
{message.role === 'user' ? 'User: ' : 'AI: '}
{message.content}
))}
{isLoading && (
{messages.map(m => (
);
}
```
Please also see the [error handling](/docs/ai-sdk-ui/error-handling) guide for more information.
### Modify messages
Sometimes, you may want to directly modify some existing messages. For example, a delete button can be added to each message to allow users to remove them from the chat history.
The `setMessages` function can help you achieve these tasks:
```tsx
const { messages, setMessages, ... } = useChat()
const handleDelete = (id) => {
setMessages(messages.filter(message => message.id !== id))
}
return <>
{messages.map(message => (
{m.role}: {m.content}
))}
{error && (
<>
An error occurred.
)}
{message.role === 'user' ? 'User: ' : 'AI: '}
{message.content}
))}
...
```
You can think of `messages` and `setMessages` as a pair of `state` and `setState` in React.
### Controlled input
In the initial example, we have `handleSubmit` and `handleInputChange` callbacks that manage the input changes and form submissions. These are handy for common use cases, but you can also use uncontrolled APIs for more advanced scenarios such as form validation or customized components.
The following example demonstrates how to use more granular APIs like `setInput` and `append` with your custom input and submit button components:
```tsx
const { input, setInput, append } = useChat()
return <>
{messages.map(m => (
);
}
```
You can retrieve these custom fields on your server side by destructuring the request body:
```ts filename="app/api/chat/route.ts" highlight="3"
export async function POST(req: Request) {
// Extract addition information ("customKey") from the body of the request:
const { messages, customKey } = await req.json();
//...
}
```
## Controlling the response stream
With `streamText`, you can control how error messages and usage information are sent back to the client.
### Error Messages
By default, the error message is masked for security reasons.
The default error message is "An error occurred."
You can forward error messages or send your own error message by providing a `getErrorMessage` function:
```ts filename="app/api/chat/route.ts" highlight="13-27"
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
export async function POST(req: Request) {
const { messages } = await req.json();
const result = streamText({
model: openai('gpt-4o'),
messages,
});
return result.toDataStreamResponse({
getErrorMessage: error => {
if (error == null) {
return 'unknown error';
}
if (typeof error === 'string') {
return error;
}
if (error instanceof Error) {
return error.message;
}
return JSON.stringify(error);
},
});
}
```
### Usage Information
By default, the usage information is sent back to the client. You can disable it by setting the `sendUsage` option to `false`:
```ts filename="app/api/chat/route.ts" highlight="13"
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
export async function POST(req: Request) {
const { messages } = await req.json();
const result = streamText({
model: openai('gpt-4o'),
messages,
});
return result.toDataStreamResponse({
sendUsage: false,
});
}
```
### Text Streams
`useChat` can handle plain text streams by setting the `streamProtocol` option to `text`:
```tsx filename="app/page.tsx" highlight="7"
'use client';
import { useChat } from 'ai/react';
export default function Chat() {
const { messages } = useChat({
streamProtocol: 'text',
});
return <>...;
}
```
This configuration also works with other backend servers that stream plain text.
Check out the [stream protocol guide](/docs/ai-sdk-ui/stream-protocol) for more information.
{m.role}: {m.content}
))}
{messages.map(m => (
);
}
```
## Attachments (Experimental)
{m.role}: {m.content}
))}
{messages.map(message => (
))}
{`${message.role}: `}
{message.content}
{message.experimental_attachments
?.filter(attachment =>
attachment.contentType.startsWith('image/'),
)
.map((attachment, index) => (
))}
{messages.map(message => (
))}
{`${message.role}: `}
{message.content}
{message.experimental_attachments
?.filter(attachment =>
attachment.contentType?.startsWith('image/'),
)
.map((attachment, index) => (
))}
{m.role}:
{m.content}
{m.toolInvocations?.map((toolInvocation: ToolInvocation) => {
const toolCallId = toolInvocation.toolCallId;
const addResult = (result: string) =>
addToolResult({ toolCallId, result });
// render confirmation tool (client-side tool with user interaction)
if (toolInvocation.toolName === 'askForConfirmation') {
return (
))}
);
}
```
## Tool call streaming
{toolInvocation.args.message}
);
}
// other tools:
return 'result' in toolInvocation ? (
{'result' in toolInvocation ? (
{toolInvocation.result}
) : (
<>
)}
Tool call {`${toolInvocation.toolName}: `}
{toolInvocation.result}
) : (
Calling {toolInvocation.toolName}...
);
})}
{m.toolInvocations?.map((toolInvocation: ToolInvocation) => {
switch (toolInvocation.state) {
case 'partial-call':
return <>render partial tool call;
case 'call':
return <>render full tool call;
case 'result':
return <>render tool result;
}
})}
))}
);
}
```
## Server-side Multi-Step Calls
You can also use multi-step calls on the server-side with `streamText`.
This works when all invoked tools have an `execute` function on the server side.
```tsx filename='app/api/chat/route.ts' highlight="15-21,24"
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
import { z } from 'zod';
export async function POST(req: Request) {
const { messages } = await req.json();
const result = streamText({
model: openai('gpt-4-turbo'),
messages,
tools: {
getWeatherInformation: {
description: 'show the weather in a given city to the user',
parameters: z.object({ city: z.string() }),
// tool has execute function:
execute: async ({}: { city: string }) => {
const weatherOptions = ['sunny', 'cloudy', 'rainy', 'snowy', 'windy'];
return weatherOptions[
Math.floor(Math.random() * weatherOptions.length)
];
},
},
},
maxSteps: 5,
});
return result.toDataStreamResponse();
}
```
---
title: Generative User Interfaces
description: Learn how to build Generative UI with AI SDK UI.
---
# Generative User Interfaces
Generative user interfaces (generative UI) is the process of allowing a large language model (LLM) to go beyond text and "generate UI". This creates a more engaging and AI-native experience for users.
{messages.map(message => (
);
}
```
To handle the chat requests and model responses, set up an API route:
```ts filename="app/api/chat/route.ts"
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
export async function POST(request: Request) {
const { messages } = await request.json();
const result = streamText({
model: openai('gpt-4o'),
system: 'You are a friendly assistant!',
messages,
maxSteps: 5,
});
return result.toDataStreamResponse();
}
```
This API route uses the `streamText` function to process chat messages and stream the model's responses back to the client.
### Create a Tool
Before enhancing your chat interface with dynamic UI elements, you need to create a tool and corresponding React component. A tool will allow the model to perform a specific action, such as fetching weather information.
Create a new file called `ai/tools.ts` with the following content:
```ts filename="ai/tools.ts"
import { tool as createTool } from 'ai';
import { z } from 'zod';
export const weatherTool = createTool({
description: 'Display the weather for a location',
parameters: z.object({
location: z.string(),
}),
execute: async function ({ location }) {
await new Promise(resolve => setTimeout(resolve, 2000));
return { weather: 'Sunny', temperature: 75, location };
},
});
export const tools = {
displayWeather: weatherTool,
};
```
In this file, you've created a tool called `weatherTool`. This tool simulates fetching weather information for a given location. This tool will return simulated data after a 2-second delay. In a real-world application, you would replace this simulation with an actual API call to a weather service.
### Update the API Route
Update the API route to include the tool you've defined:
```ts filename="app/api/chat/route.ts" highlight="3,13"
import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
import { tools } from '@/ai/tools';
export async function POST(request: Request) {
const { messages } = await request.json();
const result = streamText({
model: openai('gpt-4o'),
system: 'You are a friendly assistant!',
messages,
maxSteps: 5,
tools,
});
return result.toDataStreamResponse();
}
```
Now that you've defined the tool and added it to your `streamText` call, let's build a React component to display the weather information it returns.
### Create UI Components
Create a new file called `components/weather.tsx`:
```tsx filename="components/weather.tsx"
type WeatherProps = {
temperature: number;
weather: string;
location: string;
};
export const Weather = ({ temperature, weather, location }: WeatherProps) => {
return (
{message.role === 'user' ? 'User: ' : 'AI: '}
{message.content}
))}
Current Weather for {location}
Condition: {weather}
Temperature: {temperature}°C
{messages.map(message => (
))}
);
}
```
In this updated code snippet, you:
1. Check if the message has `toolInvocations`.
2. Check if the tool invocation state is 'result'.
3. If it's a result and the tool name is 'displayWeather', render the Weather component.
4. If the tool invocation state is not 'result', show a loading message.
This approach allows you to dynamically render UI components based on the model's responses, creating a more interactive and context-aware chat experience.
## Expanding Your Generative UI Application
You can enhance your chat application by adding more tools and components, creating a richer and more versatile user experience. Here's how you can expand your application:
### Adding More Tools
To add more tools, simply define them in your `ai/tools.ts` file:
```ts
// Add a new stock tool
export const stockTool = createTool({
description: 'Get price for a stock',
parameters: z.object({
symbol: z.string(),
}),
execute: async function ({ symbol }) {
// Simulated API call
await new Promise(resolve => setTimeout(resolve, 2000));
return { symbol, price: 100 };
},
});
// Update the tools object
export const tools = {
displayWeather: weatherTool,
getStockPrice: stockTool,
};
```
Now, create a new file called `components/stock.tsx`:
```tsx
type StockProps = {
price: number;
symbol: string;
};
export const Stock = ({ price, symbol }: StockProps) => {
return (
{message.role === 'user' ? 'User: ' : 'AI: '}
{message.content}
{message.toolInvocations?.map(toolInvocation => {
const { toolName, toolCallId, state } = toolInvocation;
if (state === 'result') {
if (toolName === 'displayWeather') {
const { result } = toolInvocation;
return (
{toolName === 'displayWeather' ? (
);
}
})}
Loading weather...
) : null}
Stock Information
Symbol: {symbol}
Price: ${price}
{messages.map(message => (
))}
);
}
```
By following this pattern, you can continue to add more tools and components, expanding the capabilities of your Generative UI application.
---
title: Completion
description: Learn how to use the useCompletion hook.
---
# Completion
The `useCompletion` hook allows you to create a user interface to handle text completions in your application. It enables the streaming of text completions from your AI provider, manages the state for chat input, and updates the UI automatically as new messages are received.
In this guide, you will learn how to use the `useCompletion` hook in your application to generate text completions and stream them in real-time to your users.
## Example
```tsx filename='app/page.tsx'
'use client';
import { useCompletion } from 'ai/react';
export default function Page() {
const { completion, input, handleInputChange, handleSubmit } = useCompletion({
api: '/api/completion',
});
return (
);
}
```
```ts filename='app/api/completion/route.ts'
import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';
// Allow streaming responses up to 30 seconds
export const maxDuration = 30;
export async function POST(req: Request) {
const { prompt }: { prompt: string } = await req.json();
const result = streamText({
model: openai('gpt-3.5-turbo'),
prompt,
});
return result.toDataStreamResponse();
}
```
In the `Page` component, the `useCompletion` hook will request to your AI provider endpoint whenever the user submits a message. The completion is then streamed back in real-time and displayed in the UI.
This enables a seamless text completion experience where the user can see the AI response as soon as it is available, without having to wait for the entire response to be received.
## Customized UI
`useCompletion` also provides ways to manage the prompt via code, show loading and error states, and update messages without being triggered by user interactions.
### Loading and error states
To show a loading spinner while the chatbot is processing the user's message, you can use the `isLoading` state returned by the `useCompletion` hook:
```tsx
const { isLoading, ... } = useCompletion()
return(
<>
{isLoading ? {message.role}
{message.content}
{message.toolInvocations?.map(toolInvocation => {
const { toolName, toolCallId, state } = toolInvocation;
if (state === 'result') {
if (toolName === 'displayWeather') {
const { result } = toolInvocation;
return (
{toolName === 'displayWeather' ? (
);
}
})}
Loading weather...
) : toolName === 'getStockPrice' ? (
Loading stock price...
) : (
Loading...
)}
{error.message}
: null}
)
```
### Controlled input
In the initial example, we have `handleSubmit` and `handleInputChange` callbacks that manage the input changes and form submissions. These are handy for common use cases, but you can also use uncontrolled APIs for more advanced scenarios such as form validation or customized components.
The following example demonstrates how to use more granular APIs like `setInput` with your custom input and submit button components:
```tsx
const { input, setInput } = useCompletion();
return (
<>
{notification?.name}
{notification?.message}
{notification?.name}
{notification?.message}
An error occurred.
}
{object?.notifications?.map((notification, index) => (
{notification?.name}
{notification?.message}
{object?.notifications?.map((notification, index) => (
))}
);
}
```
## Configure Request Options
You can configure the API endpoint and optional headers using the `api` and `headers` settings.
```tsx highlight="2-5"
const { submit, object } = useObject({
api: '/api/use-object',
headers: {
'X-Custom-Header': 'CustomValue',
},
schema: yourSchema,
});
```
---
title: OpenAI Assistants
description: Learn how to use the useAssistant hook.
---
# OpenAI Assistants
The `useAssistant` hook allows you to handle the client state when interacting with an OpenAI compatible assistant API.
This hook is useful when you want to integrate assistant capabilities into your application,
with the UI updated automatically as the assistant is streaming its execution.
The `useAssistant` hook is supported in `ai/react`, `ai/svelte`, and `ai/vue`.
## Example
```tsx filename='app/page.tsx'
'use client';
import { Message, useAssistant } from 'ai/react';
export default function Chat() {
const { status, messages, input, submitMessage, handleInputChange } =
useAssistant({ api: '/api/assistant' });
return (
{notification?.name}
{notification?.message}
{messages.map((m: Message) => (
);
}
```
```tsx filename='app/api/assistant/route.ts'
import { AssistantResponse } from 'ai';
import OpenAI from 'openai';
const openai = new OpenAI({
apiKey: process.env.OPENAI_API_KEY || '',
});
// Allow streaming responses up to 30 seconds
export const maxDuration = 30;
export async function POST(req: Request) {
// Parse the request body
const input: {
threadId: string | null;
message: string;
} = await req.json();
// Create a thread if needed
const threadId = input.threadId ?? (await openai.beta.threads.create({})).id;
// Add a message to the thread
const createdMessage = await openai.beta.threads.messages.create(threadId, {
role: 'user',
content: input.message,
});
return AssistantResponse(
{ threadId, messageId: createdMessage.id },
async ({ forwardStream, sendDataMessage }) => {
// Run the assistant on the thread
const runStream = openai.beta.threads.runs.stream(threadId, {
assistant_id:
process.env.ASSISTANT_ID ??
(() => {
throw new Error('ASSISTANT_ID is not set');
})(),
});
// forward run status would stream message deltas
let runResult = await forwardStream(runStream);
// status can be: queued, in_progress, requires_action, cancelling, cancelled, failed, completed, or expired
while (
runResult?.status === 'requires_action' &&
runResult.required_action?.type === 'submit_tool_outputs'
) {
const tool_outputs =
runResult.required_action.submit_tool_outputs.tool_calls.map(
(toolCall: any) => {
const parameters = JSON.parse(toolCall.function.arguments);
switch (toolCall.function.name) {
// configure your tool calls here
default:
throw new Error(
`Unknown tool call function: ${toolCall.function.name}`,
);
}
},
);
runResult = await forwardStream(
openai.beta.threads.runs.submitToolOutputsStream(
threadId,
runResult.id,
{ tool_outputs },
),
);
}
},
);
}
```
## Customized UI
`useAssistant` also provides ways to manage the chat message and input states via code and show loading and error states.
### Loading and error states
To show a loading spinner while the assistant is running the thread, you can use the `status` state returned by the `useAssistant` hook:
```tsx
const { status, ... } = useAssistant()
return(
<>
{status === "in_progress" ?
{`${m.role}: `}
{m.role !== 'data' && m.content}
{m.role === 'data' && (
<>
{(m.data as any).description}
))}
{status === 'in_progress' && }
{JSON.stringify(m.data, null, 2)}
)}
{error.message}
: null}
)
```
### Controlled input
In the initial example, we have `handleSubmit` and `handleInputChange` callbacks that manage the input changes and form submissions. These are handy for common use cases, but you can also use uncontrolled APIs for more advanced scenarios such as form validation or customized components.
The following example demonstrates how to use more granular APIs like `append` with your custom input and submit button components:
```tsx
const { append } = useAssistant();
return (
<>
{m.annotations && <>{JSON.stringify(m.annotations)}}
))}
);
```
### Updating and Clearing Data
You can update and clear the `data` object of the `useChat` hook using the `setData` function.
```tsx
const { setData } = useChat();
// clear existing data
setData(undefined);
// set new data
setData([{ test: 'value' }]);
// transform existing data, e.g. adding additional values:
setData(currentData => [...currentData, { test: 'value' }]);
```
#### Example: Clear on Submit
```tsx highlight="18-21"
'use client';
import { Message, useChat } from 'ai/react';
export default function Chat() {
const { messages, input, handleInputChange, handleSubmit, data, setData } =
useChat();
return (
<>
{data && {JSON.stringify(data, null, 2)}}
{messages?.map((m: Message) => (
{`${m.role}: ${m.content}`}
))}
);
}
```
---
title: Error Handling
description: Learn how to handle errors in the AI SDK UI
---
# Error Handling
### Error Helper Object
Each AI SDK UI hook also returns an [error](/docs/reference/ai-sdk-ui/use-chat#error) object that you can use to render the error in your UI.
You can use the error object to show an error message, disable the submit button, or show a retry button.
{messages.map(m => (
);
}
```
#### Alternative: replace last message
Alternatively you can write a custom submit handler that replaces the last message when an error is present.
```tsx file="app/page.tsx" highlight="15-21,33"
'use client';
import { useChat } from 'ai/react';
export default function Chat() {
const {
handleInputChange,
handleSubmit,
error,
input,
messages,
setMesages,
} = useChat({});
function customSubmit(event: React.FormEvent
{m.role}: {m.content}
))}
{error && (
<>
An error occurred.
)}
{messages.map(m => (
);
}
```
### Error Handling Callback
Errors can be processed by passing an [`onError`](/docs/reference/ai-sdk-ui/use-chat#on-error) callback function as an option to the [`useChat`](/docs/reference/ai-sdk-ui/use-chat), [`useCompletion`](/docs/reference/ai-sdk-ui/use-completion) or [`useAssistant`](/docs/reference/ai-sdk-ui/use-assistant) hooks.
The callback function receives an error object as an argument.
```tsx file="app/page.tsx" highlight="8-11"
import { useChat } from 'ai/react';
export default function Page() {
const {
/* ... */
} = useChat({
// handle error:
onError: error => {
console.error(error);
},
});
}
```
### Injecting Errors for Testing
You might want to create errors for testing.
You can easily do so by throwing an error in your route handler:
```ts file="app/api/chat/route.ts"
export async function POST(req: Request) {
throw new Error('This is a test error');
}
```
---
title: Stream Protocols
description: Learn more about the supported stream protocols in the AI SDK.
---
# Stream Protocols
AI SDK UI functions such as `useChat` and `useCompletion` support both text streams and data streams.
The stream protocol defines how the data is streamed to the frontend on top of the HTTP protocol.
This page describes both protocols and how to use them in the backend and frontend.
You can use this information to develop custom backends and frontends for your use case, e.g.,
to provide compatible API endpoints that are implemented in a different language such as Python.
For instance, here's an example using [FastAPI](https://github.com/vercel/ai/tree/main/examples/next-fastapi) as a backend.
## Text Stream Protocol
A text stream contains chunks in plain text, that are streamed to the frontend.
Each chunk is then appended together to form a full text response.
Text streams are supported by `useChat`, `useCompletion`, and `useObject`.
When you use `useChat` or `useCompletion`, you need to enable text streaming
by setting the `streamProtocol` options to `text`.
You can generate text streams with `streamText` in the backend.
When you call `toTextStreamResponse()` on the result object,
a streaming HTTP response is returned.
{m.role}: {m.content}
))}
{error && An error occurred.
}
{content}
,
tools: {},
});
```
This example calls the `streamUI` function using OpenAI's `gpt-4o` model, passes a prompt, specifies how the model's plain text response (`content`) should be rendered, and then provides an empty object for tools. Even though this example does not define any tools, it will stream the model's response as a `div` rather than plain text.
### Adding A Tool
Using tools with `streamUI` is similar to how you use tools with `generateText` and `streamText`.
A tool is an object that has:
- `description`: a string telling the model what the tool does and when to use it
- `parameters`: a Zod schema describing what the tool needs in order to run
- `generate`: an asynchronous function that will be run if the model calls the tool. This must return a React component
Let's expand the previous example to add a tool.
```tsx highlight="6-14"
const result = await streamUI({
model: openai('gpt-4o'),
prompt: 'Get the weather for San Francisco',
text: ({ content }) => {content}
,
tools: {
getWeather: {
description: 'Get the weather for a location',
parameters: z.object({ location: z.string() }),
generate: async function* ({ location }) {
yield getting weather...
);
const getWeather = async (location: string) => {
await new Promise(resolve => setTimeout(resolve, 2000));
return '82°F️ ☀️';
};
interface WeatherProps {
location: string;
weather: string;
}
const WeatherComponent = (props: WeatherProps) => (
The weather in {props.location} is {props.weather}
);
export async function streamComponent() {
const result = await streamUI({
model: openai('gpt-4o'),
prompt: 'Get the weather for San Francisco',
text: ({ content }) => {content}
,
tools: {
getWeather: {
description: 'Get the weather for a location',
parameters: z.object({
location: z.string(),
}),
generate: async function* ({ location }) {
yield {component}
-
{messages.map(message => (
- {message.display} ))}
-
{messages.map(message => (
- {message.content} ))}
-
{messages.map(message => (
- {message.display} ))}
{content}
,
tools: {
searchFlights: {
description: 'search for flights',
parameters: z.object({
source: z.string().describe('The origin of the flight'),
destination: z.string().describe('The destination of the flight'),
date: z.string().describe('The date of the flight'),
}),
generate: async function* ({ source, destination, date }) {
yield `Searching for flights from ${source} to ${destination} on ${date}...`;
const results = await searchFlights(source, destination, date);
return (
{results.map(result => (
))}
);
},
},
lookupFlight: {
description: 'lookup details for a flight',
parameters: z.object({
flightNumber: z.string().describe('The flight number'),
}),
generate: async function* ({ flightNumber }) {
yield `Looking up details for flight ${flightNumber}...`;
const details = await lookupFlight(flightNumber);
return (
{result.flightNumber}
Flight Number: {details.flightNumber}
Departure Time: {details.departureTime}
Arrival Time: {details.arrivalTime}
{input}
,
]);
const message = await submitUserMessage(input);
setConversation(currentConversation => [...currentConversation, message]);
};
return (
{conversation.map((message, i) => (
{message}
))}
{flights.map(result => (
))}
);
};
```
Now, update your `searchFlights` tool to render the new ` {
const display = await submitUserMessage(
`lookupFlight ${result.flightNumber}`,
);
setMessages((messages: ReactNode[]) => [...messages, display]);
}}
>
{result.flightNumber}
Loading...
);
setTimeout(() => {
weatherUI.done(It's a sunny day!
);
}, 1000);
return weatherUI.value;
}
```
First, you create a streamable UI with an empty state and then update it with a loading message. After 1 second, you mark the stream as done passing in the actual weather information as it's final value. The `.value` property contains the actual UI that can be sent to the client.
## Reading a Streamable UI
On the client side, you can call the `getWeather` Server Action and render the returned UI like any other React component.
```tsx filename='app/page.tsx'
'use client';
import { useState } from 'react';
import { readStreamableValue } from 'ai/rsc';
import { getWeather } from '@/actions';
export default function Page() {
const [weather, setWeather] = useState
{weather}
);
}
```
When the button is clicked, the `getWeather` function is called, and the returned UI is set to the `weather` state and rendered on the page. Users will see the loading message first and then the actual weather information after 1 second.
Learn more about handling multiple streams in a single request in the [Multiple Streamables](/docs/advanced/multiple-streamables) guide.
Learn more about handling state for more complex use cases with [ AI/UI State ](/docs/ai-sdk-rsc/generative-ui-state).
---
title: Handling Loading State
description: Overview of handling loading state with AI SDK RSC
---
# Handling Loading State
{generation}
{generation}
loading...
;
return {content}
;
},
});
return result.value;
}
```
{generation}
loading
);
const data = await fetchData();
ui.done({data}
);
})().catch(e => {
ui.error(Error: {e.message}
);
});
return ui.value;
}
```
With this method, you can catch any error with the stream, and return relevant UI. On the client, you can also use a [React Error Boundary](https://react.dev/reference/react/Component#catching-rendering-errors-with-an-error-boundary) to wrap the streamed component and catch any additional errors.
```tsx filename='app/page.tsx'
import { getStreamedUI } from '@/actions';
import { useState } from 'react';
import { ErrorBoundary } from './ErrorBoundary';
export default function Page() {
const [streamedUI, setStreamedUI] = useState(null);
return (
{messages.map(message => message)}
);
}
```
The `streamUI` function combines generating text and rendering the user interface. To migrate to AI SDK UI, you need to **separate these concerns** – streaming generations with `streamText` and rendering the UI with `useChat`.
#### After: Replace server action with route handler
The `streamText` function executes as part of a route handler and streams the response to the client. The `useChat` hook on the client decodes this stream and renders the response within the chat interface.
```ts filename="@/app/api/chat/route.ts"
import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';
export async function POST(request) {
const { messages } = await request.json();
const result = streamText({
model: openai('gpt-4o'),
system: 'you are a friendly assistant!',
messages,
tools: {
// tool definitions
},
});
return result.toDataStreamResponse();
}
```
#### After: Update client to use chat hook
```tsx filename="@/app/page.tsx"
'use client';
import { useChat } from 'ai/react';
export default function Page() {
const { messages, input, setInput, handleSubmit } = useChat();
return (
{messages.map(message => (
))}
);
}
```
### Parallel Tool Calls
In AI SDK RSC, `streamUI` does not support parallel tool calls. You will have to use a combination of `streamText`, `createStreamableUI` and `createStreamableValue`.
With AI SDK UI, `useChat` comes with built-in support for parallel tool calls. You can define multiple tools in the `streamText` and have them called them in parallel. The `useChat` hook will then handle the parallel tool calls for you automatically.
### Multi-Step Tool Calls
In AI SDK RSC, `streamUI` does not support multi-step tool calls. You will have to use a combination of `streamText`, `createStreamableUI` and `createStreamableValue`.
With AI SDK UI, `useChat` comes with built-in support for multi-step tool calls. You can set `maxSteps` in the `streamText` function to define the number of steps the language model can make in a single call. The `useChat` hook will then handle the multi-step tool calls for you automatically.
### Generative User Interfaces
The `streamUI` function uses `tools` as a way to execute functions based on user input and renders React components based on the function output to go beyond text in the chat interface.
#### Before: Render components within the server action and stream to client
```tsx filename="@/app/actions.tsx"
import { z } from 'zod';
import { streamUI } from 'ai/rsc';
import { openai } from '@ai-sdk/openai';
import { getWeather } from '@/utils/queries';
import { Weather } from '@/components/weather';
const { value: stream } = await streamUI({
model: openai('gpt-4o'),
system: 'you are a friendly assistant!',
messages,
text: async function* ({ content, done }) {
// process text
},
tools: {
displayWeather: {
description: 'Display the weather for a location',
parameters: z.object({
latitude: z.number(),
longitude: z.number(),
}),
generate: async function* ({ latitude, longitude }) {
yield {message.role}
{message.content}
Loading weather...
;
const { value, unit } = await getWeather({ latitude, longitude });
return
{messages.map(message => (
))}
);
}
```
### Handling Client Interactions
With AI SDK RSC, components streamed to the client can trigger subsequent generations by calling the relevant server action using the `useActions` hooks. This is possible as long as the component is a descendant of the `{message.role}
{message.content}
{message.toolInvocations.map(toolInvocation => {
const { toolName, toolCallId, state } = toolInvocation;
if (state === 'result') {
const { result } = toolInvocation;
return (
{toolName === 'displayWeather' ? (
);
} else {
return (
{toolName === 'displayWeather' ? (
);
}
})}
Loading weather...
) : null}
{flights.map(flight => (
);
}
```
#### After: Use another chat hook with same ID from the component
After switching to AI SDK UI, these messages are synced by initializing the `useChat` hook in the component with the same `id` as the parent component.
```tsx filename="@/app/components/list-flights.tsx"
'use client';
import { useChat } from 'ai/react';
export function ListFlights({ chatId, flights }) {
const { append } = useChat({
id: chatId,
body: { id: chatId },
maxSteps: 5,
});
return (
{
const response = await sendMessage(
`I would like to choose flight ${flight.id}!`,
);
setMessages(msgs => [...msgs, response]);
}}
>
{flight.name}
))}
{flights.map(flight => (
);
}
```
### Loading Indicators
In AI SDK RSC, you can use the `initial` parameter of `streamUI` to define the component to display while the generation is in progress.
#### Before: Use `loading` to show loading indicator
```tsx filename="@/app/actions.tsx"
import { openai } from '@ai-sdk/openai';
import { streamUI } from 'ai/rsc';
const { value: stream } = await streamUI({
model: openai('gpt-4o'),
system: 'you are a friendly assistant!',
messages,
initial: {
await append({
role: 'user',
content: `I would like to choose flight ${flight.id}!`,
});
}}
>
{flight.name}
))}
Loading...
,
text: async function* ({ content, done }) {
// process text
},
tools: {
// tool definitions
},
});
return stream;
```
With AI SDK UI, you can use the tool invocation state to show a loading indicator while the tool is executing.
#### After: Use tool invocation state to show loading indicator
```tsx filename="@/app/components/message.tsx"
'use client';
export function Message({ role, content, toolInvocations }) {
return (
{role}
{content}
{toolInvocations && (
{toolInvocations.map(toolInvocation => {
const { toolName, toolCallId, state } = toolInvocation;
if (state === 'result') {
const { result } = toolInvocation;
return (
)}
{toolName === 'getWeather' ? (
);
} else {
return (
{toolName === 'getWeather' ? (
);
}
})}
Loading...
)}
{messages.map(message => (
))}
);
}
```
## Streaming Object Generation
The `createStreamableValue` function streams any serializable data from the server to the client. As a result, this function allows you to stream object generations from the server to the client when paired with `streamObject`.
#### Before: Use streamable value to stream object generations
```ts filename="@/app/actions.ts"
import { streamObject } from 'ai';
import { openai } from '@ai-sdk/openai';
import { createStreamableValue } from 'ai/rsc';
import { notificationsSchema } from '@/utils/schemas';
export async function generateSampleNotifications() {
'use server';
const stream = createStreamableValue();
(async () => {
const { partialObjectStream } = streamObject({
model: openai('gpt-4o'),
system: 'generate sample ios messages for testing',
prompt: 'messages from a family group chat during diwali, max 4',
schema: notificationsSchema,
});
for await (const partialObject of partialObjectStream) {
stream.update(partialObject);
}
})();
stream.done();
return { partialNotificationsStream: stream.value };
}
```
#### Before: Read streamable value and update object
```tsx filename="@/app/page.tsx"
'use client';
import { useState } from 'react';
import { readStreamableValue } from 'ai/rsc';
import { generateSampleNotifications } from '@/app/actions';
export default function Page() {
const [notifications, setNotifications] = useState(null);
return (
{message.role}
{message.content}
{object?.notifications?.map((notification, index) => (
))}
);
}
```
---
title: AI SDK RSC
description: Learn about AI SDK RSC.
collapsed: true
---
# AI SDK RSC
{notification?.name}
{notification?.message}
{isLoading && (
)}
{completion}
);
}
```
## AI SDK RSC
Loading weather...
);
forecastUI.update(Loading forecast...
);
getWeatherData().then(weatherData => {
weatherUI.done({weatherData}
);
});
getForecastData().then(forecastData => {
forecastUI.done({forecastData}
);
});
// Return both streamable UIs and other data fields.
return {
requestedAt: Date.now(),
weather: weatherUI.value,
forecast: forecastUI.value,
};
}
```
The client side code is similar to the previous example, but the [tool call](/docs/ai-sdk-core/tools-and-tool-calling) will return the new data structure with the weather and forecast UIs. Depending on the speed of getting weather and forecast data, these two components might be updated independently.
## Nested Streamable UIs
You can stream UI components within other UI components. This allows you to create complex UIs that are built up from smaller, reusable components. In the example below, we pass a `historyChart` streamable as a prop to a `StockCard` component. The StockCard can render the `historyChart` streamable, and it will automatically update as the server responds with new data.
```tsx file='app/actions.tsx'
async function getStockHistoryChart({ symbol: string }) {
'use server';
const ui = createStreamableUI(
{messages.map(message => {
if (message.role === 'function') {
const { name, content } = message
const { temperature, unit, description, forecast } = content;
return (
)
```
Here's a little preview of what that might look like.
{message.content}
);
}
```
## Rendering User Interfaces on the Server
The **AI SDK RSC (`ai/rsc`)** takes advantage of RSCs to solve the problem of managing all your React components on the client side, allowing you to render React components on the server and stream them to the client.
Rather than conditionally rendering user interfaces on the client based on the data returned by the language model, you can directly stream them from the server during a model generation.
```tsx highlight="3,22-31,38" filename="app/action.ts"
import { createStreamableUI } from 'ai/rsc'
const uiStream = createStreamableUI();
const text = generateText({
model: openai('gpt-3.5-turbo'),
system: 'you are a friendly assistant'
prompt: 'what is the weather in SF?'
tools: {
getWeather: {
description: 'Get the weather for a location',
parameters: z.object({
city: z.string().describe('The city to get the weather for'),
unit: z
.enum(['C', 'F'])
.describe('The unit to display the temperature in')
}),
execute: async ({ city, unit }) => {
const weather = getWeather({ city, unit })
const { temperature, unit, description, forecast } = weather
uiStream.done(
{messages.map(message => (
);
```
Now the steps involved are simplified:
1. The user prompts the language model.
2. The language model generates a response that includes a tool call.
3. The tool call renders a React component along with relevant props that represent the user interface.
4. The response is streamed to the client and rendered directly.
> **Note:** You can also render text on the server and stream it to the client using React Server Components. This way, all operations from language model generation to UI rendering can be done on the server, while the client only needs to render the UI that is streamed from the server.
Check out this [example](/examples/next-app/interface/stream-component-updates) for a full illustration of how to stream component updates with React Server Components in Next.js App Router.
---
title: Language Models as Routers
description: Generative User Interfaces and Language Models as Routers
---
# Generative User Interfaces
Since language models can render user interfaces as part of their generations, the resulting model generations are referred to as generative user interfaces.
In this section we will learn more about generative user interfaces and their impact on the way AI applications are built.
## Deterministic Routes and Probabilistic Routing
Generative user interfaces are not deterministic in nature because they depend on the model's generation output. Since these generations are probabilistic in nature, it is possible for every user query to result in a different user interface.
Users expect their experience using your application to be predictable, so non-deterministic user interfaces can sound like a bad idea at first. However, language models can be set up to limit their generations to a particular set of outputs using their ability to call functions.
When language models are provided with a set of function definitions and instructed to execute any of them based on user query, they do either one of the following things:
- Execute a function that is most relevant to the user query.
- Not execute any function if the user query is out of bounds of the set of functions available to them.
```tsx filename='app/actions.ts'
const sendMessage = (prompt: string) =>
generateText({
model: 'gpt-3.5-turbo',
system: 'you are a friendly weather assistant!',
prompt,
tools: {
getWeather: {
description: 'Get the weather in a location',
parameters: z.object({
location: z.string().describe('The location to get the weather for'),
}),
execute: async ({ location }: { location: string }) => ({
location,
temperature: 72 + Math.floor(Math.random() * 21) - 10,
}),
},
},
});
sendMessage('What is the weather in San Francisco?'); // getWeather is called
sendMessage('What is the weather in New York?'); // getWeather is called
sendMessage('What events are happening in London?'); // No function is called
```
This way, it is possible to ensure that the generations result in deterministic outputs, while the choice a model makes still remains to be probabilistic.
This emergent ability exhibited by a language model to choose whether a function needs to be executed or not based on a user query is believed to be models emulating "reasoning".
As a result, the combination of language models being able to reason which function to execute as well as render user interfaces at the same time gives you the ability to build applications where language models can be used as a router.
## Language Models as Routers
Historically, developers had to write routing logic that connected different parts of an application to be navigable by a user and complete a specific task.
In web applications today, most of the routing logic takes place in the form of routes:
- `/login` would navigate you to a page with a login form.
- `/user/john` would navigate you to a page with profile details about John.
- `/api/events?limit=5` would display the five most recent events from an events database.
While routes help you build web applications that connect different parts of an application into a seamless user experience, it can also be a burden to manage them as the complexity of applications grow.
Next.js has helped reduce complexity in developing with routes by introducing:
- File-based routing system
- Dynamic routing
- API routes
- Middleware
- App router, and so on...
With language models becoming better at reasoning, we believe that there is a future where developers only write core application specific components while models take care of routing them based on the user's state in an application.
With generative user interfaces, the language model decides which user interface to render based on the user's state in the application, giving users the flexibility to interact with your application in a conversational manner instead of navigating through a series of predefined routes.
### Routing by parameters
For routes like:
- `/profile/[username]`
- `/search?q=[query]`
- `/media/[id]`
that have segments dependent on dynamic data, the language model can generate the correct parameters and render the user interface.
For example, when you're in a search application, you can ask the language model to search for artworks from different artists. The language model will call the search function with the artist's name as a parameter and render the search results.
{message.display}
))}