ChatStream Chat Completion

Stream Chat Completion

Chat completion can sometimes take a long time to finish, especially when the response is big. In such cases, it is useful to stream the chat completion to the client in real-time. This allows the client to display the new message as it is being generated by the model, rather than have users wait for it to finish.

User: How is it going?
Assistant: All good, how may I help you?
Why is the sky blue?
Send Message


Let's create a React component that imports the useChat hook from the ai/react module. The useChat hook will call the /api/chat endpoint when the user sends a message. The endpoint will generate the assistant's response based on the conversation history and stream it to the client.

import { useChat } from 'ai/react';
export default function Page() {
const { messages, input, setInput, append } = useChat();
return (
onChange={event => {
onKeyDown={async event => {
if (event.key === 'Enter') {
append({ content: input, role: 'user' });
{, index) => (
<div key={index}>{message.content}</div>


Next, let's create the /api/chat endpoint that generates the assistant's response based on the conversation history.

import { CoreMessage, streamText } from 'ai';
import { openai } from '@ai-sdk/openai';
export async function POST(req: Request) {
const { messages }: { messages: CoreMessage[] } = await req.json();
const result = await streamText({
model: openai('gpt-4'),
system: 'You are a helpful assistant.',
return result.toAIStreamResponse();

View Example on GitHub