ChatStream Chat Completion

Stream Chat Completion

Chat completion can sometimes take a long time to finish, especially when the response is big. In such cases, it is useful to stream the chat completion to the client in real-time. This allows the client to display the new message as it is being generated by the model, rather than have users wait for it to finish.

User: How is it going?
Assistant: All good, how may I help you?
Why is the sky blue?
Send Message


Let's create a simple conversation between a user and a model, and place a button that will call continueConversation.

'use client';
import { useState } from 'react';
import { Message, continueConversation } from './actions';
import { readStreamableValue } from 'ai/rsc';
// Force the page to be dynamic and allow streaming responses up to 30 seconds
export const dynamic = 'force-dynamic';
export const maxDuration = 30;
export default function Home() {
const [conversation, setConversation] = useState<Message[]>([]);
const [input, setInput] = useState<string>('');
return (
{, index) => (
<div key={index}>
{message.role}: {message.content}
onChange={event => {
onClick={async () => {
const { messages, newMessage } = await continueConversation([
{ role: 'user', content: input },
let textContent = '';
for await (const delta of readStreamableValue(newMessage)) {
textContent = `${textContent}${delta}`;
{ role: 'assistant', content: textContent },
Send Message


Now, let's implement the continueConversation function that will insert the user's message into the conversation and stream back the new message.

'use server';
import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';
import { createStreamableValue } from 'ai/rsc';
export interface Message {
role: 'user' | 'assistant';
content: string;
export async function continueConversation(history: Message[]) {
'use server';
const stream = createStreamableValue();
(async () => {
const { textStream } = await streamText({
model: openai('gpt-3.5-turbo'),
"You are a dude that doesn't drop character until the DVD commentary.",
messages: history,
for await (const text of textStream) {
return {
messages: history,
newMessage: stream.value,