AdvancedStopping Streams
Stopping Streams
Cancelling ongoing streams is often needed. For example, users might want to stop a stream when they realize that the response is not what they want.
The different parts of the Vercel AI SDK support cancelling streams in different ways.
AI SDK Core
The AI SDK functions have an abortSignal
argument that you can use to cancel a stream.
You would use this if you want to cancel a stream from the server side to the LLM API, e.g. by
forwarding the abortSignal
from the request.
import { openai } from '@ai-sdk/openai';import { streamText } from 'ai';
export async function POST(req: Request) { const { prompt } = await req.json();
const result = await streamText({ model: openai('gpt-4-turbo'), prompt, // forward the abort signal: abortSignal: req.signal, });
return result.toTextStreamResponse();}
AI SDK UI
The hooks, e.g. useChat
or useCompletion
, provide a stop
helper function that can be used to cancel a stream.
This will cancel the stream from the client side to the server.
'use client';
import { useCompletion } from 'ai/react';
export default function Chat() { const { input, completion, stop, isLoading, handleSubmit, handleInputChange, } = useCompletion();
return ( <div> {isLoading && ( <button type="button" onClick={() => stop()}> Stop </button> )} {completion} <form onSubmit={handleSubmit}> <input value={input} onChange={handleInputChange} /> </form> </div> );}
AI SDK RSC
The AI SDK RSC does not currently support stopping streams.