AnthropicStream

AnthropicStream(res: Response, cb?: AIStreamCallbacks): ReadableStream

The AnthropicStream function is a utility that transforms the output from Anthropic's (opens in a new tab) SDK into a ReadableStream. It uses AIStream under the hood, applying a specific parser for the Anthropic's response data structure.

This works with the official Anthropic TypeScript SDK, and it's supported in both Node.js, Edge Runtime (opens in a new tab), and browser environments.

Parameters

res: Response

The Response object returned by the request to the Anthropic SDK.

cb?: AIStreamCallbacks

This optional parameter can be an object containing callback functions to handle the start, each token, and completion of the AI response. In the absence of this parameter, default behavior is implemented.

Example

The AnthropicStream function can be coupled with a call to the Anthropic SDK to generate a readable stream of the completion. This stream can then facilitate the real-time consumption of AI outputs as they're being generated.

Here's a step-by-step example of how to implement this in Next.js:

import Anthropic from '@anthropic-ai/sdk';
import { AnthropicStream, StreamingTextResponse } from 'ai';
import { experimental_buildAnthropicPrompt } from 'ai/prompts';
 
// Create an Anthropic API client (that's edge friendly??)
const anthropic = new Anthropic({
  apiKey: process.env.ANTHROPIC_API_KEY || '',
});
 
// IMPORTANT! Set the runtime to edge
export const runtime = 'edge';
 
export async function POST(req: Request) {
  // Extract the `prompt` from the body of the request
  const { messages } = await req.json();
 
  // Ask Claude for a streaming chat completion given the prompt
  const response = await anthropic.completions.create({
    prompt: experimental_buildAnthropicPrompt(messages),
    model: 'claude-2',
    stream: true,
    max_tokens_to_sample: 300,
  });
  // Convert the response into a friendly text-stream
  const stream = AnthropicStream(response);
  // Respond with the stream
  return new StreamingTextResponse(stream);
}

In this example, the AnthropicStream function transforms the text generation stream from the Anthropic SDK into a ReadableStream of parsed result. This allows clients to consume AI outputs in real-time as they're generated, instead of waiting for the complete response.