Skip to content
Docs
CohereStream

CohereStream

CohereStream(res: Response, cb?: AIStreamCallbacks): ReadableStream

The CohereStream function is a utility that transforms the output from Cohere's (opens in a new tab) API into a ReadableStream. It uses AIStream under the hood, applying a specific parser for the Cohere's response data structure.

This works with the official Cohere API, and it's supported in both Node.js, the Edge Runtime (opens in a new tab), and browser environments.

Parameters

res: Response

The Response object returned by the request to the Cohere API.

cb?: AIStreamCallbacks

This optional parameter can be an object containing callback functions to handle the start, each token, and completion of the AI response. In the absence of this parameter, default behavior is implemented.

Example

The CohereStream function can be coupled with a fetch call to the Cohere API to generate a readable stream of the completion. This stream can then facilitate the real-time consumption of AI outputs as they're being generated.

Here's a step-by-step example of how to implement this in Next.js:

app/api/completion/route.ts
import { StreamingTextResponse, CohereStream } from 'ai';
 
export async function POST(req: Request) {
  // Extract the `prompt` from the body of the request
  const { prompt } = await req.json();
 
  const body = JSON.stringify({
    prompt,
    model: 'command-nightly',
    max_tokens: 300,
    stop_sequences: [],
    temperature: 0.9,
    return_likelihoods: 'NONE',
    stream: true,
  });
 
  const response = await fetch('https://api.cohere.ai/v1/generate', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      Authorization: `Bearer ${process.env.COHERE_API_KEY}`,
    },
    body,
  });
 
  // Check for errors
  if (!response.ok) {
    return new Response(await response.text(), {
      status: response.status,
    });
  }
 
  // Extract the text response from the Cohere stream
  const stream = CohereStream(response);
 
  // Respond with the stream
  return new StreamingTextResponse(stream);
}

In this example, the CohereStream function transforms the text generation stream from the Cohere API into a ReadableStream of parsed result. This allows clients to consume AI outputs in real-time as they're generated, instead of waiting for the complete response.


© 2023 Vercel Inc.