Skip to content
Docs
GoogleGenerativeAIStream

GoogleGenerativeAIStream

GoogleGenerativeAIStream(res: Response, cb?: AIStreamCallbacks): ReadableStream

The GoogleGenerativeAIStream function is a utility that transforms the output from Google's Generative AI SDK (opens in a new tab) into a ReadableStream. It uses AIStream under the hood, applying a specific parser for the Google's response data structure.

This works with the official Generative AI SDK, and it's supported in both Node.js, Edge Runtime (opens in a new tab), and browser environments.

Parameters

res: Response

The Response object returned by the request to the Google Generative AI SDK.

cb?: AIStreamCallbacks

This optional parameter can be an object containing callback functions to handle the start, each token, and completion of the AI response. In the absence of this parameter, default behavior is implemented.

Example

The GoogleGenerativeAIStream function can be coupled with a call to the Google Generative AI SDK to generate a readable stream of the completion. This stream can then facilitate the real-time consumption of AI outputs as they're being generated.

Here's a step-by-step example of how to implement this in Next.js:

app/api/completion/route.ts
import { GoogleGenerativeAI } from '@google/generative-ai';
import { GoogleGenerativeAIStream, Message, StreamingTextResponse } from 'ai';
 
const genAI = new GoogleGenerativeAI(process.env.GOOGLE_API_KEY || '');
 
// IMPORTANT! Set the runtime to edge
export const runtime = 'edge';
 
// convert messages from the Vercel AI SDK Format to the format
// that is expected by the Google GenAI SDK
const buildGoogleGenAIPrompt = (messages: Message[]) => ({
  contents: messages
    .filter(message => message.role === 'user' || message.role === 'assistant')
    .map(message => ({
      role: message.role === 'user' ? 'user' : 'model',
      parts: [{ text: message.content }],
    })),
});
 
export async function POST(req: Request) {
  // Extract the `prompt` from the body of the request
  const { messages } = await req.json();
 
  const geminiStream = await genAI
    .getGenerativeModel({ model: 'gemini-pro' })
    .generateContentStream(buildGoogleGenAIPrompt(messages));
 
  // Convert the response into a friendly text-stream
  const stream = GoogleGenerativeAIStream(geminiStream);
 
  // Respond with the stream
  return new StreamingTextResponse(stream);
}

In this example, the GoogleGenerativeAIStream function transforms the text generation stream from the Google Generative AI SDK into a ReadableStream of parsed result. This allows clients to consume AI outputs in real-time as they're generated, instead of waiting for the complete response.


© 2023 Vercel Inc.