LangChainStream

LangChainStream(callbacks?: AIStreamCallbacks): { stream: ReadableStream, handlers: LangChainCallbacks }

LangChainStream is a utility function designed to facilitate the integration of LangChain (opens in a new tab) - a framework for engineering prompts for AI language models - with UI components and data streaming to the client side. Returns a stream and bag of LangChain BaseCallbackHandlerMethodsClass that automatically implement streaming in such a way that you can use useChat and useCompletion.

Parameters

callbacks?: AIStreamCallbacks

This optional parameter can be an object containing callback functions that handle the start of the process, each new token, and the completion of the stream interaction. If not provided, no default behavior is implemented.

The AIStreamCallbacks object has the following properties:

• onStart?: () => Promise<void>: A function that is called at the start of the process.
• onToken?: (token: string) => Promise<void>: A function that is called for each new token. The token is passed as a parameter.
• onCompletion?: (completion: string) => Promise<void>: A function that is called when a completion is complete. The full completion is passed as a parameter.
• onFinal?: (completion: string) => Promise<void>: A function that is called when a response is complete. The full completion is passed as a parameter.

Returns

LangChainStream returns an object with two properties:

• stream: ReadableStream: This is the readable stream that can be piped into another stream. This stream contains the results of the LangChain process.
• handlers: LangChainCallbacks: This object contains handlers that can be used to handle certain callbacks provided by LangChain.

The LangChainCallbacks object has properties which are compatible with LangChain's BaseCallbackHandlerMethodsClass (opens in a new tab)

Example

import { StreamingTextResponse, LangChainStream, Message } from 'ai';
import { ChatOpenAI } from 'langchain/chat_models/openai';
import { AIMessage, HumanMessage } from 'langchain/schema';
 
export const runtime = 'edge';
 
export async function POST(req: Request) {
  const { messages } = await req.json();
 
  const { stream, handlers } = LangChainStream();
 
  const llm = new ChatOpenAI({
    streaming: true,
  });
 
  llm
    .call(
      (messages as Message[]).map(m =>
        m.role == 'user'
          ? new HumanMessage(m.content)
          : new AIMessage(m.content),
      ),
      {},
      [handlers],
    )
    .catch(console.error);
 
  return new StreamingTextResponse(stream);
}

In this example, LangChainStream is used to create a readable stream and handlers. The handlers are used to handle callbacks from the LangChain API. The stream is then used to stream the AI output to the client.