Skip to content
Docs
InkeepStream

InkeepStream

InkeepStream(res: Response, cb?: AIStreamCallbacks): ReadableStream

The InkeepStream function is a utility that transforms the output from Inkeep's (opens in a new tab) API into a ReadableStream (opens in a new tab). It uses AIStream under the hood, applying a specific parser for the Inkeep's response data structure.

This works with the official Inkeep API, and it's supported in both Node.js, the Edge Runtime (opens in a new tab), and browser environments.

Parameters

res: Response

The Response object returned by the request to the Inkeep API.

cb?: InkeepAIStreamCallbacksAndOptions

This optional parameter can be an object containing the callback functions to handle the start, each token, completion, and other events of the AI response. In the absence of this parameter, default behavior is implemented.

The InkeepAIStreamCallbacksAndOptions extends the standard AIStreamCallbacks by (1) including additional metadata in onFinal and (2) adding an onRecordsCited callback.

OptionTypeDescription
onRecordsCited(records_cited: any) => voidAn optional function that is called once for every request, after the main content of a message has completed. It includes the information about the records (sources) cited in the AI chat response. It's the payload of the `records_cited` event from the Inkeep API.
onFinal(completion: string, metadata: InkeepOnFinalMetadata) => Promise<void>An optional function that is called once for every request. It's always the final callback invoked. It's passed the content of the chat response as a string and metadata as an object of type `InkeepOnFinalMetadata`.

Check the @inkeep/ai-api (opens in a new tab) SDK for the latest typings of the Inkeep APIs. For example, the records_cited data payload can be obtained as:

import type { RecordsCited$ } from '@inkeep/ai-api/models/components';

// RecordsCited$.Inbound

InkeepOnFinalMetadata

Information included in metadata of InkeepStream's onFinal callback.

OptionTypeDescription
chat_session_idstringThe Inkeep chat_session_id. This should be included in follow-up chat requests that are part of a chat session. Used for analytics and threading chat conversations.
records_citedanyContains information about the citations used in the chat response body.

Example

The Inkeep API provides two routes:

  1. POST chat_sessions/chat_results - To create a chat session
  2. POST chat_sessions/${chat_session_id}/chat_results - To continue a chat session

The example below shows how to create a chat API endpoint compatible with Vercel AI SDK client-side utilities like useChat:

app/api/chat/route.ts
import {
  InkeepStream,
  InkeepOnFinalMetadata,
  StreamingTextResponse,
  StreamData,
} from 'ai';
import { InkeepAI } from '@inkeep/ai-api';
import type { RecordsCited$ } from '@inkeep/ai-api/models/components';
 
interface ChatRequestBody {
  messages: Array<{
    role: 'user' | 'assistant';
    content: string;
  }>;
  chat_session_id?: string;
}
 
const inkeepIntegrationId = process.env.INKEEP_INTEGRATION_ID;
 
export async function POST(req: Request) {
  const chatRequestBody: ChatRequestBody = await req.json();
  const chat_session_id = chatRequestBody.chat_session_id;
 
  const ikpClient = new InkeepAI({
    apiKey: process.env.INKEEP_API_KEY,
  });
 
  let response;
 
  if (!chat_session_id) {
    const createRes = await ikpClient.chatSession.create({
      integrationId: inkeepIntegrationId,
      chatSession: {
        messages: chatRequestBody.messages,
      },
      stream: true,
    });
 
    response = createRes.rawResponse;
  } else {
    const continueRes = await ikpClient.chatSession.continue(chat_session_id, {
      integrationId: inkeepIntegrationId,
      message: chatRequestBody.messages[chatRequestBody.messages.length - 1],
      stream: true,
    });
 
    response = continueRes.rawResponse;
  }
 
  // used to pass custom metadata to the client
  const data = new StreamData();
 
  if (!response?.body) {
    throw new Error('Response body is null');
  }
 
  const stream = InkeepStream(response, {
    onRecordsCited: async (records_cited: RecordsCited$.Inbound) => {
      // append the citations to the message annotations
      data.appendMessageAnnotation({
        records_cited,
      });
    },
    onFinal: async (complete: string, metadata?: InkeepOnFinalMetadata) => {
      // return the chat_session_id to the client
      if (metadata) {
        data.append({ onFinalMetadata: metadata });
      }
      data.close();
    },
  });
 
  return new StreamingTextResponse(stream, {}, data);
}

This example uses the StreamData and the callback methods of InkeepStream to attach metadata to the response.

Client

From useChat, this is available as:

app/chat/page.tsx
import { InkeepOnFinalMetadata } from 'ai/streams';
import type { RecordsCited$ } from '@inkeep/ai-api/models/components';
 
// ... your chat component
 
const { messages, data } = useChat();
 
/* ==For chat_session_id== */
// get the onFinalMetadata item from the global chat data
const onFinalMetadataItem = data?.find(
  item =>
    typeof item === 'object' && item !== null && 'onFinalMetadata' in item,
) as { onFinalMetadata: InkeepOnFinalMetadata } | undefined;
 
// get the chat_session_id from the onFinalMetadata item
const chatSessionId = onFinalMetadataItem?.onFinalMetadata?.chat_session_id;
 
/* For messages[n].annotations, available independently for each message */
 
const recordsCitedAnnotation =
  messages &&
  messages.length > 0 &&
  (messages[0].annotations?.find(
    item =>
      typeof item === 'object' && item !== null && 'records_cited' in item,
  ) as { records_cited: RecordsCited$.Inbound } | undefined);
 
// get the citations from the records_cited annotation
const citations = recordsCitedAnnotation?.records_cited?.citations;

© 2023 Vercel Inc.