LangSmith Observability

LangSmith is a platform for building production-grade LLM applications. It allows you to closely monitor and evaluate your application, so you can ship quickly and with confidence.

Use of LangChain's open-source frameworks is not necessary, and LangSmith integrates with the AI SDK via the AISDKExporter OpenTelemetry trace exporter.

A version of this guide is also available in the LangSmith documentation.

Setup

Install an AI SDK model provider and the LangSmith client SDK. The code snippets below will use the AI SDK's OpenAI provider, but you can use any other supported provider as well.

pnpm
npm
yarn
pnpm add @ai-sdk/openai langsmith

The AISDKExporter class is only available in langsmith SDK version >=0.2.1.

Next, set required environment variables.

export LANGCHAIN_TRACING_V2=true
export LANGCHAIN_API_KEY=<your-api-key>
export OPENAI_API_KEY=<your-openai-api-key> # The examples use OpenAI (replace with your selected provider)

You can also see this guide for other ways to configure LangSmith, or the section below for how to pass in a custom LangSmith client instance.

Trace Logging

Next.js

First, create an instrumentation.js file in your project root.

import { registerOTel } from '@vercel/otel';
import { AISDKExporter } from 'langsmith/vercel';
export function register() {
registerOTel({
serviceName: 'langsmith-vercel-ai-sdk-example',
traceExporter: new AISDKExporter(),
});
}

You can learn more how to setup OpenTelemetry instrumentation within your Next.js app here.

Afterwards, add the experimental_telemetry argument to your AI SDK calls that you want to trace. For convenience, we've included the AISDKExporter.getSettings() method which appends additional metadata for LangSmith.

import { AISDKExporter } from 'langsmith/vercel';
import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';
await streamText({
model: openai('gpt-4o-mini'),
prompt: 'Write a vegetarian lasagna recipe for 4 people.',
experimental_telemetry: AISDKExporter.getSettings(),
});

You should see a trace in your LangSmith dashboard like this one.

You can also trace tool calls:

import { AISDKExporter } from 'langsmith/vercel';
import { generateText, tool } from 'ai';
import { openai } from '@ai-sdk/openai';
import { z } from 'zod';
await generateText({
model: openai('gpt-4o-mini'),
messages: [
{
role: 'user',
content: 'What are my orders and where are they? My user ID is 123',
},
],
tools: {
listOrders: tool({
description: 'list all orders',
parameters: z.object({ userId: z.string() }),
execute: async ({ userId }) =>
`User ${userId} has the following orders: 1`,
}),
viewTrackingInformation: tool({
description: 'view tracking information for a specific order',
parameters: z.object({ orderId: z.string() }),
execute: async ({ orderId }) =>
`Here is the tracking information for ${orderId}`,
}),
},
experimental_telemetry: AISDKExporter.getSettings(),
maxSteps: 10,
});

Which results in a trace like this one.

Node.js

Add the AISDKExporter to the trace exporter to your OpenTelemetry setup.

import { AISDKExporter } from 'langsmith/vercel';
import { NodeSDK } from '@opentelemetry/sdk-node';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';
const sdk = new NodeSDK({
traceExporter: new AISDKExporter(),
instrumentations: [getNodeAutoInstrumentations()],
});
sdk.start();

Afterwards, add the experimental_telemetry argument to your AI SDK calls that you want to trace.

Do not forget to call await sdk.shutdown() before your application shuts down in order to flush any remaining traces to LangSmith.

import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';
import { AISDKExporter } from 'langsmith/vercel';
const result = await generateText({
model: openai('gpt-4o-mini'),
prompt: 'Write a vegetarian lasagna recipe for 4 people.',
experimental_telemetry: AISDKExporter.getSettings(),
});
await sdk.shutdown();

You should see a trace in your LangSmith dashboard like this one.

Sentry

If you're using Sentry, you can attach the LangSmith trace exporter to Sentry's default OpenTelemetry instrumentation as follows:

import * as Sentry from '@sentry/node';
import { BatchSpanProcessor } from '@opentelemetry/sdk-trace-base';
import { AISDKExporter } from 'langsmith/vercel';
const client = Sentry.init({
dsn: '[Sentry DSN]',
tracesSampleRate: 1.0,
});
client?.traceProvider?.addSpanProcessor(
new BatchSpanProcessor(new AISDKExporter()),
);

Configuration

Customize run name

You can customize the run name by passing the runName argument to the AISDKExporter.getSettings() method.

import { AISDKExporter } from 'langsmith/vercel';
import { openai } from '@ai-sdk/openai';
import { generateText } from 'ai';
await generateText({
model: openai('gpt-4o-mini'),
prompt: 'Write a vegetarian lasagna recipe for 4 people.',
experimental_telemetry: AISDKExporter.getSettings({
runName: 'my-custom-run-name',
}),
});

Customize run ID

You can customize the run ID by passing the runId argument to the AISDKExporter.getSettings() method. This is especially useful if you want to know the run ID before the run has been completed.

import { AISDKExporter } from 'langsmith/vercel';
import { openai } from '@ai-sdk/openai';
import { generateText } from 'ai';
await generateText({
model: openai('gpt-4o-mini'),
prompt: 'Write a vegetarian lasagna recipe for 4 people.',
experimental_telemetry: AISDKExporter.getSettings({
runId: 'my-custom-run-id',
}),
});

Nesting runs

You can also nest runs within other traced functions to create a hierarchy of associated runs. Here's an example using the traceable method:

import { AISDKExporter } from 'langsmith/vercel';
import { openai } from '@ai-sdk/openai';
import { generateText } from 'ai';
import { traceable } from 'langsmith/traceable';
const wrappedGenerateText = traceable(
async (content: string) => {
const { text } = await generateText({
model: openai('gpt-4o-mini'),
messages: [{ role: 'user', content }],
experimental_telemetry: AISDKExporter.getSettings(),
});
const reverseText = traceable(
async (text: string) => {
return text.split('').reverse().join('');
},
{
name: 'reverseText',
},
);
const reversedText = await reverseText(text);
return { text, reversedText };
},
{ name: 'parentTraceable' },
);
const result = await wrappedGenerateText(
'What color is the sky? Respond with one word.',
);

The resulting trace will look like this one.

Custom LangSmith client

You can also pass a LangSmith client instance into the AISDKExporter constructor:

import { AISDKExporter } from 'langsmith/vercel';
import { Client } from 'langsmith';
import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';
import { NodeSDK } from '@opentelemetry/sdk-node';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';
const langsmithClient = new Client({});
const sdk = new NodeSDK({
traceExporter: new AISDKExporter({ client: langsmithClient }),
instrumentations: [getNodeAutoInstrumentations()],
});
sdk.start();
await generateText({
model: openai('gpt-4o-mini'),
prompt: 'Write a vegetarian lasagna recipe for 4 people.',
experimental_telemetry: AISDKExporter.getSettings(),
});

Debugging Exporter

You can enable debug logs for the AISDKExporter by passing the debug argument to the constructor.

const traceExporter = new AISDKExporter({ debug: true });

Alternatively, you can set the OTEL_LOG_LEVEL=DEBUG environment variable to enable debug logs for the exporter as well as the rest of the OpenTelemetry stack.

Adding metadata

You can add metadata to your traces to help organize and filter them in the LangSmith UI:

import { AISDKExporter } from 'langsmith/vercel';
import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';
await generateText({
model: openai('gpt-4o-mini'),
prompt: 'Write a vegetarian lasagna recipe for 4 people.',
experimental_telemetry: AISDKExporter.getSettings({
metadata: { userId: '123', language: 'english' },
}),
});

Metadata will be visible in your LangSmith dashboard and can be used to filter and search for specific traces.

Further reading

Once you've set up LangSmith tracing for your project, try gathering a dataset and evaluating it: