LangSmith Observability
LangSmith is a platform for building production-grade LLM applications. It allows you to closely monitor and evaluate your application, so you can ship quickly and with confidence.
Use of LangChain's open-source frameworks is not necessary, and LangSmith integrates with the AI SDK via the AISDKExporter
OpenTelemetry trace exporter.
A version of this guide is also available in the LangSmith documentation.
Setup
Install an AI SDK model provider and the LangSmith client SDK. The code snippets below will use the AI SDK's OpenAI provider, but you can use any other supported provider as well.
pnpm add @ai-sdk/openai langsmith
The AISDKExporter
class is only available in langsmith
SDK version
>=0.2.1
.
Next, set required environment variables.
export LANGCHAIN_TRACING_V2=trueexport LANGCHAIN_API_KEY=<your-api-key>
export OPENAI_API_KEY=<your-openai-api-key> # The examples use OpenAI (replace with your selected provider)
You can also see this guide for other ways to configure LangSmith, or the section below for how to pass in a custom LangSmith client instance.
Trace Logging
Next.js
First, create an instrumentation.js
file in your project root.
import { registerOTel } from '@vercel/otel';import { AISDKExporter } from 'langsmith/vercel';
export function register() { registerOTel({ serviceName: 'langsmith-vercel-ai-sdk-example', traceExporter: new AISDKExporter(), });}
You can learn more how to setup OpenTelemetry instrumentation within your Next.js app here.
Afterwards, add the experimental_telemetry
argument to your AI SDK calls that you want to trace. For convenience, we've included the AISDKExporter.getSettings()
method which appends additional metadata for LangSmith.
import { AISDKExporter } from 'langsmith/vercel';import { streamText } from 'ai';import { openai } from '@ai-sdk/openai';
await streamText({ model: openai('gpt-4o-mini'), prompt: 'Write a vegetarian lasagna recipe for 4 people.', experimental_telemetry: AISDKExporter.getSettings(),});
You should see a trace in your LangSmith dashboard like this one.
You can also trace tool calls:
import { AISDKExporter } from 'langsmith/vercel';import { generateText, tool } from 'ai';import { openai } from '@ai-sdk/openai';import { z } from 'zod';
await generateText({ model: openai('gpt-4o-mini'), messages: [ { role: 'user', content: 'What are my orders and where are they? My user ID is 123', }, ], tools: { listOrders: tool({ description: 'list all orders', parameters: z.object({ userId: z.string() }), execute: async ({ userId }) => `User ${userId} has the following orders: 1`, }), viewTrackingInformation: tool({ description: 'view tracking information for a specific order', parameters: z.object({ orderId: z.string() }), execute: async ({ orderId }) => `Here is the tracking information for ${orderId}`, }), }, experimental_telemetry: AISDKExporter.getSettings(), maxSteps: 10,});
Which results in a trace like this one.
Node.js
Add the AISDKExporter
to the trace exporter to your OpenTelemetry setup.
import { AISDKExporter } from 'langsmith/vercel';
import { NodeSDK } from '@opentelemetry/sdk-node';import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';
const sdk = new NodeSDK({ traceExporter: new AISDKExporter(), instrumentations: [getNodeAutoInstrumentations()],});
sdk.start();
Afterwards, add the experimental_telemetry
argument to your AI SDK calls that you want to trace.
Do not forget to call await sdk.shutdown()
before your application shuts
down in order to flush any remaining traces to LangSmith.
import { generateText } from 'ai';import { openai } from '@ai-sdk/openai';import { AISDKExporter } from 'langsmith/vercel';
const result = await generateText({ model: openai('gpt-4o-mini'), prompt: 'Write a vegetarian lasagna recipe for 4 people.', experimental_telemetry: AISDKExporter.getSettings(),});
await sdk.shutdown();
You should see a trace in your LangSmith dashboard like this one.
Sentry
If you're using Sentry, you can attach the LangSmith trace exporter to Sentry's default OpenTelemetry instrumentation as follows:
import * as Sentry from '@sentry/node';import { BatchSpanProcessor } from '@opentelemetry/sdk-trace-base';import { AISDKExporter } from 'langsmith/vercel';
const client = Sentry.init({ dsn: '[Sentry DSN]', tracesSampleRate: 1.0,});
client?.traceProvider?.addSpanProcessor( new BatchSpanProcessor(new AISDKExporter()),);
Configuration
Customize run name
You can customize the run name by passing the runName
argument to the AISDKExporter.getSettings()
method.
import { AISDKExporter } from 'langsmith/vercel';import { openai } from '@ai-sdk/openai';import { generateText } from 'ai';
await generateText({ model: openai('gpt-4o-mini'), prompt: 'Write a vegetarian lasagna recipe for 4 people.', experimental_telemetry: AISDKExporter.getSettings({ runName: 'my-custom-run-name', }),});
Customize run ID
You can customize the run ID by passing the runId
argument to the AISDKExporter.getSettings()
method. This is especially useful if you want to know the run ID before the run has been completed.
import { AISDKExporter } from 'langsmith/vercel';import { openai } from '@ai-sdk/openai';import { generateText } from 'ai';
await generateText({ model: openai('gpt-4o-mini'), prompt: 'Write a vegetarian lasagna recipe for 4 people.', experimental_telemetry: AISDKExporter.getSettings({ runId: 'my-custom-run-id', }),});
Nesting runs
You can also nest runs within other traced functions to create a hierarchy of associated runs.
Here's an example using the traceable
method:
import { AISDKExporter } from 'langsmith/vercel';import { openai } from '@ai-sdk/openai';import { generateText } from 'ai';
import { traceable } from 'langsmith/traceable';
const wrappedGenerateText = traceable( async (content: string) => { const { text } = await generateText({ model: openai('gpt-4o-mini'), messages: [{ role: 'user', content }], experimental_telemetry: AISDKExporter.getSettings(), });
const reverseText = traceable( async (text: string) => { return text.split('').reverse().join(''); }, { name: 'reverseText', }, );
const reversedText = await reverseText(text); return { text, reversedText }; }, { name: 'parentTraceable' },);
const result = await wrappedGenerateText( 'What color is the sky? Respond with one word.',);
The resulting trace will look like this one.
Custom LangSmith client
You can also pass a LangSmith client instance into the AISDKExporter
constructor:
import { AISDKExporter } from 'langsmith/vercel';import { Client } from 'langsmith';
import { generateText } from 'ai';import { openai } from '@ai-sdk/openai';
import { NodeSDK } from '@opentelemetry/sdk-node';import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';
const langsmithClient = new Client({});
const sdk = new NodeSDK({ traceExporter: new AISDKExporter({ client: langsmithClient }), instrumentations: [getNodeAutoInstrumentations()],});
sdk.start();
await generateText({ model: openai('gpt-4o-mini'), prompt: 'Write a vegetarian lasagna recipe for 4 people.', experimental_telemetry: AISDKExporter.getSettings(),});
Debugging Exporter
You can enable debug logs for the AISDKExporter
by passing the debug
argument to the constructor.
const traceExporter = new AISDKExporter({ debug: true });
Alternatively, you can set the OTEL_LOG_LEVEL=DEBUG
environment variable to enable debug logs for the exporter as well as the rest of the OpenTelemetry stack.
Adding metadata
You can add metadata to your traces to help organize and filter them in the LangSmith UI:
import { AISDKExporter } from 'langsmith/vercel';import { generateText } from 'ai';import { openai } from '@ai-sdk/openai';
await generateText({ model: openai('gpt-4o-mini'), prompt: 'Write a vegetarian lasagna recipe for 4 people.', experimental_telemetry: AISDKExporter.getSettings({ metadata: { userId: '123', language: 'english' }, }),});
Metadata will be visible in your LangSmith dashboard and can be used to filter and search for specific traces.
Further reading
- LangSmith docs
- LangSmith guide on tracing with the AI SDK
- LangSmith guide on tracing without environment variables
Once you've set up LangSmith tracing for your project, try gathering a dataset and evaluating it: