Community ProvidersCloudflare Workers AI

Cloudflare Workers AI

threepointone/workers-ai-provider is a community provider that allows you to use Cloudflare's Workers AI models with the AI SDK.

Setup

The Cloudflare Workers AI provider is available in the workers-ai-provider module. You can install it with:

pnpm
npm
yarn
pnpm add workers-ai-provider

Then, setup an AI binding in your Cloudflare Workers project wrangler.toml file:

wrangler.toml
# ...
[ai]
binding = "AI"
# ...

Provider Instance

To create a workersai provider instance, use the createWorkersAI function, passing in the AI binding as an option:

import { createWorkersAI } from 'workers-ai-provider';
const workersai = createWorkersAI({ binding: env.AI });

Language Models

To create a model instance, call the provider instance and specify the model you would like to use as the first argument. You can also pass additional settings in the second argument:

import { createWorkersAI } from 'workers-ai-provider';
const workersai = createWorkersAI({ binding: env.AI });
const model = workersai('@cf/meta/llama-3.1-8b-instruct', {
// additional settings
safePrompt: true,
});

You can use the following optional settings to customize:

  • safePrompt boolean

    Whether to inject a safety prompt before all conversations. Defaults to false

Examples

You can use Cloudflare Workers AI language models to generate text with the generateText or streamText function:

generateText

import { createWorkersAI } from 'workers-ai-provider';
import { generateText } from 'ai';
type Env = {
AI: Ai;
};
export default {
async fetch(_: Request, env: Env) {
const workersai = createWorkersAI({ binding: env.AI });
const result = await generateText({
model: workersai('@cf/meta/llama-2-7b-chat-int8'),
prompt: 'Write a 50-word essay about hello world.',
});
return new Response(result.text);
},
};

streamText

import { createWorkersAI } from 'workers-ai-provider';
import { streamText } from 'ai';
type Env = {
AI: Ai;
};
export default {
async fetch(_: Request, env: Env) {
const workersai = createWorkersAI({ binding: env.AI });
const result = await streamText({
model: workersai('@cf/meta/llama-2-7b-chat-int8'),
prompt: 'Write a 50-word essay about hello world.',
});
return result.toTextStreamResponse({
headers: {
// add these headers to ensure that the
// response is chunked and streamed
'Content-Type': 'text/x-unknown',
'content-encoding': 'identity',
'transfer-encoding': 'chunked',
},
});
},
};

generateObject

Some Cloudflare Workers AI language models can also be used with the generateObject function:

import { createWorkersAI } from 'workers-ai-provider';
import { generateObject } from 'ai';
import { z } from 'zod';
type Env = {
AI: Ai;
};
export default {
async fetch(_: Request, env: Env) {
const workersai = createWorkersAI({ binding: env.AI });
const result = await generateObject({
model: workersai('@cf/meta/llama-3.1-8b-instruct'),
prompt: 'Generate a Lasagna recipe',
schema: z.object({
recipe: z.object({
ingredients: z.array(z.string()),
description: z.string(),
}),
}),
});
return Response.json(result.object);
},
};
tools and streamObject are currently not supported.