Image Generation
The AI SDK provides the generateImage
function to generate images based on a given prompt using an image model.
import { experimental_generateImage as generateImage } from 'ai';import { openai } from '@ai-sdk/openai';
const { image } = await generateImage({ model: openai.image('dall-e-3'), prompt: 'Santa Claus driving a Cadillac',});
You can access the image data using the base64
or uint8Array
properties:
const base64 = image.base64; // base64 image dataconst uint8Array = image.uint8Array; // Uint8Array image data
Size and Aspect Ratio
Depending on the model, you can either specify the size or the aspect ratio.
Size
The size is specified as a string in the format {width}x{height}
.
Models only support a few sizes, and the supported sizes are different for each model and provider.
import { experimental_generateImage as generateImage } from 'ai';import { openai } from '@ai-sdk/openai';
const { image } = await generateImage({ model: openai.image('dall-e-3'), prompt: 'Santa Claus driving a Cadillac', size: '1024x1024',});
Aspect Ratio
The aspect ratio is specified as a string in the format {width}:{height}
.
Models only support a few aspect ratios, and the supported aspect ratios are different for each model and provider.
import { experimental_generateImage as generateImage } from 'ai';import { vertex } from '@ai-sdk/google-vertex';
const { image } = await generateImage({ model: vertex.image('imagen-3.0-generate-001'), prompt: 'Santa Claus driving a Cadillac', aspectRatio: '16:9',});
Generating Multiple Images
generateImage
also supports generating multiple images at once:
import { experimental_generateImage as generateImage } from 'ai';import { openai } from '@ai-sdk/openai';
const { images } = await generateImage({ model: openai.image('dall-e-2'), prompt: 'Santa Claus driving a Cadillac', n: 4, // number of images to generate});
generateImage
will automatically call the model as often as needed (in
parallel) to generate the requested number of images.
Each image model has an internal limit on how many images it can generate in a single API call. The AI SDK manages this automatically by batching requests appropriately when you request multiple images using the n
parameter. By default, the SDK uses provider-documented limits (for example, DALL-E 3 can only generate 1 image per call, while DALL-E 2 supports up to 10).
If needed, you can override this behavior using the maxImagesPerCall
setting when configuring your model. This is particularly useful when working with new or custom models where the default batch size might not be optimal:
const model = openai.image('dall-e-2', { maxImagesPerCall: 5, // Override the default batch size});
const { images } = await generateImage({ model, prompt: 'Santa Claus driving a Cadillac', n: 10, // Will make 2 calls of 5 images each});
Providing a Seed
You can provide a seed to the generateImage
function to control the output of the image generation process.
If supported by the model, the same seed will always produce the same image.
import { experimental_generateImage as generateImage } from 'ai';import { openai } from '@ai-sdk/openai';
const { image } = await generateImage({ model: openai.image('dall-e-3'), prompt: 'Santa Claus driving a Cadillac', seed: 1234567890,});
Provider-specific Settings
Image models often have provider- or even model-specific settings.
You can pass such settings to the generateImage
function
using the providerOptions
parameter. The options for the provider
(openai
in the example below) become request body properties.
import { experimental_generateImage as generateImage } from 'ai';import { openai } from '@ai-sdk/openai';
const { image } = await generateImage({ model: openai.image('dall-e-3'), prompt: 'Santa Claus driving a Cadillac', size: '1024x1024', providerOptions: { openai: { style: 'vivid', quality: 'hd' }, },});
Abort Signals and Timeouts
generateImage
accepts an optional abortSignal
parameter of
type AbortSignal
that you can use to abort the image generation process or set a timeout.
import { openai } from '@ai-sdk/openai';import { experimental_generateImage as generateImage } from 'ai';
const { image } = await generateImage({ model: openai.image('dall-e-3'), prompt: 'Santa Claus driving a Cadillac', abortSignal: AbortSignal.timeout(1000), // Abort after 1 second});
Custom Headers
generateImage
accepts an optional headers
parameter of type Record<string, string>
that you can use to add custom headers to the image generation request.
import { openai } from '@ai-sdk/openai';import { experimental_generateImage as generateImage } from 'ai';
const { image } = await generateImage({ model: openai.image('dall-e-3'), value: 'sunny day at the beach', headers: { 'X-Custom-Header': 'custom-value' },});
Warnings
If the model returns warnings, e.g. for unsupported parameters, they will be available in the warnings
property of the response.
const { image, warnings } = await generateImage({ model: openai.image('dall-e-3'), prompt: 'Santa Claus driving a Cadillac',});
Error Handling
When generateImage
cannot generate a valid image, it throws a AI_NoImageGeneratedError
.
This error occurs when the AI provider fails to generate an image. It can arise due to the following reasons:
- The model failed to generate a response
- The model generated a response that could not be parsed
The error preserves the following information to help you log the issue:
responses
: Metadata about the image model responses, including timestamp, model, and headers.cause
: The cause of the error. You can use this for more detailed error handling
import { generateImage, NoImageGeneratedError } from 'ai';
try { await generateImage({ model, prompt });} catch (error) { if (NoImageGeneratedError.isInstance(error)) { console.log('NoImageGeneratedError'); console.log('Cause:', error.cause); console.log('Responses:', error.responses); }}
Image Models
Provider | Model | Support sizes (width x height ) or aspect ratios (width : height ) |
---|---|---|
Replicate | black-forest-labs/flux-schnell | 1:1, 2:3, 3:2, 4:5, 5:4, 16:9, 9:16, 9:21, 21:9 |
Replicate | recraft-ai/recraft-v3 | 1024x1024, 1365x1024, 1024x1365, 1536x1024, 1024x1536, 1820x1024, 1024x1820, 1024x2048, 2048x1024, 1434x1024, 1024x1434, 1024x1280, 1280x1024, 1024x1707, 1707x1024 |
Google Vertex | imagen-3.0-generate-001 | 1:1, 3:4, 4:3, 9:16, 16:9 |
Google Vertex | imagen-3.0-fast-generate-001 | 1:1, 3:4, 4:3, 9:16, 16:9 |
OpenAI | dall-e-3 | 1024x1024, 1792x1024, 1024x1792 |
OpenAI | dall-e-2 | 256x256, 512x512, 1024x1024 |
Fireworks | accounts/fireworks/models/flux-1-dev-fp8 | 1:1, 2:3, 3:2, 4:5, 5:4, 16:9, 9:16, 9:21, 21:9 |
Fireworks | accounts/fireworks/models/flux-1-schnell-fp8 | 1:1, 2:3, 3:2, 4:5, 5:4, 16:9, 9:16, 9:21, 21:9 |
Fireworks | accounts/fireworks/models/playground-v2-5-1024px-aesthetic | 640x1536, 768x1344, 832x1216, 896x1152, 1024x1024, 1152x896, 1216x832, 1344x768, 1536x640 |
Fireworks | accounts/fireworks/models/japanese-stable-diffusion-xl | 640x1536, 768x1344, 832x1216, 896x1152, 1024x1024, 1152x896, 1216x832, 1344x768, 1536x640 |
Fireworks | accounts/fireworks/models/playground-v2-1024px-aesthetic | 640x1536, 768x1344, 832x1216, 896x1152, 1024x1024, 1152x896, 1216x832, 1344x768, 1536x640 |
Fireworks | accounts/fireworks/models/SSD-1B | 640x1536, 768x1344, 832x1216, 896x1152, 1024x1024, 1152x896, 1216x832, 1344x768, 1536x640 |
Fireworks | accounts/fireworks/models/stable-diffusion-xl-1024-v1-0 | 640x1536, 768x1344, 832x1216, 896x1152, 1024x1024, 1152x896, 1216x832, 1344x768, 1536x640 |