AdvancedRate Limiting

Rate Limiting

Rate limiting helps you protect your APIs from abuse. It involves setting a maximum threshold on the number of requests a client can make within a specified timeframe. This simple technique acts as a gatekeeper, preventing excessive usage that can degrade service performance and incur unnecessary costs.

Rate Limiting with Vercel KV and Upstash Ratelimit

In this example, you will protect an API endpoint using Vercel KV and Upstash Ratelimit.

app/api/generate/route.ts
import kv from '@vercel/kv';
import { openai } from '@ai-sdk/openai';
import { OpenAIStream, StreamingTextResponse } from 'ai';
import { Ratelimit } from '@upstash/ratelimit';
import { NextRequest } from 'next/server';
// Create Rate limit
const ratelimit = new Ratelimit({
redis: kv,
limiter: Ratelimit.fixedWindow(5, '30s'),
});
export async function POST(req: NextRequest) {
// call ratelimit with request ip
const ip = req.ip ?? 'ip';
const { success, remaining } = await ratelimit.limit(ip);
// block the request if unsuccessfull
if (!success) {
return new Response('Ratelimited!', { status: 429 });
}
const { messages } = await req.json();
const result = await streamText({
model: openai('gpt-3.5-turbo'),
messages,
});
return new StreamingTextResponse(result.toAIStream());
}

Simplfy API Protection

With Vercel KV and Upstash Ratelimit, it is possible to protect your APIs from such attacks with ease. To learn more about how Ratelimit works and how it can be configured to your needs, see Ratelimit Documentation.