Skip to content

fix(api): two LLM endpoints spent money with no rate limit - #139

Merged
catomean merged 1 commit into
mainfrom
fix/rate-limit-public-llm
Aug 28, 2026
Merged

fix(api): two LLM endpoints spent money with no rate limit#139
catomean merged 1 commit into
mainfrom
fix/rate-limit-public-llm

Conversation

@catomean

Copy link
Copy Markdown
Collaborator

The bug

Two routes called an LLM provider with no rate limit at all:

  • app/api/demo/chat — public, unauthenticated, reachable in production
  • app/api/custom-bots/[id]/chat — anonymous callers may chat with public bots, and the call is billed to the bot owner's API key

Nothing failed, because nothing was checking.

Why they drifted

Each route hand-rolled the same block — read the IP, call checkRateLimit with inline magic numbers, return a 429 written slightly differently each time. Seven copies, three different 429 shapes, and one hand-rolled reimplementation of getClientIp. Two routes simply never got their copy.

The fix

  • RATE_LIMITS — one SSOT for every bucket's budget. A route names a bucket; it does not invent a number.
  • enforceRateLimit(request, bucket, scope?) — returns a ready-to-return 429, or null.
  • Both gaps closed. The custom-bot limit is scoped per bot id, so one hot bot cannot drain another's budget.
  • The seven existing call sites migrated onto the helper.

Never-twice

tests/__tests__/api/rate-limit-coverage.test.ts reads every route file and fails if one imports the LLM client without enforcing a limit, or calls checkRateLimit directly with its own numbers.

Mutation-tested — removing the limit from demo/chat makes it fail and name the file:

✕ every LLM-calling route enforces a rate limit
  - Array []
  + Array [ "demo/chat/route.ts" ]

Verify

format:check, lint, typecheck, 240 tests, build — all green locally.

Not in scope

app/api/consultations rate limits on a global key ('CONSULTATION_FORM') rather than per IP, so one spammer exhausts the budget for everyone. A test depends on its current shape; flagged separately.

🤖 Generated with Claude Code

https://claude.ai/code/session_01HVwg8DKHQktxJuHeLM3xpG

app/api/demo/chat is public and unauthenticated, and
app/api/custom-bots/[id]/chat lets anonymous callers chat with public
bots billed to the bot OWNER's API key. Neither had a limit. Nothing
failed, because nothing was checking.

Both are closed. The custom-bot limit is scoped per bot id so one hot
bot cannot drain another's budget.

The reason they drifted is that each route hand-rolled the same block:
read the IP, call checkRateLimit with inline magic numbers, return a
429 written slightly differently each time. So the budgets now live in
one place (RATE_LIMITS) behind one helper (enforceRateLimit), and the
seven existing copies were migrated onto it -- removing three different
429 response shapes and a hand-rolled copy of getClientIp.

Never-twice: tests/__tests__/api/rate-limit-coverage.test.ts reads every
route file and fails if one imports the LLM client without enforcing a
limit, or calls checkRateLimit directly with its own numbers. Verified
by mutation -- removing a limit makes it fail and name the file.

verify: format, lint, typecheck, 240 tests, build -- all green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HVwg8DKHQktxJuHeLM3xpG
@catomean
catomean merged commit 08e2a3a into main Aug 28, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant