fix(api): two LLM endpoints spent money with no rate limit - #139
Merged
Conversation
app/api/demo/chat is public and unauthenticated, and app/api/custom-bots/[id]/chat lets anonymous callers chat with public bots billed to the bot OWNER's API key. Neither had a limit. Nothing failed, because nothing was checking. Both are closed. The custom-bot limit is scoped per bot id so one hot bot cannot drain another's budget. The reason they drifted is that each route hand-rolled the same block: read the IP, call checkRateLimit with inline magic numbers, return a 429 written slightly differently each time. So the budgets now live in one place (RATE_LIMITS) behind one helper (enforceRateLimit), and the seven existing copies were migrated onto it -- removing three different 429 response shapes and a hand-rolled copy of getClientIp. Never-twice: tests/__tests__/api/rate-limit-coverage.test.ts reads every route file and fails if one imports the LLM client without enforcing a limit, or calls checkRateLimit directly with its own numbers. Verified by mutation -- removing a limit makes it fail and name the file. verify: format, lint, typecheck, 240 tests, build -- all green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HVwg8DKHQktxJuHeLM3xpG
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
Two routes called an LLM provider with no rate limit at all:
app/api/demo/chat— public, unauthenticated, reachable in productionapp/api/custom-bots/[id]/chat— anonymous callers may chat with public bots, and the call is billed to the bot owner's API keyNothing failed, because nothing was checking.
Why they drifted
Each route hand-rolled the same block — read the IP, call
checkRateLimitwith inline magic numbers, return a 429 written slightly differently each time. Seven copies, three different 429 shapes, and one hand-rolled reimplementation ofgetClientIp. Two routes simply never got their copy.The fix
RATE_LIMITS— one SSOT for every bucket's budget. A route names a bucket; it does not invent a number.enforceRateLimit(request, bucket, scope?)— returns a ready-to-return 429, ornull.Never-twice
tests/__tests__/api/rate-limit-coverage.test.tsreads every route file and fails if one imports the LLM client without enforcing a limit, or callscheckRateLimitdirectly with its own numbers.Mutation-tested — removing the limit from
demo/chatmakes it fail and name the file:Verify
format:check,lint,typecheck, 240 tests,build— all green locally.Not in scope
app/api/consultationsrate limits on a global key ('CONSULTATION_FORM') rather than per IP, so one spammer exhausts the budget for everyone. A test depends on its current shape; flagged separately.🤖 Generated with Claude Code
https://claude.ai/code/session_01HVwg8DKHQktxJuHeLM3xpG