Repository navigation
Conversation
…h finds nobody When /search finds no stored character for a full "First Surname", ask the Blizzard Profile API for that slug, US first, then EU (1 to 4 calls), with the kb armory client and parser, and store a hit through armory_upsert_characters. Behind EVERLOOT_ARMORY_LOOKUP=1, off by default. Limits: only names that pass the armory form's rules, only the /search page's own browser fetch (never the header type-ahead, crawlers or scripts), only regions where the slug is neither hidden nor stored, a six-hour miss cache, a 'lookup' quota of 20 an hour per address and 300 site-wide, and a 1.8 s budget. Part of #1270. Signed-off-by: CDVolvik <cjimenez@r21digital.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #1270 (proposal item 2, the on-demand lookup).
Stacked on #1335 (base
srdev/search-20261116). After #1335 merges, this PR retargets tomain.What changed
When /search finds no stored character for a full "First Surname", the site now asks the Blizzard Profile API for that slug, US first, then EU, stores a hit and lists it.
Lookup (
web/src/lib/armory-lookup.ts):kb/src/armory:ForeverApifor OAuth and the requests, andparseCharacterfor the row. Nothing is duplicated.armory_upsert_characters, the crawler's write, with its hide guard. That same function already queues the character's guild for the crawler (discovered_guilds, sourcecharacter). So item 4 of the brief needed no new queue call; a test checks the guild is queued.Abuse controls (the lookup spends a paid third-party quota from a public endpoint):
EVERLOOT_ARMORY_LOOKUP=1, which also needsEVERLOOT_ARMORY=1. It is off by default and not set inwrangler.jsonc.characterSlugOf: 2–12 and 2–24 letters). A single word, three words or a one-letter surname never triggers a lookup.lookup=1, and only when the request is the browser's own fetch:Sec-Fetch-Site: same-origin,Sec-Fetch-Dest: empty, aMozilla/agent, and no bot, crawler, headless or script marker in the agent. The header type-ahead never sendslookup=1, because it would ask on every key. The repository had no user-agent bot handling to reuse. The route was alreadynoindexandno-store, and that is unchanged.armory_lookup_regions(slug)returns only regions where the slug is neither hidden nor stored. A hidden slug is never fetched in its region. A hide that lands mid-lookup is refused by the upsert's guard.armory_lookup_miss(slug)keeps a name that both regions answered 404 for, for six hours. While it is kept, the name is not asked again and costs no quota. Errors and timeouts are not cached as misses.lookupscope on the existingconsume_intake_quotalimiter (the one the forms, MCP and market use): 20 lookups an hour per address and 300 an hour site-wide. That is at most 1,200 Blizzard calls an hour, against the client's documented 36,000. The existingPEOPLE_SEARCH_LIMIT(60 a minute) still applies first. The quota is consumed only after the hidden and miss checks, just before the first Blizzard call. If the quota cannot be read, the lookup is refused. This is the opposite of the per-minute limiter, which lets requests through when it cannot be read.Migration
supabase/migrations/20261117150000_armory_lookup.sql(not applied anywhere):armory_lookup_misses. RLS is on and no role has grants on it.armory_lookup_regionsandarmory_lookup_miss. Both are security definer, granted to service_role only.consume_intake_quotawith thelookupscope added. The other scopes are unchanged.Also changed:
docs/ENVIRONMENT.mdandweb/.env.examplelist the new switch, and that the BNET client is now also used by the lookup.For the maintainer
20261117150000_armory_lookup.sql(after fix(web): character search finds Forever characters by surname or full name #1335's migration).everloot-web):BNET_CLIENT_ID(secret): the Battle.net client id. It is the same name the crawler (.github/workflows/armory.yml) and the site's Battle.net sign-in already read. If the Worker already has it for sign-in, nothing new is needed.BNET_CLIENT_SECRET(secret): the same.EVERLOOT_ARMORY_LOOKUP=1(var): last, after the migration.Not done / follow-ups
checked_atis now, so the crawler won't read it again for 20 days unless someone presses refresh, which is allowed after an hour. Queueing these for an early details read belongs to the crawler lane (srdev/crawler-20261116).cf.botManagement) is not used, because the code doesn't read it today.supabase/tests/armory.sqlpins 22armory_*functions for the migrations the existing PGlite tests apply. It is not run against this migration, which adds two. The newsupabase/tests/armory_lookup.sqlchecks their grants and security settings itself. Whenarmory.sqlnext runs on all migrations, its count should become 24.How you tested it
web/src/lib/armory-lookup.test.ts: realForeverApiwith a mockedfetch, on the memory store. Cases:web/src/app/search/people/lookup.test.ts, on the route. It checks that:lookup=1from the browser runs one lookup counted by the quotalookup=1(header type-ahead), for Googlebot, or for a python scriptpeople-search.test.ts: a full name nobody stored is looked up once and listed. There is no lookup when the armory has the name, when the armory failed, or when the armory switch is off.intake-quota.test.ts: thelookupscope allows 20 an hour per address in development and sendsp_scope: "lookup"to the RPC.people-results.test.tsx: the /search page now asks withlookup=1. The header'ssearch-bar.test.tsxis unchanged: it still asks without it.SQL on PGlite (
kb/test/armory-lookup-sql.test.ts, which runssupabase/tests/armory_lookup.sql) checks:lookupquota at 20 per address and 300 globalcd web && npx vitest run --maxWorkers=4 --testTimeout=60000: 5997 passed, 30 skipped (528 files), 0 failed. This ran before theintake-quotatest was added; that file passes alone (24 tests).cd web && npm run typecheck: clean.npm run lint: 4 warnings, all in files this PR doesn't touch.cd kb && npm test: 2203 passed, 1 skipped (186 files).npm run typecheck: clean.cd web && npx next build: passes;/search/peoplestays a dynamic route.The title reads
type(scope): plain sentenceCommits are signed off
No scraped, datamined or spoiler data; all fixtures are invented names
For an addon change: n/a
CHANGELOG.mdhas one line under[Unreleased](### Added,Site:)The area label is set
Docs are updated (
docs/ENVIRONMENT.md,web/.env.example, code comments, migration header)