route: report x402 payability, publish why a result ranked, and disclose where the host has an edge - #660
Merged
Merged
Conversation
…why it ranked Two things a seller asked for in #645, and the measurement changed the answer to the first one. PAYABLE. They flagged that two of their listed endpoints are real products but key-gated: a well-formed call returns 401 with a "get a free key" pointer, never a 402 with a challenge. An agent that routes there to pay has nothing to pay against, and the shape generalises across the index. The obvious fix is to rank those rows down. Measured across all 65,462 rows, only 47.8% carry any payability evidence at all and 52.2% carry none, so demoting on absence would bury half the ecosystem for absence of evidence rather than evidence of absence - most of those are ordinary sellers whose price simply was not in the surface we read. So the field REPORTS and the consumer decides: "x402" when there is a price above zero or a registry accepts entry somebody settled against, "unknown" otherwise. Unknown is not "no", and the tests pin that it is never reported as one. Worth recording that no money was ever at risk here: route-execute's external path already requires a positive quoted price and bounds the spend with maxAtomic at the tier cap, so a price-less row is refused outright rather than paid blind. The defect was routing quality on the free ranking endpoint, not settlement. WHY. A seller who loses a routing decision learns nothing from silence. Results now carry the deciding factors in the order the sort applies them: the score, where it came from (slug / name / description), crawl health, the price rank, and the tiebreak order. "Matched on description only" tells a seller to fix their slug. It also makes the neutrality claim checkable by anyone rather than merely stated: everything in there is derived from the seller's own published metadata and our crawl health, and there is no paid placement to hide. 9 assertions added (67 total). Two mutations, both killed: reporting unknown as "none" collapses the distinction the field exists for, and counting an explicit $0 as payable makes free indistinguishable from paid. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…laiming neutrality
The previous commit shipped a comment saying the ranking has "no paid placement
and no operator thumb". The first half is true. The second was not, and it was
written into the source of an open-source index by the people who also sell on
it, which is the worst place to overstate.
What is literally true and needs no qualification: nobody can buy rank. There
is no paid placement, no sponsored slot, and no seller-keyed term anywhere in
the scoring function - it is four text-match rules over the seller's own slug,
name and description.
What is also true is that three rules favour our own catalog:
* it is exempt from the per-seller diversity cap, so it can take more than
ceil(top/3) slots while every external seller cannot
* its health is asserted as 1 rather than measured, because the crawler never
probes itself, and health is the FIRST tiebreak after score
* the listing-injection filter is applied to external rows only
All three are deliberate and defensible. None of them was discoverable without
reading the source. They now ride on every /api/route response in a
`neutrality` block, alongside the switch that removes us from the ranking
entirely, and each result says whether its health was measured or asserted.
A smaller claim anyone can check is worth more than a larger one nobody can,
and a seller finding the cap exemption themselves would have cost more than
the exemption is worth.
12 assertions added (79 total). They guard DRIFT in both directions, which is
the real risk once a disclosure exists: the tests read the scoring function and
fail if any seller-keyed term appears, and they read the source for each
advertised advantage and fail if one is removed without updating the
disclosure. Four mutations, all killed: adding a local-seller score bonus,
silently removing the cap exemption so the disclosure overstates, dropping an
entry from the list, and labelling asserted health as measured.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Answers the two questions a seller asked in #645, and corrects an overclaim the previous PR shipped in a code comment.
1. Payable over x402, or merely findable
They flagged that two of their listed endpoints are real products but key-gated: a well-formed call returns
401with a "get a free key" pointer, never a402with a challenge. An agent routing there to pay has nothing to pay against.The obvious fix is to rank those rows down. Measured across all 65,462 rows, only 47.8% carry any payability evidence and 52.2% carry none — demoting on absence would bury half the ecosystem for absence of evidence rather than evidence of absence. So the field reports and the consumer decides:
payable: "x402"— a price above zero, or a registry accepts entry somebody settled againstpayable: "unknown"— no evidence, which is not the same as "no"No money was ever at risk:
route-execute's external path already requires a positive quoted price and bounds spend withmaxAtomicat the tier cap, so a price-less row is refused outright. The defect was routing quality on the free ranking endpoint, not settlement.2. Why a result ranked
Every result now carries the deciding factors in the order the sort applies them: the score, where it came from (slug / name / description), health, price rank, and the tiebreak order. "Matched on description only" tells a seller to fix their slug instead of guessing.
3. The overclaim
The previous PR shipped a comment reading "no paid placement and no operator thumb". The first half is true. The second was not, written into the source of an open-source index by the people who also sell on it.
Nobody can buy rank — no paid placement, no sponsored slot, no seller-keyed term in the scoring function. But three rules favour our own catalog:
ceil(top/3)slotsAll three are deliberate and defensible; none was discoverable without reading the source. They now ride on every
/api/routeresponse in aneutralityblock, withinclude=externalnamed as the switch that removes us, and each result states whether its health was measured or asserted.Verification
scripts/test-discovery-note.js, in CI$0as payable