Skip to content

docs: regenerate openapi.json from the app and add a drift guard - #83

Open
numarasSigmaSoftware wants to merge 7 commits into
IABTechLab:mainfrom
numarasSigmaSoftware:docs/regenerate-openapi
Open

numarasSigmaSoftware wants to merge 7 commits into
IABTechLab:mainfrom
numarasSigmaSoftware:docs/regenerate-openapi

Conversation

@numarasSigmaSoftware

@numarasSigmaSoftware numarasSigmaSoftware commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

Change

Regenerate docs/api/openapi.json with scripts/generate_openapi.py against current main, and add a drift test so it cannot fall behind again.

What the regeneration picks up, classified from the JSON diff rather than by hand:

The regenerated document counts 89 operations. The six hand-written pages that said 87 now say 89; the delta is the operator-key route and the deal list route above. The README group table had not moved with the total (its rows summed to 87); Authentication now counts 5 and Deal Booking 12, so the rows sum to 89.

tests/unit/test_openapi_drift.py builds the document from the app the same way the script does and compares parsed JSON to the committed file, with the regeneration command in the failure message. Modeled on test_docs_inventory_drift.py, which already guards the endpoint and tool inventories. The absence of this guard is why the file fell behind.

Not in this PR, observed while regenerating

The committed OpenAPI document had fallen behind the application.
Regenerated with scripts/generate_openapi.py; no code change.

What the regeneration picks up:
- the operator key route POST /auth/api-keys/operator and its request
  schema (IABTechLab#59)
- operator-auth header parameters on 17 admin paths
  (IABTechLab#39)
- description-only updates on 7 paths, several of them buyer-facing:
  quotes, deals, change requests, negotiation messages, order transition
- the required idempotency_key on change-request creation
  (IABTechLab#64) and on quotes (IABTechLab#50)
- exclusiveMinimum on RateCardEntry.base_cpm (IABTechLab#78)
atc964 added a commit that referenced this pull request Sep 22, 2026
…ct the redacted guardrails

The status route's rounds were serialized whole as raw dicts, and each
internal round carries cumulative_concession_pct, computed as
(base_price - counter_price) / base_price — so
seller_price / (1 - cumulative_concession_pct) reconstructed the
seller's base_price EXACTLY, handing back the anchor the top level had
just dropped. Round rationale on a below-floor offer also states the
floor in prose.

- NegotiationStatusResponse.rounds is now list[NegotiationRoundView], a
  whitelist of round_number / buyer_price / seller_price / action /
  timestamp. concession_pct, cumulative_concession_pct and rationale do
  not cross the wire. Excluding rationale here is boundary filtering on
  this read endpoint only; whether the engine should put the floor into
  rationale text at all remains an open engine-side decision.
- The service keeps projecting full rounds: terminal_round_response
  reads rationale from them internally, so the filter belongs at the
  response model, which is the boundary.
- New test drives the REAL engine (build_negotiation_engine ->
  start_negotiation -> evaluate_buyer_offer -> record_round) with a
  below-floor offer so the round genuinely carries the nonzero
  concession and the floor-naming rationale, then asserts neither
  survives serialization. The original fixture's default 0.0 concession
  and hand-written rationale could not exhibit either leak. Negative
  control: reverting rounds to list[dict] fails this test naming
  cumulative_concession_pct and rationale.
- NegotiationStatusResponse declares quote_id (Optional, default None)
  so the projection #96 adds is passed through instead of silently
  stripped by the pinned response model.
- docs/api/openapi.json regenerated via scripts/generate_openapi.py so
  the drift guard arriving in #83 passes over the merged pair.
atc964 added a commit that referenced this pull request Sep 22, 2026
…ct the redacted guardrails

The status route's rounds were serialized whole as raw dicts, and each
internal round carries cumulative_concession_pct, computed as
(base_price - counter_price) / base_price — so
seller_price / (1 - cumulative_concession_pct) reconstructed the
seller's base_price EXACTLY, handing back the anchor the top level had
just dropped. Round rationale on a below-floor offer also states the
floor in prose.

- NegotiationStatusResponse.rounds is now list[NegotiationRoundView], a
  whitelist of round_number / buyer_price / seller_price / action /
  timestamp. concession_pct, cumulative_concession_pct and rationale do
  not cross the wire. Excluding rationale here is boundary filtering on
  this read endpoint only; whether the engine should put the floor into
  rationale text at all remains an open engine-side decision.
- The service keeps projecting full rounds: terminal_round_response
  reads rationale from them internally, so the filter belongs at the
  response model, which is the boundary.
- New test drives the REAL engine (build_negotiation_engine ->
  start_negotiation -> evaluate_buyer_offer -> record_round) with a
  below-floor offer so the round genuinely carries the nonzero
  concession and the floor-naming rationale, then asserts neither
  survives serialization. The original fixture's default 0.0 concession
  and hand-written rationale could not exhibit either leak. Negative
  control: reverting rounds to list[dict] fails this test naming
  cumulative_concession_pct and rationale.
- NegotiationStatusResponse declares quote_id (Optional, default None)
  so the projection #96 adds is passed through instead of silently
  stripped by the pinned response model.
- docs/api/openapi.json regenerated via scripts/generate_openapi.py so
  the drift guard arriving in #83 passes over the merged pair.
@numarasSigmaSoftware

Copy link
Copy Markdown
Contributor Author

@atc964 @Sirajmx Can you take a look on this PR? I can provided further info if needed.

Sirajmx pushed a commit that referenced this pull request Sep 30, 2026
…egotiation reads (#95)

* fix: stop leaking the seller's floor price and concession budget on negotiation reads

GET /proposals/{proposal_id}/negotiation had no auth dependency and no
response model, and answered with whatever negotiation_service.
get_negotiation_status produced. That projection carried the internal
NegotiationHistory's strategy, base_price, floor_price and
limits.max_rounds, so any caller who knew or guessed a proposal id
learned the seller's floor and its remaining concession budget - the two
things a negotiating seller must never reveal. The shared Negotiation
primitive excludes those same four fields deliberately.

Three changes:

- get_negotiation_status now projects only buyer-visible facts. Dropping
  the fields at the service rather than at the route means no consumer of
  the projection can re-expose them; the one internal consumer
  (apply_terminal_action -> contract_mappers.terminal_round_response)
  reads only negotiation_id and rounds, so nothing needed them.
- The route pins its wire shape with a new NegotiationStatusResponse, so
  a later addition to the history dict cannot silently reach the wire.
- The route resolves a verified buyer context the way its siblings
  (POST /proposals/{id}/counter, POST /api/v1/negotiations/messages) do,
  via _get_optional_api_key_record + _verified_buyer_context. A GET
  carries no body identity to self-assert, so an unverifiable caller
  cannot be established as a party to the negotiation and gets 401.

Not fixed here: the response is authentication-scoped, not buyer-scoped.
NegotiationHistory records no buyer identity, so there is nothing to
compare a requesting buyer against - any authenticated buyer can still
read any proposal's negotiation. Scoping needs a stored party identity,
which is a schema change beyond this defect.

* fix: type the negotiation rounds on the wire so they cannot reconstruct the redacted guardrails

The status route's rounds were serialized whole as raw dicts, and each
internal round carries cumulative_concession_pct, computed as
(base_price - counter_price) / base_price — so
seller_price / (1 - cumulative_concession_pct) reconstructed the
seller's base_price EXACTLY, handing back the anchor the top level had
just dropped. Round rationale on a below-floor offer also states the
floor in prose.

- NegotiationStatusResponse.rounds is now list[NegotiationRoundView], a
  whitelist of round_number / buyer_price / seller_price / action /
  timestamp. concession_pct, cumulative_concession_pct and rationale do
  not cross the wire. Excluding rationale here is boundary filtering on
  this read endpoint only; whether the engine should put the floor into
  rationale text at all remains an open engine-side decision.
- The service keeps projecting full rounds: terminal_round_response
  reads rationale from them internally, so the filter belongs at the
  response model, which is the boundary.
- New test drives the REAL engine (build_negotiation_engine ->
  start_negotiation -> evaluate_buyer_offer -> record_round) with a
  below-floor offer so the round genuinely carries the nonzero
  concession and the floor-naming rationale, then asserts neither
  survives serialization. The original fixture's default 0.0 concession
  and hand-written rationale could not exhibit either leak. Negative
  control: reverting rounds to list[dict] fails this test naming
  cumulative_concession_pct and rationale.
- NegotiationStatusResponse declares quote_id (Optional, default None)
  so the projection #96 adds is passed through instead of silently
  stripped by the pinned response model.
- docs/api/openapi.json regenerated via scripts/generate_openapi.py so
  the drift guard arriving in #83 passes over the merged pair.
…oute

Main gained GET /api/v1/deals, so the committed document fell one
operation behind and the drift test would fail. Regenerate it and move
the hand-written totals from 88 to 89. The README group table also
catches up: Authentication counts the operator-key route (5) and
Deal Booking counts the list route (12), so the rows sum to the total.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main updated the list_products and get_product docstrings to describe
the stored inventory-type override, so the committed document's two
descriptions fell behind and the drift test failed after merging main.
No operations changed; the total stays 89.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Kishore-2572

Copy link
Copy Markdown
Contributor

Thanks @numarasSigmaSoftware. The drift guard is a good addition. It caught every real drift I tried, ignored formatting-only differences, and gave the same result across .env and setting changes and on Python 3.11 and 3.12. The endpoint counts are right too. One thing needs fixing before merge: main has moved, and merged as-is this PR makes its own new test fail on main.

Blocking

The drift test fails once merged with main

Since this branch last merged main, #111 (Index Exchange) added account_id to SSPDealDistributeRequest. Git merges docs/api/openapi.json with no conflicts, but the result no longer matches what the app generates. That field is the only difference:

/components/schemas/SSPDealDistributeRequest/properties/account_id   (in the app, missing from the doc)

Full suite, run locally:

  • This branch: 1,697 passed, 0 failed.
  • Current main: 1,874 passed, 0 failed.
  • This branch merged with main: 1 failed (tests/unit/test_openapi_drift.py::test_openapi_json_matches_app), 1,874 passed.

CI is green because it tests the branch as it is, not the merge result. Merging main in again and running python scripts/generate_openapi.py fixes it.

That's the guard doing its job; it just needs to happen before merge. To keep future PRs from doing the same thing

Minor

The change list in the description is out of date

#95 regenerated openapi.json on main on Sep 30, so most of the listed changes are already there: the operator-key route, the auth header parameters, idempotency_key, and exclusiveMinimum. Compared with the merge base, this PR's document diff is only:

  • GET /api/v1/deals and its DealListResponse schema
  • Description changes on GET /products and GET /products/{product_id}

After regenerating, the description should list just what the PR adds on top of main. The count updates (87 → 89) are still correct and needed.

What I tested

  • Drift guard: it failed for an edited version, a removed path, a changed route description, and a deleted document. It passed for a formatting-only rewrite (re-indented, sorted keys), as its
    docstring says.
  • Generator: running scripts/generate_openapi.py on this branch reproduces the committed file byte for byte.
  • Environment: same result with the repo .env, with 9 different settings changode, auth off, Redis, ad server type, CORS and root path), on Python 3.11 and 3.12,and when run from a different directory.
  • Counts:
    • The app has 89 operations across 25 tags.
    • The README group table has 25 rows summing to 89, and each row matches its tag c
    • No hand-written page still says 87 or 88, and docs/reference/endpoints.md says 89.
    • test_docs_inventory_drift.py passes.
  • Description claims: in the regenerated document, POST /auth/api-keys/operator and GET /api/v1/deals are present, idempotency_key is required on QuoteRequest and CreateChangeRequestModel, and RateCardEntry.base_cpm has exclusiveMinimum: 0. mesorsecurity` arrays, as the description says.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants