Skip to content

Campaign measurement contract — validated inputs, pure plan, budget, campaign-results.json v1 #1012

Open

Description

@marwen-abid

Goal

Define the Go measurement contract for benchmark campaigns. This defines inputs, workload, plan, budget, results schema, and fixtures.

Scope

  • Validate interval (2s, 1s, 600ms), target ref, machine, repetitions, workers, hot-ledger cap, and independent publication/Slack options. No TOML or user-maintained config.
  • Use fixed packs-v3, chunk-1 datasets and literal query-rate lists. Each rate runs for 60 seconds.
  • Build one pure, serial plan: cold ingest, hot ingest, cold queries, hot queries. Give each ingest repetition its own store; queries use the final repetition's store. Apply the hot cap to ingestion and query sampling.
  • Derive the budget from the actual plan, with explicit overhead estimates, checked arithmetic, and a 1,260-minute ceiling. Do not reproduce the old shell budget's approximations.
  • Define and validate campaign-results.json v1. Include identities, complete coverage, per-repetition measurements, and median-low aggregates. Preserve exact integers and distinct scheduled/service latency. Reject malformed or inconsistent results; shedding and underdelivery are numerical data, not failures.

Acceptance

  • Offline Go tests pass without cloud access, native libraries, or a sibling checkout.
  • Tests verify all mappings, 82/86/80 query-rate cells per repetition, a two-repetition execution trace, store/cap rules, and budget bounds/overflow.
  • Small committed fixtures from all three accepted golden runs include source identities, hashes, and documented sanitization. Committed test helpers calculate aggregates from those fixtures and check pinned values. Hand-check median-low for odd and even repetition counts.
  • A complete synthetic result passes validation; invalid mutations fail. The payload contains all measured inputs needed for site conversion without raw S3 access or another aggregation.

Out of Scope

Production execution and aggregation belong to #1013. No provisioning, workflow wiring, uploads, publication, or Slack delivery here. SDF phases, targets, E2E models, and GO/NO-GO assessments remain site-owned.

Related: #936 (Relay envelope), #856 (query command), #972 (prepared datasets), #1016 (site policy).

Metadata

Metadata

Assignees

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions