Baselines-vanishing: this repo's share — correct the seam-test contract with the vpc fix, fix the announce layer, restore vintage semantics

[Polichinel]

Our seat of the baselines-vanishing cluster (register Cluster I; siblings: views-platform/views-pipeline-core#485, views-platform/views-models#405). Everything here should land with (not before) the pipeline-core root-binding fix — sequencing notes in the register.

1. Seam test locks the current (defective) binding — tests/test_vpc_seam_contract.py:112 AST-pins root=data_generated, so the correct per-model fix in vpc will fail our CI until we correct the locked contract in the same coordinated change. Also add one fixture using the real per-model on-disk layout (all current fixtures share a single root, which is why this class was invisible to CI). (C-215/C-192)
2. Announce layer (evaluation.py:194-202, 281) — baselines merge into the constituents list, so an absent baseline is announced under the wrong word; strict_constituents silently covers baselines; an all-comparators-absent (0-of-N) state renders like one routine absence; an undeclared baseline produces no in-report banner at all. (C-219)
3. Arm-on-fix items — 17 pgm configs declare the cm baseline trio and our level/partition guard string-compares and aborts the whole report on mismatch (C-220); _maybe_sort's fall-through sorts AP ascending, i.e. worst-first (C-225). Both must be handled in the same change that makes rows resolve.
4. Vintage semantics (C-222) — the file source has no staleness check, comparison rows carry no run-date/data-version despite frames stamping them, and reports/ has no report-of-record marker. This is why scrape-era reports with baseline rows sit beside newer ones without, with nothing saying which is authoritative.
5. Corruption path (C-221) — non-atomic frame save upstream + unguarded pre-check + a retry that labels deterministic load failures "transient" + the subject's degraded status discarded into config-suggesting notes.

Full entries with evidence and tiering: reports/technical_risk_register.md (C-215–C-226, Cluster I).

🤖 Generated with Claude Code

[Polichinel]

The pipeline-core half has landed on development (views-platform/views-pipeline-core#485, PR #522, merge e50ab5c; ships in 3.3.0). Posting the exact seam-test change so item 1 here is a paste, not a re-derivation. I can't work in this repo — this is for whoever picks it up.

What changed on our side

ReportingStage.generate_evaluation_report no longer constructs MetricFrameFileSource. It constructs views_pipeline_core.managers.reporting.metric_frame_source.PerModelMetricFrameSource(primary_model=…, primary_root=context.model_path.data_generated, run_type=…, target=…), which implements your EvaluationSource port by composing one MetricFrameFileSource per model, each rooted at that model's own data/generated (ModelPathManager(model, validate=False).data_generated; the subject's root is supplied because an ensemble isn't resolvable by name). Your locked layout root/<model>/<run_type>/metricframe_<target> is untouched — only the root each model is looked up under changed. metric_frame / provenance signatures unchanged.

What breaks in your CI against 3.3.0

tests/test_vpc_seam_contract.py::test_stage_constructs_source_with_our_init_contract — _ctor_calls("MetricFrameFileSource") on the stage source finds nothing → assert calls, "stage no longer constructs MetricFrameFileSource". Everything else in that file still passes (the generate(source=…, target=…) call is unchanged; test_frame_path_contract_uses_producers_own_prefix is unaffected).

The correction (replaces that one test)

metric_frame_source = pytest.importorskip(
    "views_pipeline_core.managers.reporting.metric_frame_source"
)


def test_stage_constructs_the_per_model_source_rooted_at_the_subject():
    """The stage builds pipeline-core's per-model EvaluationSource; the SUBJECT is rooted
    at ``context.model_path.data_generated``, every other model at its own (vpc #485)."""
    calls = _ctor_calls("PerModelMetricFrameSource")
    assert calls, "stage no longer constructs PerModelMetricFrameSource"
    sig = inspect.signature(metric_frame_source.PerModelMetricFrameSource.__init__)
    for call in calls:
        assert not call.args
        kwargs = _keywords(call)
        sig.bind(None, **{k: object() for k in kwargs})
        (root_kw,) = [kw for kw in call.keywords if kw.arg == "primary_root"]
        assert isinstance(root_kw.value, ast.Attribute)
        assert root_kw.value.attr == "data_generated"


def test_per_model_source_binds_our_file_source_per_model(tmp_path):
    """Each model's lookup goes through OUR MetricFrameFileSource at OUR locked layout,
    rooted at that model's directory — the contract both sides now hold."""
    roots = {"ens": tmp_path / "ensembles/ens/data/generated",
             "m1": tmp_path / "models/m1/data/generated"}
    source = metric_frame_source.PerModelMetricFrameSource(
        primary_model="ens", primary_root=roots["ens"], run_type="calibration",
        target="sb", root_of=lambda name: roots[name],
    )
    prefix = evaluation_stage.METRICFRAME_DIR_PREFIX
    for model in ("ens", "m1"):
        file_source = source._source_for(model)
        assert isinstance(file_source, MetricFrameFileSource)
        assert file_source._frame_dir(model) == roots[model] / model / "calibration" / f"{prefix}sb"

The second test is also the "one fixture using the real per-model on-disk layout" this issue asks for, at the seam. Our own tests of the class (tests/test_managers/test_metric_frame_source.py) use the same shape with your real MetricFrameFileSource and only MetricFrame.load stubbed.

Sequencing as you wrote it: land this with the 3.3.0 bump, not before — against ≤3.2.0 the new test skips on the importorskip and the old one would pass, so an intermediate state is green either way.

Items 2 and 3 here (announce layer, vintage semantics) are untouched by our change.

[Polichinel]

views-pipeline-core 3.3.0 is on PyPI (https://github.com/views-platform/views-pipeline-core/releases/tag/3.3.0). This is the release that changes the seam: the stage constructs PerModelMetricFrameSource, not MetricFrameFileSource, so your test_stage_constructs_source_with_our_init_contract fails against it. The replacement test is in my comment above; land it with your bump to >=3.3.0. Nothing else in test_vpc_seam_contract.py is affected.

[Polichinel]

Item 1 done in #297 → #298 (main == development at d8122b1), landed with the >=3.3.0 bump as sequenced. Thank you for the paste — it was the right shape and made this a same-day change.

Two things I tightened in review, for the record:

• Test 1 now asserts the exact kwarg set {primary_model, primary_root, run_type, target} — the pasted version would have passed a stage that re-expressed the subject-rooted defect through root_of=. Verified against five mutant stages; all fail.
• Test 2 is now a real on-disk round-trip: frames saved under two roots, resolved through the public metric_frame(), values read back exactly, plus a negative case proving the subject-rooted shape returns None for the other model. The pasted version compared paths via _source_for/_frame_dir without touching disk (your own open C-331 names this gap).

Also in the same PR: views-evaluation>=2.0.0 (held on 09-18 until your 3.3.0 required it), suite 443 green on the released 3.3.0 + 2.0.0, C-215 → Resolved. Honest residuals recorded: C-220 is now armed (its cross-level scrub is owed before the one-time baseline evaluation run), and the end-to-end proof — a real ensemble report with baseline rows — is still owed.

Items 2–4 remain open (announce layer, arm-on-fix traps, vintage semantics).

One ask back: no vpc ADR records the per-model rule yet (your ADR-062 is retire-by-refusal). Our 08-21 draft of exactly this decision is sitting in your documentation/ADRs/ as an uncommitted file — worth filing so the rule outlives the PR.

[Polichinel]

views-reporting 0.3.4 is on PyPI (2026-09-19 02:45, https://github.com/views-platform/views-reporting/releases/tag/v0.3.4) — views-pipeline-core>=3.3.0, views-evaluation[frames]>=2.0.0.

Verified from PyPI with a cold cache: a plain views-reporting now resolves to 0.3.4 + pipeline-core 3.3.0 + views-evaluation 2.0.0 (this morning it silently backtracked to 0.3.3 + vpc 3.2.0 — the stage that blanks baseline rows), and the pinned pair views-reporting==0.3.4 + views-pipeline-core==3.3.0 resolves (it was unsatisfiable on 0.3.3's immutable views-evaluation<2.0.0).

One correction for the record: 0.3.3 did NOT "predate MetricFrameFileSource" — it shipped in 0.3.3 (#264). The refusal views-models hit was the resolver backtracking to vpc 3.2.0 under the eval-pin conflict, not a missing consumer. Same outcome, different cause; 0.3.4 fixes it either way.

What 0.3.4 does not prove: baseline rows in a real ensemble report from a PyPI install still needs the views-models side — baseline-target configs (#405 / C-216), the chungus vpc<3 pins (C-217), the cross-level scrub before anything else (C-220, now armed), and the one-time baseline evaluation run (C-223). Until then a fresh report shows baselines as declared-but-absent — honest, not complete.