feat(perf): versioned report envelope around the baseline aggregation (#951 increment 4)

CHANGELOG.md

@@ -1,6 +1,7 @@
 # Changelog
 
 ## Unreleased
+- [BE] 🧾 **성능 baseline 버전드 리포트 엔벨로프 (#951 4차 증분)**: `app.perf.baseline_report.build_baseline_report(profile_name, *, repeat, seed=None)`를 추가했습니다. `aggregate_baseline` 원본 출력을 구매자용 엔벨로프로 감쌉니다 — `report_version`, `generated_at`(UTC ISO-8601), `schema_fingerprint`(계측한 workload 스냅샷의 `"sha256:"` 다이제스트, 리포트를 스키마에 소급 결속), `summary`(`{headline, path_count, slowest_path_by_wall_p95}` — wall p95 최댓값 경로를 이름·개수만으로 지목, 소요시간 값 없음). 전체 통계는 `statistics` 키에 그대로 보존. `app.spec.normalization_report`(#947)의 엔벨로프 패턴을 그대로 따르며 임계값·판정 없음. 테스트 8종.
 - [BE] 📊 **성능 baseline 반복 집계 (#951 3차 증분)**: `app.perf.baseline_stats`가 고정 시드의 동일 workload에 대해 `run_baseline`을 `repeat`회 실행하고 경로별 `wall_seconds`·`peak_bytes`를 min·max·mean·p50·p95·p99 분포 요약으로 축약합니다(`statistics.quantiles`, 표준 라이브러리만). 임계값·합격 판정 없음. `python -m app.perf.baseline_stats --profile small --repeat 5 [--json]` CLI, `repeat < 1`은 `ValueError`, 취소 시 부분 집계를 반환하지 않습니다.
 - [BE] 📏 **측정 기반 성능 baseline 하네스 (#951 2차 증분)**: `app.perf.baseline`이 생성된 workload 스냅샷에 대해 순수(부수효과 없는) 처리 경로 — canonical 해시, JSON 왕복, self-diff, PostgreSQL/Snowflake DDL export, 데이터 딕셔너리 Markdown — 를 계측해 경로별 `wall_seconds`·`peak_bytes`·`result_size_bytes`만 기록합니다. 임계값·합격 판정 없음(용량 목표는 이 하네스의 측정값으로 산출). `python -m app.perf.baseline --profile small --json` CLI 포함, 취소 시 `tracemalloc`을 정리하고 부분 리포트를 반환하지 않습니다.
 - [BE] 📈 **성능·용량 프로파일 (1차 증분 — 워크로드 생성기)**: `app.perf.workload_profiles`에 결정론적·익명 스키마 스냅샷 생성기를 추가했습니다. `small`/`medium`/`large` 프로파일이 이슈 #951 표의 스키마·relation·컬럼·FK·인덱스 개수를 정확히 맞추며, 시드 고정 시 바이트 단위로 재현됩니다. 편향 케이스도 제공합니다 — 단일 5,000컬럼 relation, 밀집 FK 클러스터, 깊은 종속 체인, 분리된 컴포넌트, 다국어·따옴표 식별자 + 대형 코멘트, RANGE 파티션 계층. 실제 인명·기관명·운영 데이터 값 없음. 지연·처리량·메모리 임계값은 이 모듈에 넣지 않으며(계측된 baseline에서 산출), `docs/doctoring/performance-and-capacity-profile.md`에 계약과 후속 증분(baseline 하네스·`docs/PERFORMANCE.md`·벤치 워크플로·Rust 결정 게이트)을 기록했습니다.

backend/app/perf/baseline_report.py

@@ -0,0 +1,128 @@
+"""Versioned report envelope for the measured baseline (issue #951).
+
+:mod:`app.perf.baseline_stats` produces the per-path distribution summary
+from repeated baseline runs. This module wraps that raw statistics block in
+a buyer-facing envelope: a stable schema fingerprint of the workload it
+measured, a generation timestamp, a contract version, and a plain-language
+summary an engineer can read without opening the JSON.
+
+The envelope is additive -- the full :func:`aggregate_baseline` output is
+preserved verbatim under ``statistics`` -- so a downstream consumer can
+ignore the envelope entirely.
+
+Like everything under :mod:`app.perf`, this records observations only. It
+carries **no latency, throughput, or memory threshold** and makes no
+pass/fail judgement; capacity targets are set from measured baseline runs
+and never invented here.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import json
+from datetime import datetime, timezone
+from typing import Any
+
+from app.perf.baseline_stats import aggregate_baseline
+from app.perf.workload_profiles import generate_workload_snapshot
+
+#: Report envelope contract version. Distinct from the statistics block's own
+#: fields; bump when the envelope shape changes.
+REPORT_VERSION = "1"
+
+
+def _schema_fingerprint(snapshot: dict[str, Any] | None) -> str:
+    """Return a stable ``"sha256:"``-prefixed fingerprint of a snapshot.
+
+    The snapshot is serialized with sorted keys and a string fallback for
+    non-JSON values, so the same schema always yields the same fingerprint
+    regardless of dict ordering. This matches
+    ``app.spec.normalization_report.schema_fingerprint``; the two should be
+    unified into one shared helper once both land on ``main``.
+    """
+    canonical = json.dumps(
+        snapshot or {}, sort_keys=True, default=str, separators=(",", ":")
+    )
+    return "sha256:" + hashlib.sha256(canonical.encode("utf-8")).hexdigest()
+
+
+def _summarize(statistics: dict[str, Any]) -> dict[str, Any]:
+    """Build the plain-language summary block from an ``aggregate_baseline`` result.
+
+    Picks the path with the largest 95th-percentile wall time as the one an
+    engineer should look at first. Reports names and counts only -- never a
+    duration value, so the summary carries no implied threshold.
+    """
+    paths: dict[str, Any] = statistics.get("paths", {})
+    path_names = sorted(paths)
+    if path_names:
+        slowest = max(
+            path_names, key=lambda name: paths[name]["wall_seconds"]["p95"]
+        )
+    else:
+        slowest = ""
+
+    profile = statistics.get("profile", "?")
+    repeat = statistics.get("repeat", 0)
+    if not path_names:
+        headline = f"{profile} profile: no measured paths."
+    else:
+        headline = (
+            f"{profile} profile, {len(path_names)} measured paths over "
+            f"{repeat} run(s); slowest by wall-time 95th percentile is "
+            f"{slowest}."
+        )
+    return {
+        "headline": headline,
+        "path_count": len(path_names),
+        "slowest_path_by_wall_p95": slowest,
+    }
+
+
+def build_baseline_report(
+    profile_name: str, *, repeat: int, seed: int | None = None
+) -> dict[str, Any]:
+    """Run the repeat-baseline aggregation and wrap it in a versioned envelope.
+
+    Args:
+        profile_name: One of
+            :func:`app.perf.workload_profiles.list_profiles`.
+        repeat: Number of baseline runs to aggregate (forwarded to
+            :func:`app.perf.baseline_stats.aggregate_baseline`; must be >= 1).
+        seed: Optional PRNG seed forwarded to both the aggregation and the
+            fingerprinted workload snapshot, so the fingerprint identifies
+            exactly the schema that was measured.
+
+    Returns:
+        A dict with:
+
+        ``report_version``
+            :data:`REPORT_VERSION`.
+        ``generated_at``
+            UTC ISO-8601 timestamp of this envelope.
+        ``schema_fingerprint``
+            ``"sha256:"``-prefixed fingerprint of the generated workload
+            snapshot that was measured.
+        ``summary``
+            ``{headline, path_count, slowest_path_by_wall_p95}`` -- names and
+            counts only, no duration values.
+        ``statistics``
+            The full :func:`aggregate_baseline` output, unmodified.
+
+        The report contains only observations; it has no thresholds and no
+        verdict.
+
+    Raises:
+        ValueError: Propagated from :func:`aggregate_baseline` if ``repeat``
+            is less than 1.
+        KeyError: If ``profile_name`` is not a known profile.
+    """
+    statistics = aggregate_baseline(profile_name, repeat=repeat, seed=seed)
+    snapshot = generate_workload_snapshot(profile_name, seed=seed)
+    return {
+        "report_version": REPORT_VERSION,
+        "generated_at": datetime.now(timezone.utc).isoformat(),
+        "schema_fingerprint": _schema_fingerprint(snapshot),
+        "summary": _summarize(statistics),
+        "statistics": statistics,
+    }

backend/tests/test_perf_baseline_report.py

@@ -0,0 +1,86 @@
+"""Tests for :mod:`app.perf.baseline_report`.
+
+The envelope must wrap the raw statistics additively, fingerprint the exact
+workload it measured, pick a real slowest path, stay deterministic under a
+fixed seed, and carry no invented performance threshold.
+"""
+
+from __future__ import annotations
+
+import re
+from datetime import datetime
+from pathlib import Path
+
+import pytest
+
+from app.perf.baseline_report import REPORT_VERSION, build_baseline_report
+
+_EXPECTED_PATHS = {
+    "canonical_hash",
+    "json_round_trip",
+    "schema_self_diff",
+    "ddl_export_postgresql",
+    "ddl_export_snowflake",
+    "data_dictionary_markdown",
+}
+
+
+def test_report_has_the_full_envelope_and_preserves_statistics() -> None:
+    report = build_baseline_report("small", repeat=2)
+    assert report["report_version"] == REPORT_VERSION
+    assert set(report) == {
+        "report_version",
+        "generated_at",
+        "schema_fingerprint",
+        "summary",
+        "statistics",
+    }
+    assert set(report["statistics"]["paths"]) == _EXPECTED_PATHS
+    assert report["statistics"]["repeat"] == 2
+
+
+def test_schema_fingerprint_is_sha256_prefixed_and_seed_stable() -> None:
+    a = build_baseline_report("small", repeat=1, seed=11)
+    b = build_baseline_report("small", repeat=1, seed=11)
+    assert a["schema_fingerprint"].startswith("sha256:")
+    assert a["schema_fingerprint"] == b["schema_fingerprint"]
+
+
+def test_different_seeds_fingerprint_differently() -> None:
+    a = build_baseline_report("small", repeat=1, seed=1)
+    b = build_baseline_report("small", repeat=1, seed=2)
+    assert a["schema_fingerprint"] != b["schema_fingerprint"]
+
+
+def test_summary_names_a_real_slowest_path() -> None:
+    report = build_baseline_report("small", repeat=2)
+    summary = report["summary"]
+    assert summary["path_count"] == 6
+    assert summary["slowest_path_by_wall_p95"] in _EXPECTED_PATHS
+    assert "small" in summary["headline"]
+
+
+def test_generated_at_is_timezone_aware() -> None:
+    report = build_baseline_report("small", repeat=1)
+    assert datetime.fromisoformat(report["generated_at"]).tzinfo is not None
+
+
+def test_repeat_below_one_propagates_value_error() -> None:
+    with pytest.raises(ValueError):
+        build_baseline_report("small", repeat=0)
+
+
+def test_unknown_profile_raises_key_error() -> None:
+    with pytest.raises(KeyError):
+        build_baseline_report("enterprise", repeat=1)
+
+
+def test_module_states_targets_are_measured_and_invents_no_threshold() -> None:
+    raw = Path("app/perf/baseline_report.py").read_text(encoding="utf-8").lower()
+    prose = re.sub(r"\s+", " ", raw)
+    assert "measured baseline runs and never invented" in prose
+    assert "no latency, throughput, or memory threshold" in prose
+    assert "makes no pass/fail judgement" in prose
+    assert re.search(r"\b\d+(\.\d+)?\s*(ms|milliseconds|seconds)\b", raw) is None
+    assert re.search(r"p9[59]\s*[<>=:]", raw) is None
+    assert re.search(r"\b\d+\s*(rps|qps|req/s)\b", raw) is None

docs/doctoring/performance-and-capacity-profile.md

@@ -1,7 +1,8 @@
 # Performance & capacity profile
 
 Status: **in progress** — increments 1 (workload generators), 2 (measured
-baseline harness), and 3 (repeat-run aggregation) landed. Tracks issue
+baseline harness), 3 (repeat-run aggregation), and 4 (versioned report
+envelope) landed. Tracks issue
 [#951](https://github.com/ContextualWisdomLab/pg-erd-cloud/issues/951)
 ("[Performance Gap] Establish large-schema SLOs, workload benchmarks, and a
 measured Rust boundary").
@@ -89,6 +90,22 @@ Still observations only: no threshold, no verdict. The percentile targets
 a capacity profile eventually publishes are set from measured baseline
 runs and never invented here.
 
+## Decision — versioned report envelope (this increment)
+
+`app/perf/baseline_report.py` `build_baseline_report(profile_name, *,
+repeat, seed=None)` wraps the raw `aggregate_baseline` output in a
+buyer-facing envelope, mirroring what `app.spec.normalization_report`
+(#947) does for the normalization assessment: `report_version`,
+`generated_at` (UTC ISO-8601), a `schema_fingerprint` (`"sha256:"`-prefixed
+digest of the exact workload snapshot that was measured, so a report can be
+tied back to its schema), and a `summary` block —
+`{headline, path_count, slowest_path_by_wall_p95}` — that names the path an
+engineer should look at first (largest wall-time 95th percentile) using
+**names and counts only, never a duration value**. The full statistics
+block is preserved verbatim under `statistics`. The `schema_fingerprint`
+helper is a local copy of `app.spec.normalization_report.schema_fingerprint`
+for now (the two branches are unmerged); unify them once both land.
+
 ## Deferred (later increments on #951)
 
 - **Baseline harness — remaining paths** — DBML/Mermaid/Prisma/spec export,