Skip to content

fix(explorer): ingest tuning_mode from all bundle generations; explicit not-recorded state; no repr hashing#1170

Merged
joeharris76 merged 6 commits into
developfrom
fix/tuning-explorer-ingest
Jul 16, 2026
Merged

fix(explorer): ingest tuning_mode from all bundle generations; explicit not-recorded state; no repr hashing#1170
joeharris76 merged 6 commits into
developfrom
fix/tuning-explorer-ingest

Conversation

@joeharris76

Copy link
Copy Markdown
Owner

Pull Request

Description

Implements TODO tuning-explorer-ingest-mode-extraction-20260712 (review finding C6). The explorer pipeline read only config.tuning_mode, which no seed-corpus bundle populates — all 528 ingested as null and facet-defaulted to an invented "untuned" token. Now: _tuning_mode falls back to execution.tuning_mode (missing stays None — never coerced to a real mode); _tuning_hash hashes only machine-readable dict detail canonically and refuses repr strings; facetMatching.ts gains an explicit NOT_RECORDED_TUNING_MODE="not-recorded" sentinel (legacy "untuned" URL tokens still match null rows — matching semantics unchanged per the ADR-2 gate); TuningBadge renders neutral "Not Recorded" for null/sentinel instead of "Custom Tuning". Moves the TODO to DONE.

Type of Change

  • Bug fix

Testing

  • Full corpus re-ingest (528 bundles / 58 cohorts): dataset now shows 325 tuned / 203 not-recorded, exactly matching the independent w0 sweep
  • Python: 276 explorer_pipeline tests pass (8 new); TS: 775 vitest tests pass after npm ci (7 new facetMatching, 4 new TuningBadge); tsc --noEmit clean; ruff clean
  • Opus review: 0 Critical / 0 Required / 2 Nit. Verified: the mode-only shared hash is benign (tuning_hash is never a join/dedup/cohort key — cohorts group on tuning_mode; receipt display remains honest), facet semantics provably unchanged, badge null-path unreachable in production (all 5 call sites guard).

Public Contract Check

  • No public contract surfaces changed (internal ingest + UI display state).
  • Contract-map impact checked — none.
  • No count claims.

Documentation

  • No user-facing doc changes needed; behavior change is corpus visibility (325 runs now correctly labeled tuned).
  • Regression note: rows previously matching only via the null→"untuned" default now also match "not-recorded".
  • API wording unaffected.

Code Quality

  • ruff + tsc clean; Python and TS suites green.
  • Backwards compatible (legacy URL tokens honored).
  • Tests added on both sides of the language boundary.
  • Contracts valid.

Artifact Hygiene

  • No raw artifacts committed (results.duckdb and sweep logs stayed in /tmp).
  • No binary/raw evidence files.

Notes

Skipped Nits (both outside this TODO's scope_limit, follow-up candidates for tuning-mode-vocabulary-and-facet-implementation-20260712):

  1. Home.tsx facet chip still labels the null state "Not labelled"/untuned while the badge now says "Not Recorded" — label unification belongs to the ADR-2 implementation TODO.
  2. ComparabilityReceipt.formatTuning appends the mode-derived hash suffix — receipt semantics intentionally untouched here.

Part of the tuning remediation batch (#1161). Unblocks tuning-mode-vocabulary-and-facet-implementation-20260712.

🤖 Generated with Claude Code

https://claude.ai/code/session_01QTnjb2i44JUL71kmu9sk1t


Generated by Claude Code

claude added 2 commits July 16, 2026 01:33
…it not-recorded state; no repr hashing

The explorer pipeline read only config.tuning_mode, but all 528 seed-corpus
bundles store the value at execution.tuning_mode (325, all "tuned") or
nowhere (203), so every corpus run ingested as tuning_mode=NULL and
facetMatching null-coalesced that into an invented "untuned" token no
producer emits.

- transformer.py _tuning_mode: config.tuning_mode first, falling back to
  execution.tuning_mode; a bundle that never recorded a mode stays None
  (no invented mode).
- transformer.py _tuning_hash: hash only machine-readable config -- dict
  detail is hashed canonically (json sort_keys); string/repr detail is
  dropped (a repr is not canonical, cosmetic differences changed the hash);
  mode alone may still hash.
- facetMatching.ts: shared NOT_RECORDED_TUNING_MODE ("not-recorded")
  sentinel; not-recorded rows match only an explicit not-recorded
  selection, never a real mode. The legacy "untuned" chip/URL token keeps
  matching the same rows, so current matching semantics are unchanged
  (rule changes gated on ADR-2).
- TuningBadge: null/undefined/sentinel render a neutral "Not Recorded"
  badge instead of "Custom Tuning". ComparabilityReceipt already showed
  "Not recorded" and stays consistent.

Corpus re-ingest verification: 528 bundles -> tuned=325,
NULL(not-recorded)=203, matching the evidence sweep exactly. Python:
276 explorer_pipeline tests pass (8 new). TS: 775 vitest tests pass
(7 new facetMatching + 4 new TuningBadge); tsc --noEmit clean.

Completes TODO tuning-explorer-ingest-mode-extraction-20260712.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QTnjb2i44JUL71kmu9sk1t
@joeharris76
joeharris76 enabled auto-merge (squash) July 16, 2026 01:40

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: faf8fb53f4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread results-explorer/src/lib/facetMatching.ts Outdated
claude added 4 commits July 16, 2026 01:56
…ilters

facetMatching.ts's in-memory matcher already treated NULL tuning_mode
as matching an explicit not-recorded (or legacy untuned) selection,
but facetModel.ts's SQL where-clause builder and queryFilters.ts's
DuckDB query filter only knew the legacy untuned sentinel and a plain
IN (?) clause respectively. ?tuning=not-recorded therefore returned no
rows on Home/listResults and the Query page while working correctly
wherever in-memory matching was used.

Moves the two sentinel tokens to facetModel.ts as the single source of
truth (facetMatching.ts re-exports them for existing consumers) and
generalizes addNullableSentinelClause to accept multiple null-matching
tokens, reused by both SQL call sites.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SKw2UhgHuzYPo3y5MfarD6
… chromium e2e green

Merged origin/develop (spellcheck ran against the PR merge ref, which
pulled in tests/unit/core/tuning/test_ddl_generator_registry.py from
#1167 with two codespell hits) and corrected the flagged words:
"re-used" -> "reused", "re-import/re-declare" -> "re-import or
redeclare". codespell is now clean over the file and all
explorer/transformer paths; the 19 registry tests still pass.

Chromium job triage: the failing e2e/failures/compare-hard-block.spec.ts
specs assert Compare-page guardrail headings (no tuning-badge text) and
pass locally against regenerated fixtures with this branch's changes
(3/3, plus 106-passed full-suite run where a different spec timed out
under load and also passes in isolation) -- data-load timeout flake, not
a regression; no expectation updates required. docs linkcheck failures
are pre-existing external link rot (vldb.org unreachable,
amplab.cs.berkeley.edu expired certificate) in files this PR does not
touch.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QTnjb2i44JUL71kmu9sk1t
@joeharris76
joeharris76 merged commit 39f7b4e into develop Jul 16, 2026
24 of 26 checks passed
@joeharris76
joeharris76 deleted the fix/tuning-explorer-ingest branch July 16, 2026 03:32
joeharris76 pushed a commit that referenced this pull request Jul 16, 2026
Merged origin/develop (PR #1172's module-size allowlist bump to 2946,
plus #1174/#1175/#1170/#1177 tuning soundness/explorer work) - merge was
conflict-free.

The auto-mode warning added in the previous commit pushed
benchbox/cli/commands/run.py to 2957 lines, 11 over the just-bumped 2946
limit in tests/system/test_module_size_thresholds.py. Relocated the
warning logic to benchbox/cli/tuning_resolver.py (which already owns
tuning messaging via display_tuning_resolution) as a new
warn_sql_auto_mode() function, and replaced the ~13-line inline block in
run.py with a single call. Folded the branch's existing generic "using
basic unified config" debug log into the same function (it only ever
fires alongside the auto-mode case), which brought run.py to exactly
2946 lines without bumping the allowlist.

Added unit coverage for warn_sql_auto_mode in test_tuning_resolution.py:
warns on SQL platforms, silent on DataFrame platforms and non-auto modes,
respects quiet=True, and always logs the debug line.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QTnjb2i44JUL71kmu9sk1t
joeharris76 added a commit that referenced this pull request Jul 16, 2026
…ence; honest auto-mode message (#1178)

* docs(tuning): fix env contract, rewrite stale README, document precedence; honest auto-mode message

Re-verified the 2026-07-12 tuning review findings (R5) against HEAD
(post-#1172 resolver hints/keyword-first fixes) before implementing:

- w1: BENCHBOX_TUNING_ENABLED set config key tuning.enabled, which nothing
  at runtime read except a unit test. Removed both env-var mapping
  mechanisms (module-level env_mappings dict and the ConfigManager
  default's environment_overrides entry) from benchbox/cli/config.py, the
  dead "enabled" default key, and the false "activates tuned runs in CI"
  doc claims in docs/usage/configuration.md and
  docs/reference/cli/configuration.md. BENCHBOX_TUNING_CONFIG and
  BENCHBOX_TUNING_PATH behavior is unchanged. Added a CHANGELOG entry.
- w2: rewrote examples/tunings/README.md - the real flag is --tuning (not
  --tuning-config), the named example scripts don't exist (pointed at the
  real examples/unified_runner.py and
  examples/features/tuning_comparison.py instead), file layout matches
  examples/tunings/<platform>/<benchmark>_tuned.yaml, auto-discovery via
  --tuning tuned is the primary UX, and dropped the unqualified "2-10x"
  performance claim.
- w3: documented the actual --tuning resolution order in
  docs/reference/cli/tuning.md (default notuning -> keyword check before
  path check -> tuned's default_config_file/BENCHBOX_TUNING_CONFIG beats
  discovery -> BENCHBOX_TUNING_PATH -> cwd-relative examples/ -> cwd ->
  fallback with warning), the cwd-relative caveat for installed-package
  users, the tuning list/show/platforms subcommands, and the correct
  --output default (<platform>_tuning.yaml / DataFrame's
  <platform>_<profile>_tuning.yaml, not tuning_config.yaml).
- w4: --tuning auto now warns explicitly on SQL platforms that smart
  defaults are DataFrame-only today and the run proceeds with a basic
  config, instead of silently building an untuned config while claiming
  smart defaults were applied. Message change only, no behavior change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QTnjb2i44JUL71kmu9sk1t

* fix(cli): relocate SQL auto-mode warning to resolver; merge develop

Merged origin/develop (PR #1172's module-size allowlist bump to 2946,
plus #1174/#1175/#1170/#1177 tuning soundness/explorer work) - merge was
conflict-free.

The auto-mode warning added in the previous commit pushed
benchbox/cli/commands/run.py to 2957 lines, 11 over the just-bumped 2946
limit in tests/system/test_module_size_thresholds.py. Relocated the
warning logic to benchbox/cli/tuning_resolver.py (which already owns
tuning messaging via display_tuning_resolution) as a new
warn_sql_auto_mode() function, and replaced the ~13-line inline block in
run.py with a single call. Folded the branch's existing generic "using
basic unified config" debug log into the same function (it only ever
fires alongside the auto-mode case), which brought run.py to exactly
2946 lines without bumping the allowlist.

Added unit coverage for warn_sql_auto_mode in test_tuning_resolution.py:
warns on SQL platforms, silent on DataFrame platforms and non-auto modes,
respects quiet=True, and always logs the debug line.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QTnjb2i44JUL71kmu9sk1t

* fix(cli): don't call the SQL --tuning auto fallback config "untuned"

warn_sql_auto_mode's warning called the fallback UnifiedTuningConfiguration()
"basic (untuned)", but ConstraintConfiguration.enabled defaults to True,
so that config has primary/foreign/unique/check constraints all enabled
- the same "constraints-only" state the FALLBACK resolution path already
labels correctly elsewhere in this module. A user comparing against
--tuning notuning (all constraints disabled) could read "untuned" and
assume parity with that baseline when it isn't one. Message-only fix;
the constraints-enabled behavior itself is unchanged.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SKw2UhgHuzYPo3y5MfarD6

---------

Co-authored-by: Claude <noreply@anthropic.com>
joeharris76 pushed a commit that referenced this pull request Jul 16, 2026
…un.py's size guard

Merges origin/develop (PR #1170 explorer ingest, #1177 soundness tests,
#1172 author-experience quickfixes, #1174/#1175) into the provenance
branch and fixes the resulting module-size guardrail trip.

Merge: git merge auto-resolved cleanly (no textual conflicts) - the two
branches touched disjoint regions of every shared file:
- benchbox/cli/tuning_resolver.py: #1172 reordered resolve_tuning()'s
  keyword-vs-path-existence checks (Case 3/4 swap) in a different
  function than my new resolve_template_reference()/promote_tuning_
  provenance() additions; both compose without change.
- benchbox/core/tuning/interface.py: #1174's module-level platform-
  compatibility-map extraction landed near the top of the file (~line
  98-213); my UnifiedTuningConfiguration.get_configuration_hash()
  addition is far below (~line 1525), alongside the pre-existing
  BenchmarkTunings.get_configuration_hash() (~line 908) - both hash
  methods verified present and correctly scoped to their own class.
  benchbox/platforms/base/adapter.py: 40 lines changed by develop, none
  overlapping my tuning_source/tuning_source_file/tuning_config kwarg
  reads.
- tests/unit/cli/test_tuning_resolution.py: develop added
  test_tuned_keyword_not_shadowed_by_local_path and
  test_notuning_keyword_not_shadowed_by_local_path; my
  TestResolveTemplateReference class sits after them - both sets present
  and passing.
- benchbox/core/results/schema.py, models.py, builder.py, loader.py,
  base.py, result_capture.py, duckdb.py, result_factory.py: untouched by
  develop's PRs, so my provenance work there is unaffected.

Fix: the merge pushed benchbox/cli/commands/run.py to 3000 lines against
its 2946-line allowlist entry (tests/system/test_module_size_thresholds.py).
Rather than bump the allowlist, relocated the tuning-provenance-promotion
logic to benchbox/cli/tuning_resolver.py as a new top-level function,
promote_tuning_provenance() (positional args, not the SimpleNamespace-typed
private helper run.py had), and shortened its 4 call sites in run.py to
one line each. Also deduplicated the three near-identical "tuning-related
DatabaseConfig override" dict-building blocks (_dry_run_build_db_config,
_run_direct, _data_or_load_build_db_config) into one shared
_tuning_override_entries() helper (pure mechanical extraction - the three
override dicts differed only in a couple of extra keys, kept explicit at
each call site). Net effect: run.py is back to exactly 2946 lines (the
existing limit, unchanged) with no functional difference - re-verified
below.

Also dropped a defensive `return None` in _data_or_load_build_db_config's
except branch that shadowed the return-None-on-failure fallthrough
pattern already used identically by its sibling _run_direct (both rely on
click's ctx.exit(1) to terminate; matches existing codebase convention,
saved one line).

Verification:
- BENCHBOX_SKIP_TEST_LOCK=1 uv run -- python -m pytest tests/unit/core/results
  tests/unit/core/tuning tests/unit/cli/test_tuning* -q -n 0: 1298 passed,
  3 skipped.
- BENCHBOX_SKIP_TEST_LOCK=1 uv run -- python -m pytest tests/unit/cli -q:
  1489 passed (3 pre-existing, environment-only failures unrelated to this
  change - root-permission artifacts in output-directory validation tests,
  confirmed via git stash comparison against develop HEAD before this branch).
- uv run -- python -m pytest tests/system/test_module_size_thresholds.py
  tests/uat/test_no_cli_surface_drift.py -q: 6 passed (no CLI-surface-drift
  hits on any file this branch touches; no allowlist bump needed).
- make lint: clean.
- E2E: `benchbox run --platform duckdb --benchmark tpch --scale 0.01
  --tuning tuned --non-interactive` (via BENCHBOX_TUNING_PATH pointed at
  the FK-disabled template copy, per the original PR's notes) still
  produces platform.tuning with tuning_source=auto_discovered and a
  populated requested_config_hash, unchanged after the merge and refactor.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QTnjb2i44JUL71kmu9sk1t
joeharris76 pushed a commit that referenced this pull request Jul 16, 2026
Brings in develop's post-branch-cut landings: #1179 (dry-run schema
normalization + ClickHouse DDL corruption fix in metadata.py's own
hand-rolled metadata-table DDL, unrelated to this branch's clickhouse
workload.py tuned-rendering changes), #1170/#1177/#1178 (explorer ingest,
soundness tests incl. the original test_capability_source_drift.py from
PR #1177, docs/resolver), #1185/#1187/#1188/#1189 (corpus, metadata.py
drift-detection widening, template packaging incl. tuning_resolver, FK load
ordering), #1191/#1192 (TODO bookkeeping).

Only one real conflict: tests/unit/core/tuning/test_capability_source_drift.py
was created independently on both sides (this branch's w1 registry-
consolidation tests; PR #1177's pre-existing three-source drift allowlist
tests). Resolved by keeping both: this branch's registry-derivation +
frozen-baseline tests as the primary content, with #1177's
TestGeneratorRegistryVsCompatibilityMapDrift/TestCapabilityMapperVsCompatibilityMapDrift
classes appended unchanged (their assertions cover the three sources'
independent, as-presented-today behavior, which this branch didn't change --
only how _PLATFORM_COMPATIBILITY_MAP and the workload-profile dispatch are
*computed* changed, not what they return). All 16 tests in the merged file
pass.

No other conflicts: benchbox/core/dryrun.py, benchbox/core/tuning/metadata.py,
and benchbox/cli/tuning_resolver.py merged cleanly (three-way, no textual
overlap with this branch's changes) since develop's edits to those files
touch different functions/sections than this branch modified or read from.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QTnjb2i44JUL71kmu9sk1t
github-actions Bot pushed a commit that referenced this pull request Jul 16, 2026
…nd facet semantics (#1182)

* feat(tuning): implement ADR-002 mode vocabulary, fallback labeling, and facet semantics (w1, w2)

ADR-002 (docs/development/tuning-adr-002-mode-vocabulary-fallback-facets.md)
decided the pinned tuning_mode vocabulary {tuned, tuned-fallback, notuning,
auto, custom} plus a not-recorded sentinel, distinct labeling for template-less
"tuned" runs, and refusal of that fallback under --official.

- benchbox/core/tuning/modes.py: new production constants module for the
  five canonical values + NOT_RECORDED sentinel (the shared YAML fixture from
  #1177 pins against this).
- benchbox/cli/tuning_resolver.py: TuningResolution.canonical_mode maps
  (mode, source) onto the pinned vocabulary - TUNED+FALLBACK becomes
  "tuned-fallback", CUSTOM_FILE becomes "custom" (never the raw path), every
  other pair keeps its plain mode value.
- benchbox/cli/commands/run.py:
  - _reject_official_fallback() refuses --official runs whose resolution is
    tuned-fallback, checked immediately after resolution (before any wizard
    offer), for both the direct/non-interactive and interactive dispatch
    paths.
  - Wizard-produced configs (both the fallback-declined-wizard branch and the
    standalone interactive tuning step) now rebuild TuningResolution with
    source=INTERACTIVE_WIZARD (mode TUNED) or BASELINE (mode NOTUNING)
    instead of leaving the stale pre-wizard resolution in place, so a
    genuinely wizard-configured run reads back as "tuned", not
    "tuned-fallback".
  - _canonical_tuning_mode() feeds all three _build_execution_context() call
    sites the resolved canonical mode instead of the raw --tuning CLI value
    (a keyword or a local file path); quick-restart re-resolves via
    resolve_tuning() (non_interactive=True, no wizard re-prompt) so its
    canonical mode reflects the restored platform/benchmark instead of a
    stale pre-restart resolution.
- _project/scripts/explorer_pipeline/transformer.py: explorer ingest now
  validates config/execution tuning_mode against the pinned vocabulary before
  accepting it - a legacy raw file path or the wizard's old "balanced" string
  is treated as not-recorded, never guessed into a bucket (per ADR-2's
  consequences section).

PR #1176 (tuning_source/hash provenance threading) is unmerged and not
present on this branch - confirmed no tuning_source field exists anywhere in
run.py/schema.py/loader.py, so wizard provenance is implemented directly via
TuningResolution.source with no #1176 dependency.

Tests: TestProductionConstantsMatchTheSharedFixture and
TestCanonicalModeMapsResolutionsOntoTheSharedVocabulary pin modes.py and
canonical_mode against the shared fixture; TestFallbackLabelingEndToEnd
drives the CLI end-to-end (BENCHBOX_TUNING_PATH + empty cwd forces a
template-discovery miss) and asserts execution_context.tuning_mode ==
"tuned-fallback", and that --official refuses it. test_transformer.py gets
regression coverage for the legacy-value normalization.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QTnjb2i44JUL71kmu9sk1t

* feat(tuning): explorer facet rule, receipt mechanism warning, and physical_rendering_id secondary facet (w3)

ADR-002 §3 decisions for the explorer:

- TuningBadge.tsx: added tuned-fallback (warning tone) and custom (warning
  tone, distinct from the generic unknown-mode fallback) entries to the
  pinned TUNING_CONFIG map.
- Home.tsx: unified the not-recorded null-state wording to "Not Recorded"
  everywhere (facet chip summary, dropdown option, descriptive copy) -
  previously "not labelled"/"no tuning label", inconsistent with
  TuningBadge's #1170 wording (the "skipped nit" this TODO called out).
- ComparabilityReceipt.tsx: new "Physical tuning mechanisms" field warns
  (status "diff", never a match failure) when two or more results labeled
  `tuned` have non-identical physical_mechanisms sets - e.g. one platform
  rendering six mechanisms and another rendering zero for the same tuned
  template, previously invisible at the tuning_mode facet level (finding
  behind ADR-2). Omitted when fewer than two tuned results are present, or
  when any tuned result predates physical_mechanisms ingest (undefined is
  distinct from a recorded empty set - "rendered nothing" is a real value).
- facetMatching.ts: physicalRenderingIdsMatch() - a matchable secondary-facet
  predicate for TPC benchmarks (absent on either side -> matches; both
  present -> must be equal), plus a physical_rendering_id field on
  FacetMatchRow/ResultRow. Implemented as a predicate function rather than a
  full URL-persisted FACET_KEYS chip: ADR-2 frames this as an on-demand
  narrowing tool for comparisons, not a change to the default coarse facet
  UI, so no facetModel.ts/Home.tsx chip surface was added for it.

Data plumbing: platform.tuning.logical_profile.{physical_mechanisms,
physical_rendering_id} was already emitted by schema.py but never read by
the explorer. Wired end-to-end: transformer.py extracts it into new
DetailResult fields -> duckdb_builder.py `results` table/views carry it
(physical_mechanisms as a comma-joined VARCHAR, matching this pipeline's
existing plain-VARCHAR conventions) -> duckdbQueries.ts -> types.ts
DetailResult/ResultRow. Both new TS fields are optional (like the existing
plans_published precedent) so none of the ~10 existing DetailResult/ResultRow
test factories across the suite needed updating.

Tests: TuningBadge.test.tsx (tuned-fallback/custom cases),
facetMatching.test.ts (tuned-fallback/custom exact-match,
physicalRenderingIdsMatch), ComparabilityReceipt.test.tsx (match/diff/omitted
mechanism-warning cases), Home.test.tsx (2 assertions updated for the
"Not Recorded" wording unification), test_duckdb_browser_contract.py
(new columns added to the schema contract's expected column sets).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QTnjb2i44JUL71kmu9sk1t

* test(tuning): extend external-table guard coverage and complete the ADR-002 TODO (w4, w5)

- tests/unit/cli/test_cli_data_load_modes.py: the external-table guard
  (_tuning_arg_is_tuning_bearing(), added in the w1/w2 commit) now covers
  every tuning-bearing --tuning value, not just the literal "tuned" keyword
  it previously checked at all four call sites. Adds regression coverage for
  --tuning auto and a custom tuning-file path (both now rejected under
  --table-mode external) plus a --tuning notuning-is-still-allowed case, and
  updates the two existing rejects_tuned assertions for the generalized
  error message ("tuning enabled" instead of "--tuning tuned").
- Completes tuning-mode-vocabulary-and-facet-implementation-20260712: status
  Completed, completed_date 2026-07-16, all five work units marked done with
  implementation notes (moved to _project/DONE/main/active/ in the first
  commit of this branch; this commit lands the content update - status,
  completed_date, and per-work-unit notes).

Verification: `uv run -- python -m pytest tests/unit -q -k tuning` (1263
passed), `cd results-explorer && npx vitest run` (794 passed) and
`npx tsc --noEmit` (clean) all green; `todo_cli.py check-graph` passed (116
items, no dangling references).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QTnjb2i44JUL71kmu9sk1t

* fix(explorer): preserve unknown-vs-empty physical_mechanisms through the ingest pipeline

Review finding: the DetailResult.physical_mechanisms unknown-vs-empty
distinction (None = no logical_profile recorded at all; [] = a profile WAS
recorded and genuinely has zero mechanisms) was dead through the production
path -- three collapse points silently coerced both to the same value, so a
legacy tuned bundle (no logical_profile) compared against a modern
zero-mechanism tuned run triggered a false "Tuned runs rendered different
physical mechanisms" warning, misattributing "unknown" as "rendered
nothing".

Fixed all three collapse points:

1. _project/scripts/explorer_pipeline/transformer.py: `_logical_profile()`
   now returns None when no logical_profile object was recorded at all
   (distinct from a present-but-empty dict), and `_physical_mechanisms()` /
   `_physical_rendering_id()` propagate that None through instead of
   defaulting to `[]`. DetailResult.physical_mechanisms is now
   `list[str] | None` (models.py) to carry the tri-state.
2. _project/scripts/explorer_pipeline/duckdb_builder.py: the `results` table
   now stores SQL NULL only when physical_mechanisms is None (unknown);
   `",".join(sorted([]))` naturally produces "" for a recorded-empty list,
   which previously got collapsed into NULL by the `if ... else None`
   truthiness check.
3. results-explorer/src/lib/duckdbQueries.ts getDetailResult(): maps the DB
   row's NULL to `undefined` (unknown) and "" to `[]` (recorded-empty)
   instead of treating both as falsy -> `[]`. ComparabilityReceipt's
   existing `physical_mechanisms === undefined` skip-guard now actually
   fires for legacy/unrecorded data reaching it through the real pipeline,
   not just in component-level tests that constructed DetailResult objects
   directly.

Tests: Python side pins the None-vs-[] distinction in `_physical_mechanisms`
directly (test_transformer.py) and through the full bundle -> DuckDB
pipeline via a two-bundle `results` table query asserting NULL vs "" are
stored distinctly (test_duckdb_browser_contract.py). TS side adds a
production-path test in duckdbQueries.test.ts: builds wide rows shaped as
the real result_detail_metrics view would return them (NULL and ""), runs
them through the real (unmocked) getDetailResult(), and asserts
buildComparabilityFields() correctly (a) omits the mechanism-warning field
for a legacy+modern pairing and (b) includes it as a match for two modern
zero-mechanism results.

Verification: `uv run -- python -m pytest tests/unit/scripts/explorer_pipeline -q`
(284 passed), `cd results-explorer && npx vitest run` (798 passed),
`npx tsc --noEmit` (clean).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QTnjb2i44JUL71kmu9sk1t

* chore: retrigger CI (lost workflow trigger)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QTnjb2i44JUL71kmu9sk1t

* fix(explorer): bump read-model version for physical_rendering_id column

listResults()/detail projections in duckdbQueries.ts now select
physical_rendering_id unconditionally, but the compatibility guard
still accepted v3 snapshots (built before this column existed) since
neither EXPLORER_READ_MODEL_VERSION nor EXPECTED_READ_MODEL_VERSION was
bumped. An old v3 results.duckdb would pass the guard and then hit a
DuckDB binder error on the missing column instead of the intended
rebuild-required message. Bumped both to 4; the existing
db-remediation-pin test keeps them coupled going forward.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SKw2UhgHuzYPo3y5MfarD6

* fix(ci): bump run.py module-size allowlist for ADR-002 tuning-mode vocabulary work

This PR's canonical tuning-mode vocabulary + tuned-fallback honesty wiring
added 126 lines to run.py, tripping the module-size guardrail (limit 2946,
actual 3060).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D5cPJRHok7vur6jxL3NBdZ

* fix(spellcheck): reword hyphenated re-declaring to redeclaring

Same codespell finding fixed on the sibling PR #1180 (these two test files
are shared between both tuning-remediation branches).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D5cPJRHok7vur6jxL3NBdZ

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants