feat(runid,fit): run.json output_schema — column roles for tabular outputs#436
Merged
Conversation
Every camdl command writes tabular outputs (simulate: traj.tsv/obs.tsv; fit: draws.tsv/trace.tsv; predict: predictive/observed; survey/profile: grids), and a consumer must reverse-engineer each file's columns — which is the x-axis, which are grouping keys, which are model quantities vs sampler diagnostics. The run store already declares machine-readable schemas for its observation and predictive surfaces; the tabular outputs are the surface that principle doesn't yet cover, and that gap produced a real display bug (the fit-watcher x-axis). Propose a uniform column schema, expressed with a small closed vocabulary of column roles (time / iteration / chain / dimension / state / parameter / observable / quantile / diagnostic / …), surfaced centrally in the one manifest every command writes: run.json (a new identity-inert output_schema field). Declare, don't rename. Chosen over the fit-only fit.meta.json home because run.json is universal — a simulate or profile leaf has no fit.meta.json, so a fit-only home would fork the mechanism per command. Includes the run-store background, the role vocabulary, the per-kind column map, the watcher reproduction, the identity-safety verdict, a one-source-of-truth drift test, and a v1 (fit+sim) / v2 (predict/survey/profile/quantities) staging.
Add the column-schema vocabulary to the run.json RunRecord: a closed ColumnRole enum (time / iteration / chain / replicate / scenario / dimension / state / flow / incidence / param_estimated / param_fixed / observable / quantile / diagnostic), plus TableRole, ColumnSpec, TableSchema, and an `output_schema: BTreeMap<String, TableSchema>` field on RunRecord keyed by leaf-relative path. RunRecord is never hashed and the field is skip_serializing_if empty, so this is identity-inert and golden-neutral until a writer populates it: no run re-keys, and existing manifests round-trip unchanged (pinned by the empty-omitted test). The snake_case wire tags a consumer reads are pinned by the round-trip test. The point of the vocabulary is to let a consumer render any camdl tabular output mechanically — x-axis = time|iteration, group by chain|replicate| scenario, facet by dimension, series = state|param|observable, ribbons = quantile, overlays = diagnostic — instead of reverse-engineering a header. This is the first step of that arc; the fit (draws.tsv / trace) and sim (traj.tsv / obs) writers populate it next. Proposal: docs/dev/proposals/2026-07-15-run-output-column-schema.md
Wire the fit stage to declare its tabular outputs' column schema in
run.json: after the stage runs, classify each written file's real header
(draws.tsv, chain_{n}/trace.tsv, parameter_traces.tsv) by role against the
model's parameter set, and attach it to the record before finalize via a
new ResolvedClaim::set_output_schema setter.
Reading the actual header rather than reconstructing a per-method column
recipe makes the declaration drift-proof by construction: the file is its
own source of truth, and classification needs only set membership, never
column order. A consumer reads role=iteration for the x-axis, role=chain
for grouping, param_estimated for the mixing series — no per-method
knowledge, no hardcoded column names.
Identity-inert: RunRecord is never hashed, and the schema attaches after
the run_id is fixed at claim time. classify() and a real-header tempdir
test pin the role assignment.
Proposal: docs/dev/proposals/2026-07-15-run-output-column-schema.md
Update the proposal to the implemented design: the schema attaches at finalize time via a claim setter (no store-signature change), and it is built by classifying each written file's real header (drift-proof by construction) rather than a per-method recipe plus a drift test. Scope v1 to fit; sim and the rest follow the same seam. Add run-spec §2.5 declaring the run.json.output_schema shape and the column-role vocabulary — the consumer contract for camdl-watch and other tooling.
vsbuffalo
force-pushed
the
hardening/sampler-output-schema
branch
from
July 15, 2026 18:01
4397ba6 to
9bd225c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
run.jsonoutput_schema— column roles for tabular outputs (fit)Every camdl run writes tabular TSVs (
draws.tsv, per-chaintrace.tsv, …), anda consumer must reverse-engineer each file's columns to display it — which is the
x-axis, which are grouping keys, which are model quantities vs diagnostics. This
produced a real bug: the fit watcher plots MCMC traces against draw-position
instead of the true sweep numbers, because nothing declares which column is the
iteration axis (the index is spelled
sweep/step/draw/iterationby method).This adds a machine-readable
output_schemato each run'srun.json, declaringevery tabular file's columns and their roles from a closed vocabulary (
time/iteration/chain/dimension/state/flow/param_estimated/param_fixed/observable/quantile/diagnostic). A consumer readsrole: iterationfor the x-axis,role: chainfor grouping — no per-methodknowledge, no hardcoded names.
Design
written file's real header post-run, not a per-method recipe. The file is
its own source of truth; classification needs only set membership, never
column order.
RunRecordis never hashed, and the schema attachesafter the
run_idis fixed at claim time (verified against the store). It isadditive and skip-serialized when empty, so existing cached runs round-trip
unchanged.
draws.tsv,chain_{n}/trace.tsv,parameter_traces.tsv), the surface behind the bug. Sim (traj.tsv/obs.tsv)and predict/survey/profile follow the same seam (proposal §6, v1.5).
Consumer contract
docs/camdl-run-spec.md§2.5 documents the shape and vocabulary — camdl-watchreads it to fix the trace x-axis (forward the
iterationcolumn, plot against it).Proposal:
docs/dev/proposals/2026-07-15-run-output-column-schema.md