Make docker_build_env.sh editable, add payload override and build stage detection - #15
Merged
Merged
Conversation
atharvas
added a commit
that referenced
this pull request
Aug 23, 2026
…raint Measured the blast radius rather than assuming it: 5 Grafana panels read candidate_containers and all 5 are COUNT(*) over rows. manifest_warnings and build_manifest appear 0 times in the dashboard; the 3 panels reading resource_metrics/test_duration_s all source from error_logs. So changing a column is invisible and deleting a row moves 5 figures at once -- which puts Plan 4's follow-on delete permanently out of scope. Two spec assumptions found false against the real data: 1. The overrides record has NO expected_n field. Its 10 real keys are benchmark_dest, benchmark_storage_key, extra_*_commands, issue_number, oracle_h, owner, repo, pip_pins, restore_regex. So "create the table" and "make #18 live" are different tasks; expected_n ships hand-declared and nullable per operator ruling. 2. Plan 4's ranking scheme is degenerate: zero of the 636 suspect rows have a harbor_runs row (the join is sound -- 7 containers do, all of them fast rows under 720s). Ranking by repo instead. Also found: parser.py never receives base_commit, so invariant #15 would ship inert. test.sh already has it as a Jinja var and must pass it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
atharvas
added a commit
that referenced
this pull request
Aug 23, 2026
Six checks in parser.py, evaluated after the agent has run and echoed into reward.json under an "invariants" block. harbor_runs.reward_payload already stores the whole object, so this needs no migration. Two FATAL (baseline_sha_mismatch, degenerate_baseline) and four warn (oracle_speedup_direction, dilution_ratio, snapshot_asv_factor, baseline_from_cache). Severity means something different here than at build time: a FATAL marks the reward untrustworthy rather than failing a build, since the run is already paid for. Invariant #15 had NO producer on EITHER side. parser.py's argparse never took base_commit, and lsv_init.py recorded no sha at all -- so the check would have shipped comparing two values that did not exist. Both closed: lsv_init.py now records baseline_sha at init, and test.sh passes --base-commit (which it already had as a Jinja var). Guarded by tests that fail if either producer is removed; both proven by deletion. expected_n is read from FORMULACODE_EXPECTED_N, injected at render time rather than fetched: formulacode_task_overrides is RLS-locked with no anon grant and the trial container only carries the anon key. Absent -> the dilution check skips, which is the common case and the honest one. Also fixed a decorative check of my own making: trial_context_keys() was a hardcoded tuple, so deleting a key from the context builder left the producer-coverage test green while the invariant reading it skipped forever. It now introspects the builder, and deleting expected_n fails the test. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ArjunS07
added a commit
that referenced
this pull request
Aug 29, 2026
…taged LSV's own short-circuit (`if not force and all(has_baseline)`) never fires on tasks with unmeasurable benchmarks (asv "skipped: NotImplementedError") because those bids never get a baseline, so force=False still re-measured (307s). Instead, when the baked DB is staged and baked_sha == HEAD, skip initialize_diffcheck entirely and reuse the baked lsv_init_results.json. The staged DB already holds the baselines + coverage that per-trial lsv_measure reads; nothing downstream uses the init result (oracle snapshot capture + BENCHMARK_DIR export use session). Validated: trial-mode lsv_init 307s -> 0s. Correctness unchanged (unmeasured benchmarks contribute no speedup either way; baseline_sha carried for invariant #15). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AuBeJ3cTwhXAWk8Cp5PH2f
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
docker_build_env.shfrom immutable files so agents can add build prerequisites (Cython, distutils, etc.)env_payload_override.jsonsupport: agents can write a corrected package list when original versions are broken_parse_failed_stage()to detect which Dockerfile stage (env/pkg/run) failed from Docker build logsbuild_env_shandenv_payloadoverride tocandidate_containerstableAGENTS.md.j2with new editability rules, stage-specific failure docs, and payload override tipsTest plan
tskit-dev/msprime#1196) to verify end-to-end