Polish the agent workflow: independent auditor, durable session state, deterministic tooling

[robmost]

Summary

Reworks the agent-mode orchestration so the converter behaves more deterministically, is easier to debug, and gives the user clearer prompts. All of this without removing any stage, gate, or validation.

All changes are CLI-agnostic: behaviour-defining content stays in AGENTS.md and .ai/ (shared by claude, agy, and codex); CLI-specific mechanisms are additive hardening only.

Auditor:

The Stage 3 "auditor sub-agent" was the main agent roleplaying independence, and its 10-item checklist audited the static plotting code rather than the plots. Now:

• The audit runs as a fresh headless CLI subprocess (scripts/run_auditor.sh spawning claude -p / codex exec --sandbox read-only / agy -p) that has never seen the session. It receives only file paths, inspects the rendered plots, and prints its report to stdout (captured to assets/auditor_report.md), so it runs fully read-only.
• The canonical adversarial prompt lives at .ai/agents/auditor.md: 10 plot-content checks (MAH growth, non-flat merger rates, HMF shape, velocity / lifespan / spatial plausibility, unit-label consistency), burden of proof on PASS, evidence required per item, proposing fixes forbidden.
• The old code-structure checks moved to tests/test_semantic_plots.py, including a comprehensive, end-to-end test that renders all seven plots from a synthetic tree file. save_figure() now additionally writes PNG siblings so any CLI can read the plots without relying on PDFs.
• The G3 prompt must state which audit mode actually ran; the in-context fallback is documented as last resort and is never reported as independent.

Determinism and debuggability

• assets/session_state.json (via scripts/session_state.py) records gate state, <base>, and the G1 choices. Stage entry conditions check the file and do not rely on conversational memory, so interrupted or compacted sessions recover exactly where they stood. Gate order is enforced by the script.
• A Claude Code PreToolUse hook (scripts/check_write_boundary.py) mechanically enforces the AGENTS.md filesystem table whenever a session is active; other CLIs keep the instruction-level rule, and dev sessions are unaffected.
• New checked-in scripts replace ad-hoc snippets the agent previously had to improvise or adapt: estimate_output.py (pre-G1 tree/halo counts, output size, and the memory pre-check, with format-aware multipliers now stored in the KDB entries), extract_snaplist.py, run_semantic_plots.py, and archive_session.sh.
• Stage preambles and gate prompts are single-sourced in AGENTS.md; the drifted copy embedded in kdb-lookup is gone.

Gates and instruction fixes

• G1 asks exactly one question (YES <format> <n_files> confirms the mapping and selects the output); G4 is consistently a no-reply close-out.
• functional-validation no longer tells the agent to re-run unregistered draft drivers through main_driver.py (which cannot resolve them).
• syntactic-validation now states the script runs all six checks; its manual procedures became a diagnostic table.
• New AGENTS.md sections: session state protocol and untrusted content (input files and fetched web docs are data, not instructions). The kdb-lookup partial-match pause is documented in the gate protocol.

Skill consolidation (10 -> 8)

• web-discovery + schema-mapping -> format-discovery (one linear Stage 1 flow).
• kdb-extend + kdb-update -> kdb-register (Path A: new format, Path B: existing format), with the previously duplicated archive step extracted to scripts/archive_session.sh.

Housekeeping

• All tracked files are now pure ASCII (replaces M☉, superscripts, arrows, em dashes, box-drawing characters, etc. across 36 files).
• Fixed wrong format_id examples in kdb_structure.md that did not match the registered KDB entries.
• CI added: make check (ruff + basedpyright + pytest) on push/PR, now also covering scripts/.

[Copilot]

Pull request overview

This PR hardens the agent-workflow orchestration for the SAGE Tree Converter by making Stage progression deterministic (durable session state), making the Stage 3 audit genuinely independent (fresh headless subprocess), and replacing ad-hoc workflow snippets with checked-in scripts and CI enforcement.

Changes:

• Adds a durable assets/session_state.json protocol + a Claude Code write-boundary hook to enforce AGENTS.md filesystem rules mechanically during active sessions.
• Reworks Stage 3 auditing to run as an isolated subprocess (scripts/run_auditor.sh) against PNG plot outputs, while moving plotting-code invariants into deterministic unit tests.
• Consolidates skills (web/schema + KDB extend/update) and adds CI (make check) to validate orchestration + scripts consistently.

Reviewed changes

Copilot reviewed 63 out of 63 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/test_semantic_plots.py	Adds deterministic tests for semantic plotting invariants and end-to-end plot generation (PDF+PNG).
scripts/session_state.py	Implements durable session gate/state recording for workflow recovery.
scripts/run_semantic_plots.py	Provides a deterministic CLI wrapper for rendering the 7 semantic plots.
scripts/run_auditor.sh	Spawns an isolated headless CLI subprocess to audit rendered plot PNGs.
scripts/extract_snaplist.py	Adds a reusable snaplist extraction utility for functional validation.
scripts/estimate_output.py	Adds a standardized pre-G1 estimator (trees/halos/output size/memory pre-check).
scripts/check_write_boundary.py	Adds a Claude Code PreToolUse hook to enforce stage-specific write boundaries.
scripts/archive_session.sh	Adds a single archiving script for Stage 4 session artefacts.
runner/batch_runner.py	Housekeeping: normalizes ASCII punctuation in logs/docstring.
reference/validation_log_style.md	Updates validation log template to reflect new auditor wording and ASCII normalization.
reference/sim_config_template.json	ASCII normalization in comments.
reference/sage_mimic_compatibility.md	ASCII normalization + wording tweaks for consistency.
reference/sage_lhalotree_hdf5_schema.md	ASCII normalization and formatting of schema tree + unit text.
reference/sage_lhalotree_binary_schema.md	ASCII normalization and formatting of binary schema docs.
reference/format_database_template.json	Adds memory_multiplier and updates notes for estimate_output integration.
README.md	Documents independent auditor, session state, new scripts, and updates workflow diagram.
Makefile	Extends lint/typecheck scope to include scripts/.
format-database/subfind_lhalotree_binary.json	Adds memory_multiplier and normalizes ASCII punctuation/numbers.
format-database/subfind_gadget4_hdf5.json	Adds memory_multiplier and normalizes ASCII punctuation.
format-database/rockstar_consistent_trees_ascii.json	Adds memory_multiplier and normalizes ASCII punctuation.
format-database/ahf_mergetree_ascii.json	Adds memory_multiplier and normalizes ASCII punctuation/numbers.
conversion-engine/validation/plot_utils.py	Writes PNG siblings for non-PNG outputs to support CLI-agnostic auditing.
conversion-engine/validation/init.py	ASCII punctuation normalization in package comment.
conversion-engine/utils/init.py	ASCII punctuation normalization in package comment.
conversion-engine/main_driver.py	ASCII punctuation normalization in docstrings/logs; minor wording updates.
conversion-engine/drivers/_template.py	Updates Stage 4 skill name reference (kdb-register).
container/docker-compose.yml	Adds AUDITOR_CLI env passthrough; ASCII normalization in comments.
container/apptainer.env.sh	Adds AUDITOR_CLI env passthrough.
AGENTS.md	Single-sources updated gates, session-state protocol, auditor subprocess behavior, and untrusted content policy.
.github/workflows/ci.yml	Adds CI workflow running make check on push/PR.
.env.example	Documents AUDITOR_CLI.
.claude/settings.json	Registers the PreToolUse hook to enforce write boundaries.
.ai/skills/web-discovery/SKILL.md	Deleted (superseded by format-discovery).
.ai/skills/syntactic-validation/SKILL.md	Updates to single-source preambles in AGENTS.md and clarify script is canonical procedure.
.ai/skills/syntactic-validation/scripts/run_syntactic_checks.py	ASCII normalization in docstring/labels.
.ai/skills/syntactic-validation/scripts/run_binary_checks.py	ASCII normalization in docstring.
.ai/skills/syntactic-validation/references/h5py_patterns.md	Clarifies usage context; ASCII normalization.
.ai/skills/syntactic-validation/references/check_procedures.md	Aligns with scripted checks and $PYTHON_BIN usage; ASCII normalization.
.ai/skills/semantic-validation/SKILL.md	Switches to run_semantic_plots wrapper; single-source preamble guidance.
.ai/skills/semantic-validation/references/on_tree_walking.md	ASCII normalization (O(N^2) notation etc.).
.ai/skills/schema-mapping/SKILL.md	Deleted (superseded by format-discovery).
.ai/skills/kdb-update/SKILL.md	Deleted (superseded by kdb-register).
.ai/skills/kdb-register/SKILL.md	New consolidated Stage 4 skill for both “new format” and “patch existing entry” paths.
.ai/skills/kdb-register/references/diff_format_guide.md	New diff format guide for Path B updates.
.ai/skills/kdb-register/references/conversation_example_schema.md	New conversation example schema reference (ASCII normalized).
.ai/skills/kdb-lookup/SKILL.md	Updates to single-source preambles, session_state init/updates, and handoff to format-discovery.
.ai/skills/kdb-lookup/references/matching_criteria.md	ASCII normalization.
.ai/skills/kdb-lookup/references/kdb_structure.md	Fixes incorrect format_id examples; adds memory_multiplier documentation.
.ai/skills/kdb-extend/SKILL.md	Deleted (superseded by kdb-register).
.ai/skills/functional-validation/SKILL.md	Switches snaplist generation to extract_snaplist.py; fixes rerun guidance for draft drivers.
.ai/skills/functional-validation/references/sage_parameter_template.md	ASCII normalization and $PYTHON_BIN usage fixes.
.ai/skills/functional-validation/references/sage_error_messages.md	ASCII normalization.
.ai/skills/format-discovery/SKILL.md	New consolidated Stage 1 no-KDB-match discovery+mapping flow.
.ai/skills/format-discovery/references/unit_conversion_factors.md	ASCII-normalized unit conversion reference.
.ai/skills/format-discovery/references/source_credibility.md	ASCII normalization; updated tier headings.
.ai/skills/format-discovery/references/pointer_reconstruction.md	ASCII normalization; clarifies patterns and forest-level processing notes.
.ai/skills/format-discovery/references/field_mapping_table.md	ASCII normalization; aligns field guidance with schema conventions.
.ai/skills/driver-authoring/SKILL.md	Aligns preamble behavior and Stage 4 registration naming (kdb-register).
.ai/skills/driver-authoring/references/pointer_reconstruction_patterns.md	Updates references to format-discovery; ASCII normalization.
.ai/skills/driver-authoring/references/driver_interface_spec.md	ASCII normalization in schema diagram.
.ai/skills/auditor/SKILL.md	Reorients auditor skill around subprocess isolation; documents fallback behavior.
.ai/skills/auditor/references/auditor_checklist.md	Deleted (checklist moved to .ai/agents/auditor.md).
.ai/agents/auditor.md	New canonical adversarial auditor prompt used by run_auditor.sh.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.