Repository navigation
feat(dottie-os): local fail-open Trackio for HELPER FT - #86
jcdavis131 wants to merge 28 commits into
Conversation
Stage-5 factory-trainer kick re-hashes pack/train/holdout against MANIFEST.json before training may start. Mismatch, missing file, or missing sha field exits non-zero with expected vs actual. Success writes evidence JSON next to the run log. No override, no auto-promote. Co-authored-by: Cam Davis <jcdavis131@gmail.com>
Require ticket/HQ SHA-256 anchors on the CLI, refuse host-rewritten pack+MANIFEST, hash-while-copy into a 0700 run dir, and reject symlinks. The train hook cannot be called without passing the gate. Co-authored-by: Cam Davis <jcdavis131@gmail.com>
After the integrity gate passes, kick_helper_ft.py POSTs /train/kickoff with the named pack_id, shas, run-private copies, and evidence. The forge-runner handler refuses inbox:latest, missing evidence, and sha/pack mismatches. Evidence is atomic and unique per run_id; copies are re-hashed after train; split soundness and true-int row counts refuse bad packs. No override, no auto-promote. Co-authored-by: Cam Davis <jcdavis131@gmail.com>
The kickoff handler hashes run-private pack/train/holdout itself and compares those digests with the request shas and the CLI/HQ anchors. Evidence ok:true is not trust. POST /train/kickoff is the only HELPER FT start route; inbox:latest and sibling routes refuse before the trainer hook. Tests use the real WSGI TestClient and stub only launch. Co-authored-by: Cam Davis <jcdavis131@gmail.com>
Fetch GET {SWARM_HQ_URL}/api/packs/<pack_id>/pin over HTTPS. Refuse unless
recordedBy=cam, stampVerified, pack_id matches, and all three shas match
MANIFEST and the run-private copies. No ticket, Mill-signed, --cam-pin
paste, or Redis fallback; unreachable HQ refuses. :8765 re-checks the
same pin. Tests mock the HQ HTTP API.
Co-authored-by: Cam Davis <jcdavis131@gmail.com>
Pin refusal cases now hit urllib.request.urlopen: missing stamp, stamp for another pack, one sha off, non-Cam recordedBy, and unreachable HQ each refuse before kickoff. No --cam-pin paste path. Co-authored-by: Cam Davis <jcdavis131@gmail.com>
--cam-pin is an unknown-argument refusal. CAM_PIN and a local cam-pin.json are refused before hashing or kickoff. The :8765 handler re-fetches the Swarm HQ pin itself; a request-body pin is not provenance. Co-authored-by: Cam Davis <jcdavis131@gmail.com>
Send x-agent-token from SWARM_HQ_AGENT_TOKEN (never logged or written to evidence). Parse the flat pin JSON; unknown or missing keys refuse. 401 unauthorized and 404 missing pin refuse with distinct reasons. Co-authored-by: Cam Davis <jcdavis131@gmail.com>
GET /api/packs/<pack_id>/pin now includes rows_total, rows_train, rows_holdout, and pack_matches. kick_helper_ft and :8765 refuse unless stampVerified and pack_matches are both true, then still compare the pin's pack_id, three shas, and three row counts with MANIFEST and the counted run-private copies. pack_matches is never sufficient. Co-authored-by: Cam Davis <jcdavis131@gmail.com>
Verify Ed25519 cam_sig over swarm-hq-pin-v1 canonical bytes against the committed trust pubkey. Hardcode SWARM_HQ_ORIGIN (placeholder refuses), drop SWARM_HQ_URL, refuse any 3xx, and never send the agent token off-origin. default_launch_trainer cannot return launched:true; forge_runner wires the :8765 gate. Copies are 0400 in 0700; post-train re-hash runs after training. Co-authored-by: Cam Davis <jcdavis131@gmail.com>
Refuse HELPER FT unless trust key, key id, and cam_pin_spec match the committed HEAD blobs and HEAD is an ancestor of origin/main. Cam owns those paths in CODEOWNERS. Production verify denylists the HQ vector test public key; vector tests use a helper that never reads trust/. Co-authored-by: Cam Davis <jcdavis131@gmail.com>
Swarm HQ vector at a2cf90e5, sha256 ef12aada8b666079513f716fd9dee7c88176fa302050699ce76460b7aa180666. CLI and :8765 canonical/verify paths share the fixture; the test key is never loaded from trust/. Co-authored-by: Cam Davis <jcdavis131@gmail.com>
Cover Critic's shared-vector attack: production verify denylists hgbGCZRFE4CiwVB4rLZ015a8QkWEPPAjZT5UIQKDazk= even if that key was loaded, on both kick_helper_ft and helper_ft_kickoff. Co-authored-by: Cam Davis <jcdavis131@gmail.com>
…opies Decode Ed25519 points and refuse [8]A == identity, non-canonical y >= p, and S >= L on the committed key and signature R/S. Pass the trainer only run-dir copy paths, wait in the training-host process, then re-hash those recorded paths into checkpoint integrity_evidence.json. Document origin/main ancestry as a local-ref hint. Co-authored-by: Cam Davis <jcdavis131@gmail.com>
Require [L]A == identity on the committed key and signature R so a prime-order point plus torsion is refused. After writing 0400 integrity_evidence.json, emit one INTEGRITY_EVIDENCE line with the file sha for the HQ ticket. Co-authored-by: Cam Davis <jcdavis131@gmail.com>
Record HELPER FT runs with trackio.init/log/finish when the optional package is present. Tracking stays local (no Hub sync/tokens), stores under the run dir, and never changes MATCH/REFUSED gate outcomes. Co-authored-by: Cam Davis <jcdavis131@gmail.com>
Assert init config carries the three shas and run_id, losses and Critic eval scores attach, refused kicks log status=refused, and a raising tracker does not change gate outcomes. Co-authored-by: Cam Davis <jcdavis131@gmail.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
log_eval_scores now takes a required 64-hex results_sha256 and logs it with every score. Missing or malformed values skip and warn. Docs state Trackio is a dashboard; verdicts come only from sha-pinned results and HQ events. Co-authored-by: Cam Davis <jcdavis131@gmail.com>
Cover skip-and-warn on missing or malformed results_sha256, attach the sha on valid eval logs, and lock the stamp-gate disclaimer in helper, kick, and README. Co-authored-by: Cam Davis <jcdavis131@gmail.com>
Completion result and sha256_evidence come from the 0400 evidence file kick() wrote, never scraped stdout. Nonzero trainer exit, a signal, or a timeout is REFUSED even when hashes match. kick() POSTs kick and completion events to the pinned HQ origin through open_hq. Co-authored-by: Cam Davis <jcdavis131@gmail.com>
Refuse not_for_train packs on a real kick unless explicit smoke mode. Smoke never POSTs HQ events, marks evidence unscorable, returns no checkpoint path, and deletes ckpt/run-dir copies/inbox. wait() return code is the trainer exit. Only kick() prints HELPER_FT_INTEGRITY_EVIDENCE. Co-authored-by: Cam Davis <jcdavis131@gmail.com>
Walk the whole row state object and refuse risk_hint, halt_class, provenance, agent_config, and category. List up to 10 row-id:field hits. Tests use a 60-row synthetic 185133-shaped fixture and a stripped re-cut; the live pack is not committed. Co-authored-by: Cam Davis <jcdavis131@gmail.com>
Kickoff and kick() refuse inbox:latest, inbox_latest, and the live default inbox dir. The only training-start HTTP route in this repo is POST :8765/train/kickoff. jev-ava on :8769 is not in this tree. Co-authored-by: Cam Davis <jcdavis131@gmail.com>
Mill's amendment: walk the whole state object. Refuse risk_hint, halt_class, provenance, agent_config, and category at state.meta.* and at the top level. Formatting leaks stay pack-time. Co-authored-by: Cam Davis <jcdavis131@gmail.com>
Bring cursor/helper-ft-trackio-53d2 up to date with the integrity-gate branch without rebasing it. Trackio completion now logs sha256_evidence, result, and exit_code from kick()'s return only. Smoke runs log smoke=true and are never scorable. New gates (not_for_train, state.leak, inbox:latest, trainer.exit) log status=refused with the reason. Co-authored-by: Cam Davis <jcdavis131@gmail.com>
The merge push landed while the PR still targeted the old #85 base, so CI (pull_request to main only) did not start. Empty commit after the base change to main. Co-authored-by: Cam Davis <jcdavis131@gmail.com>
lint-and-test failed check_handoff_fresh.py because this branch still carries unsquashed #85 history (36 commits behind 43f4189; budget 20). Main is only 9 behind. Restamp the trusted block at 93de347 with the measured dottie-os 201 and dottie-loop ruff clean. Co-authored-by: Cam Davis <jcdavis131@gmail.com>
| logged = 0 | ||
| for row in rows: | ||
| step = row.get("step") | ||
| step_i = int(step) if isinstance(step, (int, float)) else None |
There was a problem hiding this comment.
🔒 Agentic Security Review
Severity: MEDIUM
log_losses_from_text() does int(step) on trainer stdout/log mappings with no fail-open wrapper. kick() then treats any exception after that call as KickRefused('kickoff'), and :8765 _handle_train_kickoff calls the same helper before re-hash, outside _track_kickoff_result's try/except. A HuggingFace-style line such as {'loss': 0.1, 'step': 1e309} parses to inf and raises OverflowError, so a run that would have matched hashes is refused (or the WSGI handler 500s) because of dashboard parsing.
Impact: Untrusted trainer logs become a live control-plane input to the HELPER FT integrity gate, violating the fail-open invariant that Trackio must not change MATCH/REFUSED.
Reviewed by Cursor Security Reviewer for commit 17e7c33. Configure here.
| run_id = ev_id | ||
| ev_path = body.get("evidence_path") or evidence.get("evidence_path") | ||
| if isinstance(ev_path, str) and ev_path: | ||
| run_dir = Path(ev_path).parent |
There was a problem hiding this comment.
🔒 Agentic Security Review
Severity: MEDIUM
Unauthenticated POST /train/kickoff (127.0.0.1:8765) takes evidence_path from the JSON body, sets run_dir to its parent, and on every non-ok result calls session.refuse() → start() → mkdir(parents=True) of <parent>/trackio. That write happens even when the request is already refused (inbox latest, missing trust, bad mode), as long as run_id is a non-empty string. The path is not lstat'd, not required to exist, and is only filtered by a substring blocklist (/trust/, /dataset(s)/, /factory/).
Impact: Any local peer who can reach the loopback kickoff port can make the training-host process create arbitrary directory trees (and Trackio jsonl under them) with the daemon's uid, including paths that never existed.
Reviewed by Cursor Security Reviewer for commit 17e7c33. Configure here.




#85 has merged to main. This draft now targets
main.Stacked Trackio work from #86, merged onto
origin/mainat09b07651d9e7233c771fe3552c089a535bb84410(includes #89 leakgate fixtures). Normal merge commit, no rebase, no force-push.Adds optional local-only Trackio experiment tracking to the HELPER FT path.
pip install trackiois enough; missing or raising Trackio logs a warning and the run continues. Integrity gate outcomes are unchanged: a REFUSED kick still refuses.Trackio is a dashboard, never stamp-gate evidence. Verdicts come only from the sha-pinned results files and the HQ events.
Merge resolution (#89 + Trackio)
collect_state_leaksgold-predictive detector, generatedleakgate_evalfixtures,pack_dironenforce_pack_gates).sha256_evidence,result, andexit_codefromkick()'s return only; smoke logssmoke=trueand is never scorable; refused kicks logstatus=refusedwith the reason..github/workflows,trust/, CODEOWNERS, and pack-guard steps are unchanged from main.HANDOFF.mdrestamped socheck_handoff_fresh.pystays under the 20-commit budget on this unsquashed stack (main itself was only 9 behind).Tests
python3 -m unittest discover -s apps/dottie-os/tests -v— 201 OK.Head:
17e7c33a9107b27bebf34708adb1fd521102e8e9. Draft. Do not merge until review.