feat(predictor): spot_train.sh --preflight-only (Friday shell-run dry path, no promotion)#175
Merged
Merged
Conversation
… path, no promotion) Owed-item #2 of ROADMAP "Friday shell-run — per-module dry-path activation" (P1). Under the Friday shell_run, the PredictorTraining spot state must boot, run import + lib-pin + ArcticDB/universe- freshness connectivity checks, then exit 0 — with NO weight training, NO weight promotion, and ZERO S3/config writes, to catch bootstrap- class breakage (lib-pin drift, sys.path, stale ArcticDB, SSM timeout, image gap) before the real Saturday PredictorTraining. Flag name (verbatim, for the SF keystone follow-on): --preflight-only Insertion point New `if [ "$MODE" = "preflight-only" ]` branch placed AFTER the bootstrap + deps run_ssm steps and BEFORE the `# ── Smoke test` section. It runs one new `run_ssm "preflight-only"` step then `exit 0`, short-circuiting before the smoke step (dry_run=True training) and before the full-training step (dry_run=False). No-train / no-promote / no-write proof (steps SKIPPED under the flag) • run_meta_training() is NEVER invoked — the entire walk-forward + meta-model fit is skipped (it lives inside the smoke/full SSM steps that the exit 0 never reaches). • meta_trainer.py's `if not dry_run:` upload/promote block (the only weights/meta/* + manifest + dated-archive writer) is unreachable. • train_handler.main()'s training_summary write, triple-barrier cutover gate, training email, and health-status write all live AFTER run_meta_training in main() — none are reached. • The probe does only: import alpha_engine_lib + training package (no main() call), run the existing TrainingPreflight (env-var check + S3 bucket reachability — a head/list, no put_object), and a read-only ArcticDB `list_symbols()` + single `read().tail(1)` universe-freshness probe. No put_object, no upload_file, no download_from_arctic, no config write, zero external API (yfinance / Anthropic) calls. Reuses existing python dry path? Partially. The repo's python `dry_run=True` (run_meta_training) still trains the full model and only skips S3 writes — it is a "train-but-don't-promote" smoke, NOT a bootstrap-only preflight, so it does not satisfy the zero-train/zero-API invariant. This change REUSES the existing training/preflight.py `TrainingPreflight` (env + S3 connectivity) rather than rebuilding one, and reuses store.arctic_reader._get_arctic for the read-only ArcticDB probe. No parallel preflight was built. Tests tests/test_spot_train_preflight_only.py (mirrors test_spot_train_aws_region.py static-analysis style): flag parses, dedicated branch exists + exits 0, branch precedes smoke + full-training, step body references no train/promote/write tokens (run_meta_training(/train_main(/put_object/upload_file/ download_from_arctic), reuses TrainingPreflight + imports alpha_engine_lib + read-only list_symbols(), and keeps the #247 AWS_REGION export guard. SSM-transport path for normal modes unchanged. bash -n clean. Full suite 1050 passed (8 new/sibling infra tests green). No new deps, no secrets. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Owed-item #2 of ROADMAP "Friday shell-run — per-module dry-path activation" (P1).
Under the Friday
shell_run, the PredictorTraining spot state must boot the spot, run import + lib-pin + ArcticDB/universe-freshness connectivity checks, then exit 0 — with NO weight training, NO weight promotion, and ZERO S3/config writes. Catches bootstrap-class breakage (lib-pin drift, sys.path, stale ArcticDB, SSM timeout, image gap) before the real Saturday PredictorTraining.Flag name (verbatim, for the SF keystone follow-on):
--preflight-onlyInsertion point
New
if [ "$MODE" = "preflight-only" ]branch placed after the bootstrap + depsrun_ssmsteps and before the# ── Smoke testsection. It runs one newrun_ssm "preflight-only"step thenexit 0, short-circuiting before the smoke step (dry_run=Truetraining) and before the full-training step (dry_run=False).No-train / no-promote / no-write proof — steps SKIPPED under the flag
run_meta_training()is never invoked — the entire walk-forward + meta-model fit is skipped (it lives inside the smoke/full SSM steps that theexit 0never reaches).meta_trainer.py'sif not dry_run:upload/promote block — the onlyweights/meta/*+manifest+ dated-archive/{date}/writer — is unreachable.train_handler.main()'straining_summary_*write, triple-barrier cutover gate, training email, and health-status write all live afterrun_meta_traininginmain()— none are reached.alpha_engine_lib+ the training package (nomain()call), run the existingTrainingPreflight(env-var check + S3 bucket reachability — a head/list, noput_object), and a read-only ArcticDBlist_symbols()+ singleread().tail(1)universe-freshness probe. Noput_object, noupload_file, nodownload_from_arctic, noconfig/*write, zero external API (yfinance / Anthropic) calls.Reuses an existing python dry path?
Partially, and deliberately. The repo's python
dry_run=True(run_meta_training) still trains the full model and only skips S3 writes — it is a "train-but-don't-promote" smoke, NOT a bootstrap-only preflight, so it does not meet the zero-train/zero-API invariant this item requires. This change reuses the existingtraining/preflight.py::TrainingPreflight(env + S3 connectivity) instead of rebuilding one, and reusesstore.arctic_reader._get_arcticfor the read-only ArcticDB probe. No parallel preflight was built.Tests
tests/test_spot_train_preflight_only.py(mirrorstest_spot_train_aws_region.pystatic-analysis style — the SSM/EC2 path can't run in CI): flag parses; dedicated branch exists +exit 0; branch precedes smoke + full-training; step body references no train/promote/write tokens (run_meta_training(/train_main(/put_object/upload_file/download_from_arctic); reusesTrainingPreflight+ importsalpha_engine_lib+ read-onlylist_symbols(); keeps the #247AWS_REGIONexport guard. SSM-transport path for normal modes unchanged.bash -nclean. Full suite 1050 passed (8 new/sibling infra tests green). No new deps, no secrets.🤖 Generated with Claude Code