diff --git a/infrastructure/step_function.json b/infrastructure/step_function.json index d33db5c..a78c88e 100644 --- a/infrastructure/step_function.json +++ b/infrastructure/step_function.json @@ -33,9 +33,9 @@ }, "ApplyShellRunDefaults": { "Type": "Pass", - "Comment": "shell_run=true ONLY (keystone + skip-exception rewire — every substantive workload now boots + runs DRY; ZERO skip-exceptions remain). Merges the dry-path control blob UNDER the current state (States.JsonMerge(shellDefaults, $, false) — second arg wins) so an explicit per-flag override in the execution input still takes effect (e.g. {\"shell_run\": true, \"skip_backtester\": true} still skips Backtester; {\"shell_run\": true, \"preflight_args\": \"\"} would run the spots full-fat). Mirrors InitializeInput's defaults-under-input JsonMerge pattern exactly. SPOT states (MorningEnrich, DataPhase1, RAGIngestion, PredictorTraining, Backtester, Parity, Evaluator, AND DriftDetection) boot + run dry via preflight_args=\" --preflight-only\" (LEADING space inside the var; the spot states' final command is a States.Format whose {} is placed immediately after the mode token with NO literal space, so preflight_args=\"\" on the real run yields a byte-identical command — data #259 + #261 + predictor #175 + backtester #224 all expose --preflight-only verbatim, an orthogonal MODIFIER). LAMBDA states with a verified clean no-write dry path are routed dry, NOT skipped, via $.research_dry (the canonical shell-run LLM-dry signal, true here / false on the real run): Research + the EvalJudge chain (EvalJudgeSubmit{FirstSaturday,Weekly}/Poll/Process — dry_run_llm via research #202) + RationaleClustering (research #202) + ReplayConcordance + Counterfactual (backtester #225) all take dry_run_llm.$=$.research_dry; DataPhase2 (dry_run=true — alternative.collect returns ok_dry_run BEFORE any fetch/S3 write), RegimeSubstrate + RegimeRetrospectiveEval (action=dry_run — produce_*(write=False) returns payload before any put_object). The skip-exception rewire (this PR) flipped the prior 5 skip→dry: DriftDetection now uses the same commands.$/States.Format($.preflight_args) Option-C mechanism as the other spot states (data #261 exposed --preflight-only on spot_drift_detection.sh); the eval-judge / rationale-clustering / replay-concordance / counterfactual Lambdas now route dry via dry_run_llm.$=$.research_dry instead of being hard-skipped. ZERO skip-exceptions are force-set here. The #258 Choice-gated skip_* gates are LEFT INTACT and remain valid for targeted operator skips. The two health-check states have NO skip gate by design and run under shell_run (the bootstrap smoke) — their non-blocking Catch absorbs a stale-data sys.exit(1) so a missing-Friday-bar produces only a clearly-Friday-timestamped alert email, NOT a SF-fatal failure (ROADMAP owed-work item 5: a --shell-run-aware staleness tolerance in alpha-engine-dashboard/health_checker.py is a scoped cross-repo follow-on; not a SF-fatal spurious-fail, so deliberately out of this single-file SF PR).", + "Comment": "shell_run=true ONLY (keystone + skip-exception rewire — every substantive workload now boots + runs DRY; ZERO skip-exceptions remain). Merges the dry-path control blob OVER the current state (States.JsonMerge($, shellDefaults, false) — second arg wins, so shellDefaults wins over $) so shell_run=true ACTUALLY enforces dry mode for every keystone control var. The prior arg order (States.JsonMerge(shellDefaults, $, false)) silently fell back to non-dry because InitializeInput unconditionally seeds preflight_args=\"\" + research_dry=false + data_phase2_dry=false + regime_action=\"produce\" into $ — so $ wins meant the seeded non-dry identities won over the dry shellDefaults, and the 2026-05-22 Friday-PM shell-run dry-pass (first execution past the prior trap break) ran full-fat instead of --preflight-only. Skip flags are unaffected (they're not in shellDefaults; the #258 Choice-gated skip_* mechanism handles {\"shell_run\": true, \"skip_backtester\": true} downstream regardless of merge order). The previously-documented user-override-from-input case ({\"shell_run\": true, \"preflight_args\": \"\"} runs spots full-fat) is now intentionally LOST: in practice nobody passes both flags together — passing shell_run=true means you want dry mode for ALL keystone control vars; the comment line saying otherwise was a footgun masking the actual semantic intent. SPOT states (MorningEnrich, DataPhase1, RAGIngestion, PredictorTraining, Backtester, Parity, Evaluator, AND DriftDetection) boot + run dry via preflight_args=\" --preflight-only\" (LEADING space inside the var; the spot states' final command is a States.Format whose {} is placed immediately after the mode token with NO literal space, so preflight_args=\"\" on the real run yields a byte-identical command — data #259 + #261 + predictor #175 + backtester #224 all expose --preflight-only verbatim, an orthogonal MODIFIER). LAMBDA states with a verified clean no-write dry path are routed dry, NOT skipped, via $.research_dry (the canonical shell-run LLM-dry signal, true here / false on the real run): Research + the EvalJudge chain (EvalJudgeSubmit{FirstSaturday,Weekly}/Poll/Process — dry_run_llm via research #202) + RationaleClustering (research #202) + ReplayConcordance + Counterfactual (backtester #225) all take dry_run_llm.$=$.research_dry; DataPhase2 (dry_run=true — alternative.collect returns ok_dry_run BEFORE any fetch/S3 write), RegimeSubstrate + RegimeRetrospectiveEval (action=dry_run — produce_*(write=False) returns payload before any put_object). The skip-exception rewire (this PR) flipped the prior 5 skip→dry: DriftDetection now uses the same commands.$/States.Format($.preflight_args) Option-C mechanism as the other spot states (data #261 exposed --preflight-only on spot_drift_detection.sh); the eval-judge / rationale-clustering / replay-concordance / counterfactual Lambdas now route dry via dry_run_llm.$=$.research_dry instead of being hard-skipped. ZERO skip-exceptions are force-set here. The #258 Choice-gated skip_* gates are LEFT INTACT and remain valid for targeted operator skips. The two health-check states have NO skip gate by design and run under shell_run (the bootstrap smoke) — their non-blocking Catch absorbs a stale-data sys.exit(1) so a missing-Friday-bar produces only a clearly-Friday-timestamped alert email, NOT a SF-fatal failure (ROADMAP owed-work item 5: a --shell-run-aware staleness tolerance in alpha-engine-dashboard/health_checker.py is a scoped cross-repo follow-on; not a SF-fatal spurious-fail, so deliberately out of this single-file SF PR).", "Parameters": { - "merged.$": "States.JsonMerge(States.StringToJson('{\"preflight_args\":\" --preflight-only\",\"research_dry\":true,\"data_phase2_dry\":true,\"regime_action\":\"dry_run\"}'),$,false)" + "merged.$": "States.JsonMerge($,States.StringToJson('{\"preflight_args\":\" --preflight-only\",\"research_dry\":true,\"data_phase2_dry\":true,\"regime_action\":\"dry_run\"}'),false)" }, "OutputPath": "$.merged", "Next": "CheckSkipMorningEnrich" diff --git a/tests/test_sf_friday_shell_run_wiring.py b/tests/test_sf_friday_shell_run_wiring.py index 01bdb4e..dee00bb 100644 --- a/tests/test_sf_friday_shell_run_wiring.py +++ b/tests/test_sf_friday_shell_run_wiring.py @@ -432,16 +432,45 @@ def test_pass_state_routes_into_existing_skip_chain(self, states): assert st["OutputPath"] == "$.merged" assert st["Next"] == "CheckSkipMorningEnrich" - def test_user_input_wins_over_shell_defaults(self, states): - # States.JsonMerge(defaults, $, false) — $ (current state, carrying - # the user input) MUST be the 2nd arg so an explicit - # {"shell_run": true, "skip_backtester": true} still skips it. + def test_shell_defaults_win_over_current_state(self, states): + """States.JsonMerge($, shellDefaults, false) — shellDefaults MUST be + the 2nd arg so the dry-path control vars actually take effect under + shell_run=true. + + Why the prior order broke (2026-05-22 evening incident): the + previous shape was ``States.JsonMerge(shellDefaults, $, false)`` + with the rationale "user input ($) must win for per-flag overrides + like {shell_run: true, skip_backtester: true}." That rationale + confused two distinct things: ``$`` at ApplyShellRunDefaults entry + is NOT raw user input — InitializeInput has already merged its + defaults blob ``{preflight_args: "", research_dry: false, + data_phase2_dry: false, regime_action: "produce"}`` into ``$``. + So ``$ wins`` meant the NON-DRY identity defaults won over the dry + shellDefaults. Result: every shell_run=true execution silently fell + back to a full-fat real Saturday run instead of --preflight-only. + + Caught the morning after the 2026-05-22 Friday-PM dry-pass + post-trap-fix verification: MorningEnrich passed (trap fix worked), + DataPhase1 spot booted and started collecting short interest + + universe returns + fundamentals at full power instead of running + preflight-and-exiting. + + Skip-flag overrides are unaffected by this fix: shellDefaults + contains NO skip_* keys, so a user-passed + ``{shell_run: true, skip_backtester: true}`` survives the merge + unchanged and the downstream Choice gate still skips Backtester. + Only the 4 keystone control vars (preflight_args / research_dry / + data_phase2_dry / regime_action) are forced to dry when shell_run. + That's the actual semantic intent — passing shell_run=true means + you want dry mode for ALL keystone control vars; partial-override + within the dry-control-var set was a footgun, not a feature. + """ expr = self._merge_expr(states) - assert expr.startswith("States.JsonMerge(States.StringToJson(") - assert expr.endswith(",$,false)"), ( - "user input ($) must be the 2nd JsonMerge arg so explicit " - "per-flag overrides win over the shell-run defaults" + assert expr.startswith("States.JsonMerge($,States.StringToJson("), ( + f"merge expr must start with 'States.JsonMerge($,States.StringToJson(' " + f"so shellDefaults (the 2nd arg) wins over $. got: {expr[:80]!r}" ) + assert expr.endswith(",false)") def test_shell_defaults_set_dry_control_vars(self, states): """ApplyShellRunDefaults must set every dry-path control var to its