From e538d88c38f93f95ae5d11e2fb3f4ab559343635 Mon Sep 17 00:00:00 2001 From: Aaron Sachs <898627+asachs01@users.noreply.github.com> Date: Thu, 3 Sep 2026 17:43:35 +0000 Subject: [PATCH] docs(autoresearch): skip-not-orphan when a fixed cron fires before the experiment window closes A cron's fixed cadence and an experiment's measurement window are two independent clocks and will not always land on the same moment. Without explicit guidance, a fired cron forces a choice between evaluating early (violates the window) or letting the experiment look orphaned/stale. Found live: marketing's weekly autoresearch cron fired 18h43m before exp_1786858829_uzaff's window closed. Marketing correctly bridged via heartbeat rather than early-evaluating, but the skill gave no explicit instruction to do so. Adds an explicit skip-not-orphan check at the top of Step 2 in all 5 autoresearch SKILL.md template copies: if the window hasn't closed yet, do nothing with that experiment this fire and let it run to the next one. Live agent copies (14 agents + analyst) already carry this same insertion, applied directly since live agents read live files, not the template. --- .../plugins/cortextos-agent-skills/skills/autoresearch/SKILL.md | 1 + .../plugins/cortextos-agent-skills/skills/autoresearch/SKILL.md | 1 + templates/agent/.claude/skills/autoresearch/SKILL.md | 1 + templates/analyst/.claude/skills/autoresearch/SKILL.md | 1 + templates/orchestrator/.claude/skills/autoresearch/SKILL.md | 1 + 5 files changed, 5 insertions(+) diff --git a/templates/agent-codex/plugins/cortextos-agent-skills/skills/autoresearch/SKILL.md b/templates/agent-codex/plugins/cortextos-agent-skills/skills/autoresearch/SKILL.md index 9cd2d5d922..2d9a962189 100644 --- a/templates/agent-codex/plugins/cortextos-agent-skills/skills/autoresearch/SKILL.md +++ b/templates/agent-codex/plugins/cortextos-agent-skills/skills/autoresearch/SKILL.md @@ -34,6 +34,7 @@ Read the output carefully. Pay attention to: ### Step 2: Evaluate Previous Experiment If there is an active experiment (check `experiments/active.json`): +- **Check whether the experiment's own window has actually closed (`started_at` + `window`) before doing anything else.** Your cron's fixed cadence and the experiment's window are two independent clocks — they will not always land on the same moment (added 2026-09-03, boss/marketing: a weekly cron fired 18h43m before that cycle's window closed). If the window has NOT closed yet: **skip evaluation this fire, do nothing else with this experiment, and let it run to your next cron fire.** Do not evaluate early to avoid "wasting" this fire (violates the window), and do not treat the still-running experiment as orphaned or stale just because this particular fire landed early — it isn't overdue, your clock is just out of phase with it this cycle. Log a one-line heartbeat/memory note that you're bridging, then move on to other cron work this fire. - Compare ALL relevant aspects: the surface changes you made, the context around those changes, and the output metric - Measure the metric using the configured measurement method - Run evaluate-experiment: diff --git a/templates/agent-opencode/plugins/cortextos-agent-skills/skills/autoresearch/SKILL.md b/templates/agent-opencode/plugins/cortextos-agent-skills/skills/autoresearch/SKILL.md index f54b85c5a8..ef1f8bc188 100644 --- a/templates/agent-opencode/plugins/cortextos-agent-skills/skills/autoresearch/SKILL.md +++ b/templates/agent-opencode/plugins/cortextos-agent-skills/skills/autoresearch/SKILL.md @@ -34,6 +34,7 @@ Read the output carefully. Pay attention to: ### Step 2: Evaluate Previous Experiment If there is an active experiment (check `experiments/active.json`): +- **Check whether the experiment's own window has actually closed (`started_at` + `window`) before doing anything else.** Your cron's fixed cadence and the experiment's window are two independent clocks — they will not always land on the same moment (added 2026-09-03, boss/marketing: a weekly cron fired 18h43m before that cycle's window closed). If the window has NOT closed yet: **skip evaluation this fire, do nothing else with this experiment, and let it run to your next cron fire.** Do not evaluate early to avoid "wasting" this fire (violates the window), and do not treat the still-running experiment as orphaned or stale just because this particular fire landed early — it isn't overdue, your clock is just out of phase with it this cycle. Log a one-line heartbeat/memory note that you're bridging, then move on to other cron work this fire. - Compare ALL relevant aspects: the surface changes you made, the context around those changes, and the output metric - Measure the metric using the configured measurement method - Run evaluate-experiment: diff --git a/templates/agent/.claude/skills/autoresearch/SKILL.md b/templates/agent/.claude/skills/autoresearch/SKILL.md index fdabb69f59..6e85a1c72a 100644 --- a/templates/agent/.claude/skills/autoresearch/SKILL.md +++ b/templates/agent/.claude/skills/autoresearch/SKILL.md @@ -35,6 +35,7 @@ Read the output carefully. Pay attention to: ### Step 2: Evaluate Previous Experiment If there is an active experiment (check `experiments/active.json`): +- **Check whether the experiment's own window has actually closed (`started_at` + `window`) before doing anything else.** Your cron's fixed cadence and the experiment's window are two independent clocks — they will not always land on the same moment (added 2026-09-03, boss/marketing: a weekly cron fired 18h43m before that cycle's window closed). If the window has NOT closed yet: **skip evaluation this fire, do nothing else with this experiment, and let it run to your next cron fire.** Do not evaluate early to avoid "wasting" this fire (violates the window), and do not treat the still-running experiment as orphaned or stale just because this particular fire landed early — it isn't overdue, your clock is just out of phase with it this cycle. Log a one-line heartbeat/memory note that you're bridging, then move on to other cron work this fire. - Compare ALL relevant aspects: the surface changes you made, the context around those changes, and the output metric - Measure the metric using the configured measurement method - Run evaluate-experiment: diff --git a/templates/analyst/.claude/skills/autoresearch/SKILL.md b/templates/analyst/.claude/skills/autoresearch/SKILL.md index 4f67789469..bfe29ae199 100644 --- a/templates/analyst/.claude/skills/autoresearch/SKILL.md +++ b/templates/analyst/.claude/skills/autoresearch/SKILL.md @@ -35,6 +35,7 @@ Read the output carefully. Pay attention to: ### Step 2: Evaluate Previous Experiment If there is an active experiment (check `experiments/active.json`): +- **Check whether the experiment's own window has actually closed (`started_at` + `window`) before doing anything else.** Your cron's fixed cadence and the experiment's window are two independent clocks — they will not always land on the same moment (added 2026-09-03, boss/marketing: a weekly cron fired 18h43m before that cycle's window closed). If the window has NOT closed yet: **skip evaluation this fire, do nothing else with this experiment, and let it run to your next cron fire.** Do not evaluate early to avoid "wasting" this fire (violates the window), and do not treat the still-running experiment as orphaned or stale just because this particular fire landed early — it isn't overdue, your clock is just out of phase with it this cycle. Log a one-line heartbeat/memory note that you're bridging, then move on to other cron work this fire. - Compare ALL relevant aspects: the surface changes you made, the context around those changes, and the output metric - Measure the metric using the configured measurement method - Run evaluate-experiment: diff --git a/templates/orchestrator/.claude/skills/autoresearch/SKILL.md b/templates/orchestrator/.claude/skills/autoresearch/SKILL.md index e359d2b06f..5a3b33f9be 100644 --- a/templates/orchestrator/.claude/skills/autoresearch/SKILL.md +++ b/templates/orchestrator/.claude/skills/autoresearch/SKILL.md @@ -35,6 +35,7 @@ Read the output carefully. Pay attention to: ### Step 2: Evaluate Previous Experiment If there is an active experiment (check `experiments/active.json`): +- **Check whether the experiment's own window has actually closed (`started_at` + `window`) before doing anything else.** Your cron's fixed cadence and the experiment's window are two independent clocks — they will not always land on the same moment (added 2026-09-03, boss/marketing: a weekly cron fired 18h43m before that cycle's window closed). If the window has NOT closed yet: **skip evaluation this fire, do nothing else with this experiment, and let it run to your next cron fire.** Do not evaluate early to avoid "wasting" this fire (violates the window), and do not treat the still-running experiment as orphaned or stale just because this particular fire landed early — it isn't overdue, your clock is just out of phase with it this cycle. Log a one-line heartbeat/memory note that you're bridging, then move on to other cron work this fire. - Compare ALL relevant aspects: the surface changes you made, the context around those changes, and the output metric - Measure the metric using the configured measurement method - Run evaluate-experiment: