From b3d913e64e123b681cd0c975f88581d88a0464cc Mon Sep 17 00:00:00 2001 From: Aaron Sachs <898627+asachs01@users.noreply.github.com> Date: Sat, 5 Sep 2026 05:28:01 +0000 Subject: [PATCH 1/3] fix(agent-template): sync #154 log-namespacing + datetime into HEARTBEAT.md template The #154 fix (agent-name-namespaced kb-ingest logs) landed in the shared template but was never rolled out to most agents' checked-in HEARTBEAT.md copies. A same-night fleet sweep (KB cross-contamination false alarm, retracted after investigation) surfaced the gap and separately found that agent-name+PID alone still collides across days/hours on a long-running box (time-of-day-only and PID-only filenames observed reused). This adds a UTC date+time component to the log filename, closing both collision axes in one form. The 15 per-agent checked-in copies (gitignored under orgs/) were patched directly as part of task_1788584322313_13550432; this PR is the template-side half so future deploys inherit the fix. --- templates/agent/HEARTBEAT.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/templates/agent/HEARTBEAT.md b/templates/agent/HEARTBEAT.md index 2b447d006b..5e1c41e243 100644 --- a/templates/agent/HEARTBEAT.md +++ b/templates/agent/HEARTBEAT.md @@ -175,11 +175,16 @@ timeout and it still failed" reports tonight never actually tested a raised time **RUN EACH FILE AS A BACKGROUND / UNCAPPED CALL. NEVER GATE COMPLETION ON A TIMEOUT, EXIT CODE, OR ELAPSED TIME.** ```bash cortextos bus kb-ingest "$f" --org "$CTX_ORG" --agent "$CTX_AGENT_NAME" --scope private --force \ - > "/tmp/kb-ingest-${CTX_AGENT_NAME}-$$-$(basename "$f").log" 2>&1 & + > "/tmp/kb-ingest-${CTX_AGENT_NAME}-$(date -u +%Y%m%d-%H%M%S)-$$-$(basename "$f").log" 2>&1 & ``` `$$` alone collides across concurrent agent sessions on a shared host — `$CTX_AGENT_NAME` in the filename is required, not cosmetic (infra, 2026-08-25: a single heartbeat's log came back -interleaved with 7+ other agents' ingest output under the un-namespaced path). +interleaved with 7+ other agents' ingest output under the un-namespaced path). `$CTX_AGENT_NAME-$$` +alone is still not sufficient: PIDs get reused across days on a long-running box, and time-of-day-only +labels (`hb2342`, `hb0342`) collide identically — infra, 2026-09-05, caught its own prior-day 429 log +being misread as current via an hb-time-only filename. The UTC date+time component kills both the +cross-agent and cross-day/cross-hour collisions in one form; PID remains only as the final tiebreaker +for two calls landing in the same second. The only valid completion signal is the literal text `Ingest complete` appearing in that log — not `rc=0`, not a chunk count alone, not silence. If it hasn't appeared by the time you move on to other work, check From 410f8f36046741914a6d5dea26cca99b86f1bfd1 Mon Sep 17 00:00:00 2001 From: Aaron Sachs <898627+asachs01@users.noreply.github.com> Date: Sat, 5 Sep 2026 07:43:59 +0000 Subject: [PATCH 2/3] fix(agent-template): use BASHPID not $$ in kb-ingest log filename CodeRabbit review on PR #178: $$ always expands to the originating shell's PID even inside a backgrounded job, so two &-launched kb-ingest calls in the same loop iteration (same date-second, same PID, same source file) would produce identical filenames and the second silently clobbers the first's log. Verified directly before applying: $$ stays constant across backgrounded jobs from the same parent shell, ${BASHPID} correctly differs per forked job even without an explicit subshell. Swapping to ${BASHPID} closes the gap without touching any other part of the naming scheme. --- templates/agent/HEARTBEAT.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/templates/agent/HEARTBEAT.md b/templates/agent/HEARTBEAT.md index 5e1c41e243..af02fda9ef 100644 --- a/templates/agent/HEARTBEAT.md +++ b/templates/agent/HEARTBEAT.md @@ -175,7 +175,7 @@ timeout and it still failed" reports tonight never actually tested a raised time **RUN EACH FILE AS A BACKGROUND / UNCAPPED CALL. NEVER GATE COMPLETION ON A TIMEOUT, EXIT CODE, OR ELAPSED TIME.** ```bash cortextos bus kb-ingest "$f" --org "$CTX_ORG" --agent "$CTX_AGENT_NAME" --scope private --force \ - > "/tmp/kb-ingest-${CTX_AGENT_NAME}-$(date -u +%Y%m%d-%H%M%S)-$$-$(basename "$f").log" 2>&1 & + > "/tmp/kb-ingest-${CTX_AGENT_NAME}-$(date -u +%Y%m%d-%H%M%S)-${BASHPID}-$(basename "$f").log" 2>&1 & ``` `$$` alone collides across concurrent agent sessions on a shared host — `$CTX_AGENT_NAME` in the filename is required, not cosmetic (infra, 2026-08-25: a single heartbeat's log came back @@ -183,7 +183,12 @@ interleaved with 7+ other agents' ingest output under the un-namespaced path). ` alone is still not sufficient: PIDs get reused across days on a long-running box, and time-of-day-only labels (`hb2342`, `hb0342`) collide identically — infra, 2026-09-05, caught its own prior-day 429 log being misread as current via an hb-time-only filename. The UTC date+time component kills both the -cross-agent and cross-day/cross-hour collisions in one form; PID remains only as the final tiebreaker +cross-agent and cross-day/cross-hour collisions in one form. **Use `${BASHPID}`, not `$$`** — `$$` +always expands to the ORIGINATING shell's PID even for a backgrounded job, so two `&`-launched +kb-ingest calls from the same loop iteration keep the identical `$$` (verified: same date-second + +same `$$` + same file = the second silently overwrites the first's log) — `${BASHPID}` is the actual +forked background process's own PID and reliably differs per job (CodeRabbit catch on PR #178, +2026-09-05, verified before merging). PID remains only as the final tiebreaker for two calls landing in the same second. The only valid completion signal is the literal text `Ingest complete` appearing in that log — not `rc=0`, From 0c1a7ba5b5a446efdec507b5e8d05ea077036876 Mon Sep 17 00:00:00 2001 From: Aaron Sachs <898627+asachs01@users.noreply.github.com> Date: Sat, 5 Sep 2026 07:49:18 +0000 Subject: [PATCH 3/3] revert(agent-template): BASHPID is undefined in zsh, the actual execution shell The previous commit (410f8f36) applied CodeRabbit's ${BASHPID} suggestion to fix a real $$ collision, but ${BASHPID} is a bash-only special variable. The actual shell every agent's Bash tool (and this box's $SHELL) executes commands in is zsh, where ${BASHPID} is undefined and silently expands to empty -- verified directly (ZSH_VERSION set, BASH_VERSION empty). My prior verification ran an explicit `bash script.sh` subprocess, a different and non-representative shell context, and shipped a regression: literal double-hyphen, empty-differentiator filenames in real use. Reverting to $$. The theoretical same-loop-iteration collision CodeRabbit flagged is real in isolation, but the Step 10 loop always launches distinct files per iteration (basename already differentiates them), so it does not occur in the documented usage. Added a note to the template explaining why BASHPID doesn't work here, so it isn't re-proposed blind. --- templates/agent/HEARTBEAT.md | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/templates/agent/HEARTBEAT.md b/templates/agent/HEARTBEAT.md index af02fda9ef..0fdaf70580 100644 --- a/templates/agent/HEARTBEAT.md +++ b/templates/agent/HEARTBEAT.md @@ -175,7 +175,7 @@ timeout and it still failed" reports tonight never actually tested a raised time **RUN EACH FILE AS A BACKGROUND / UNCAPPED CALL. NEVER GATE COMPLETION ON A TIMEOUT, EXIT CODE, OR ELAPSED TIME.** ```bash cortextos bus kb-ingest "$f" --org "$CTX_ORG" --agent "$CTX_AGENT_NAME" --scope private --force \ - > "/tmp/kb-ingest-${CTX_AGENT_NAME}-$(date -u +%Y%m%d-%H%M%S)-${BASHPID}-$(basename "$f").log" 2>&1 & + > "/tmp/kb-ingest-${CTX_AGENT_NAME}-$(date -u +%Y%m%d-%H%M%S)-$$-$(basename "$f").log" 2>&1 & ``` `$$` alone collides across concurrent agent sessions on a shared host — `$CTX_AGENT_NAME` in the filename is required, not cosmetic (infra, 2026-08-25: a single heartbeat's log came back @@ -183,12 +183,18 @@ interleaved with 7+ other agents' ingest output under the un-namespaced path). ` alone is still not sufficient: PIDs get reused across days on a long-running box, and time-of-day-only labels (`hb2342`, `hb0342`) collide identically — infra, 2026-09-05, caught its own prior-day 429 log being misread as current via an hb-time-only filename. The UTC date+time component kills both the -cross-agent and cross-day/cross-hour collisions in one form. **Use `${BASHPID}`, not `$$`** — `$$` -always expands to the ORIGINATING shell's PID even for a backgrounded job, so two `&`-launched -kb-ingest calls from the same loop iteration keep the identical `$$` (verified: same date-second + -same `$$` + same file = the second silently overwrites the first's log) — `${BASHPID}` is the actual -forked background process's own PID and reliably differs per job (CodeRabbit catch on PR #178, -2026-09-05, verified before merging). PID remains only as the final tiebreaker +cross-agent and cross-day/cross-hour collisions in one form. + +⚠️ **`${BASHPID}` was tried and reverted the same day (CodeRabbit's original suggestion on PR #178): +it is a BASH-ONLY special variable and is UNDEFINED in zsh, which is the actual shell every agent's +Bash tool (and this box's `$SHELL`) executes commands in — verified directly (`ZSH_VERSION` set, +`BASH_VERSION` empty, `${BASHPID}` expands to nothing). Shipping it produced literal double-hyphen, +empty-differentiator filenames in the real execution environment despite passing verification via an +explicit `bash script.sh` subprocess — a different, non-representative shell context. `$$` remains +the working differentiator here; it does not distinguish two `&`-launched siblings from the exact +same loop iteration, but the Step 10 loop always launches DISTINCT files per iteration (basename +already differentiates), so that theoretical gap does not occur in the documented usage.** PID +remains only as the final tiebreaker for two calls landing in the same second. The only valid completion signal is the literal text `Ingest complete` appearing in that log — not `rc=0`,