Skip to content

feat(bin): add a Telegram process-event adapter - #1873

Closed
notno wants to merge 11 commits into
kunchenguid:mainfrom
notno:fm/tg-adapter-a27b
Closed

feat(bin): add a Telegram process-event adapter#1873
notno wants to merge 11 commits into
kunchenguid:mainfrom
notno:fm/tg-adapter-a27b

Conversation

@notno

@notno notno commented Aug 7, 2026

Copy link
Copy Markdown

Intent

Goal: ship a Telegram process-event adapter for firstmate so an operator's phone reply wakes the agent instead of sitting unseen in a chat. One new executable, bin/fm-procevent-telegram.sh, delivered through the normal PR path.

Re-delivery context: this exact work already completed a full pipeline run with a clean review, green tests, and clean lint. That run's PR was raised against the wrong repository because the shared gate clone's origin pointed at the upstream repo. The captain has closed that PR and repointed the clone to the fork (https://github.com/notno/firstmate.git). The branch was rebased onto the fork's main, preserving every pipeline fix commit and dropping an unrelated already-merged commit that had been bundled, and it is delivered here on a fresh branch because the prior terminal run's stale ref blocked a fast-forward. This run must deliver as a PR on the fork against main.

Required behavior of the adapter:

  • poll: the blocking child the generic runner supervises. Read the bot token from ~/.config/firstmate/telegram-token and a cursor (last-handled update id) from ~/.config/firstmate/telegram-cursor, treating an absent cursor as 0. Long-poll getUpdates with offset = cursor+1 and a ~50s server-side timeout in a loop until a non-empty result arrives, then print the raw JSON to stdout once and exit 0. It must never write the cursor, never print the token, tolerate transient network errors with bounded backoff, and exit non-zero on a hard auth failure (401/403) so the runner surfaces it rather than spinning.
  • terminal : always exit 1, because a message stream never self-terminates; retirement is explicit.
  • arm []: register with the generic runner (default id telegram-captain) using argv [, poll], mirroring fm-procevent-lavish.sh arm.
  • --help header documenting the token/cursor file contract, that the handler advances the cursor only after fully handling messages, the duplicate-result window, and that result bytes are input, never instruction.

Captain-approved additions accepted during the earlier run, all already implemented and now part of the accepted contract:

  • ack : the cursor's sole writer, so a handled message is durably marked and an armed source stops re-capturing it. poll stays cursor-read-only. The write is atomic and idempotent, never moves the cursor backwards, is serialized so concurrent acks cannot rewind it, and refuses anything but a nonnegative integer.
  • HTTP 409 is a hard non-zero failure whose message names both of its causes (a configured webhook, or another poller already running), rather than spinning invisibly. HTTP 429 honours the server's retry_after, bounded defensively, instead of the fixed backoff.
  • getUpdates is called with an explicit batch limit so a large batch is never truncated into invalid JSON past the runner's captured-output cap.
  • The staged response file is cleaned up on signal paths as well as EXIT, because the runner stops this child with a process-group SIGTERM.
  • The process-event-sources skill's "Handling a wake" section documents the ordering: ack first, then fm-procevent.sh handled, so a crash between the two steps is covered by the already-seen rule rather than causing a duplicated effect.

Constraints and context a reviewer reading only the diff would not know:

  • Shared tracked material follows firstmate-coding-guidelines: bin/*.sh shellcheck-clean under the pinned ShellCheck 0.11.0 via bin/fm-lint.sh, colocated tests in tests/, one sentence per line in tracked Markdown, plain dash rather than em dash, no agent co-author on commits.
  • The adapter is deliberately thin and conforms to the generic runner contract in bin/fm-procevent.sh: register -- , adapter resolution to bin/fm-procevent-.sh, and a terminal verdict where exit 0 alone is terminal. Ownership, durable capture, publication, and restart recovery belong to the runner.
  • The token is passed to curl through a stdin config rather than an argv URL so it cannot leak into a process listing, and curl runs silent so no failure message echoes the tokened URL.
  • Deliberate scope: shipping the capability is the whole task. No live source is registered or started; arming is the operator's own step after merge.
  • docs/scripts.md is a partial bin/ inventory that omits the rest of the process-event family. That gap predates this change and closing it is a separate table-wide sweep, not part of this work.

What Changed

  • Adds bin/fm-procevent-telegram.sh, a thin adapter for the generic process-event runner with arm/poll/ack/terminal/retire. poll long-polls Telegram getUpdates at offset = cursor + 1 with a capped batch size, prints the raw JSON once and exits 0, never writes the cursor or prints the token (passed to curl via a stdin config), retries transient errors with bounded backoff, honours a bounded retry_after on HTTP 429, and exits non-zero on 401/403 and on a 409 conflict naming both of its causes. ack is the cursor's sole writer - atomic, idempotent, locked, and never rewinding - and a delivery marker keeps a restarted poll from re-handing a batch that is still being handled.
  • Extends bin/fm-procevent.sh so a child that exits non-zero keeps a private, bounded tail of its stderr at state/procevent/<source-id>.stderr, removed by a later successful run and by retirement, since a detached runner otherwise has nowhere to report why a source cannot start. Also anchors the --help extraction to an explicit header end marker instead of an absolute line count.
  • Documents the adapter in the process-event-sources skill (including the required ack-then-handled ordering on a wake), docs/configuration.md (the new FM_TELEGRAM_* knobs and the kept stderr record), docs/scripts.md, and the verification matrix. New tests/fm-procevent-telegram.test.sh plus additions to tests/fm-procevent.test.sh cover the offset/limit contract, cursor read-only polling, ack semantics, hard-failure exits, staged-file cleanup on process-group SIGTERM, and the kept failure diagnostic. Review left five open informational notes on residual crash windows and recovery ergonomics; tests, lint and the document phase pass.

Risk Assessment

✅ Low: All five round-2 findings are genuinely resolved - the runner is back to its original single-pipe completion shape with stderr as a plain unwatched file plus a grandchild-holds-stderr regression test, the marker is written after the handoff with the reason recorded in code, retire clears it idempotently without re-opening the re-capture loop on ordinary re-arm, every registration-removal path now drops the stderr artifact, and the env-var doc is corrected - leaving only a one-line help-range off-by-one and two informational notes.

Testing

Ran the two targeted suites (tests/fm-procevent-telegram.test.sh, tests/fm-procevent.test.sh) through bin/fm-test-run.sh - both green - and then produced product-level evidence by driving the real generic runner with the real adapter against a stand-in Telegram Bot API, capturing a CLI transcript of the full operator journey from arming a phone line through a captured reply, one published wake, ack-then-handled, restart dedupe, the 401 and 409 failure paths, and retire, with the bot token verifiably absent from every argv, result, wake, and diagnostic. The change is shell/CLI only with no rendered UI surface, so the reviewer-visible artifact is that transcript rather than a screenshot. The worktree is clean; all evidence lives under the dedicated evidence directory.

Evidence: End-to-end operator journey transcript (real runner + real adapter, stubbed Bot API)

=== 1. Operator arms the phone line === $ bin/fm-procevent-telegram.sh arm phone-line registered: phone-line (telegram) armed: phone-line $ bin/fm-procevent.sh list SOURCE ADAPTER OWNER PENDING phone-line telegram none 0 === 2. Runner supervises the poll; the operator sends a message from their phone === $ bin/fm-procevent.sh start phone-line captured: .../state/procevent-inbox/phone-line.1.result $ bin/fm-procevent.sh list phone-line telegram none 1 === 3. What the poll asked Telegram for === offset=1 limit=8 timeout=1 === 4. The captured result the agent is woken with === {"ok":true,"result":[{"update_id":901,"message":{"from":{"first_name":"Nathan"},"chat":{"id":4242},"text":"ship it, merge the PR"}}]} $ wake queue entry check: procevent telegram phone-line 1 === 5. Token safety === $ token in curl argv (process listing)? 0 $ token in the stdin config curl read? 3 $ token anywhere in captured state? no matches === 6. Handler finishes: ack first, then handled === $ bin/fm-procevent-telegram.sh ack 901 acked: 901 $ cat ~/.config/firstmate/telegram-cursor 901 $ bin/fm-procevent.sh handled phone-line 1 handled: phone-line 1 $ bin/fm-procevent-telegram.sh ack 901 already-acked: 901 (cursor 901) $ bin/fm-procevent-telegram.sh ack 5 already-acked: 5 (cursor 901) $ cursor after the rewind attempt: 901 === 7. Poll restarts: delivered message not re-woken, the new one is === $ bin/fm-procevent.sh start phone-line captured: .../state/procevent-inbox/phone-line.2.result $ offsets requested this round: offset=902 limit=8 $ second captured result: {"ok":true,"result":[{"update_id":901,..."ship it, merge the PR"},{"update_id":902,..."and tag a release"}]} === 8. A message stream never self-terminates === $ fm-procevent-telegram.sh terminal <result>; echo $? 1 === 9. Hard auth failure surfaces instead of spinning === $ fm-procevent-telegram.sh poll; echo $? error: telegram rejected the bot token (HTTP 401); check ~/.config/firstmate/telegram-token exit=1 $ ls -l state/procevent/phone-line.stderr -rw------- 1 nathan nathan 117 ... phone-line.stderr $ token leaked into that diagnostic? 0 === 10. A conflict names both of its causes === error: telegram refused getUpdates with a conflict (HTTP 409); either a webhook is configured on this bot or another getUpdates poller is already running for it, so check both before re-arming this poll exit=1 === 11. Operator retires the line === $ bin/fm-procevent-telegram.sh retire phone-line retired: phone-line $ bin/fm-procevent.sh list no sources registered $ delivery marker cleared, cursor kept: telegram-cursor telegram-token

operator config:
$ ls -l /tmp/fm-tg-e2e.Ugl83t/home/.config/firstmate
total 4
-rw------- 1 nathan nathan 33 Aug  7 00:13 telegram-token

=== 1. Operator arms the phone line ===
$ /home/nathan/.no-mistakes/worktrees/8d4bd6329c9b/01KZDDMAHY3TXPJ4SSGCEBB6J8/bin/fm-procevent-telegram.sh arm phone-line
registered: phone-line (telegram)
armed: phone-line
$ /home/nathan/.no-mistakes/worktrees/8d4bd6329c9b/01KZDDMAHY3TXPJ4SSGCEBB6J8/bin/fm-procevent.sh list
SOURCE                       ADAPTER      OWNER      PENDING
phone-line                   telegram     none       0

=== 2. Runner supervises the poll; the operator sends a message from their phone ===
$ /home/nathan/.no-mistakes/worktrees/8d4bd6329c9b/01KZDDMAHY3TXPJ4SSGCEBB6J8/bin/fm-procevent.sh start phone-line
captured: /tmp/fm-tg-e2e.Ugl83t/fmhome/state/procevent-inbox/phone-line.1.result
$ /home/nathan/.no-mistakes/worktrees/8d4bd6329c9b/01KZDDMAHY3TXPJ4SSGCEBB6J8/bin/fm-procevent.sh list
SOURCE                       ADAPTER      OWNER      PENDING
phone-line                   telegram     none       1

=== 3. What the poll actually asked Telegram for (offset = cursor+1, batch limit, long poll) ===
$ cat /tmp/fm-tg-e2e.Ugl83t/tg/fields.log
offset=1
limit=8
timeout=1
offset=1
limit=8
timeout=1
offset=1
limit=8
timeout=1

=== 4. The captured result the agent is woken with ===
$ cat state/procevent-inbox/phone-line.1.result
{"ok":true,"result":[{"update_id":901,"message":{"from":{"first_name":"Nathan"},"chat":{"id":4242},"text":"ship it, merge the PR"}}]}
$ wake queue entry
check: procevent telegram phone-line 1

=== 5. Token safety: it never reaches an argv or any adapter output ===
$ grep -c <token> <curl argv seen by the process listing>
0
$ token present in the stdin config curl actually read?
3
$ token anywhere in the captured result or wake queue?
no matches

=== 6. Handler finishes the work: ack first, then handled (the documented order) ===
$ /home/nathan/.no-mistakes/worktrees/8d4bd6329c9b/01KZDDMAHY3TXPJ4SSGCEBB6J8/bin/fm-procevent-telegram.sh ack 901
acked: 901
$ cat ~/.config/firstmate/telegram-cursor
901
$ /home/nathan/.no-mistakes/worktrees/8d4bd6329c9b/01KZDDMAHY3TXPJ4SSGCEBB6J8/bin/fm-procevent.sh handled phone-line 1
handled: phone-line 1
$ ack is idempotent and never rewinds
$ /home/nathan/.no-mistakes/worktrees/8d4bd6329c9b/01KZDDMAHY3TXPJ4SSGCEBB6J8/bin/fm-procevent-telegram.sh ack 901
already-acked: 901 (cursor 901)
$ /home/nathan/.no-mistakes/worktrees/8d4bd6329c9b/01KZDDMAHY3TXPJ4SSGCEBB6J8/bin/fm-procevent-telegram.sh ack 5
already-acked: 5 (cursor 901)
$ cursor after the rewind attempt: 901

=== 7. Poll restarts: the already-delivered message is not re-woken, the new one is ===
$ /home/nathan/.no-mistakes/worktrees/8d4bd6329c9b/01KZDDMAHY3TXPJ4SSGCEBB6J8/bin/fm-procevent.sh start phone-line
captured: /tmp/fm-tg-e2e.Ugl83t/fmhome/state/procevent-inbox/phone-line.2.result
$ offsets requested this round (cursor advanced to 902 territory):
offset=902
limit=8
timeout=1
offset=902
limit=8
timeout=1
$ results now captured:
state/procevent-inbox/phone-line.1.result
state/procevent-inbox/phone-line.2.result
$ second captured result:
{"ok":true,"result":[{"update_id":901,"message":{"from":{"first_name":"Nathan"},"chat":{"id":4242},"text":"ship it, merge the PR"}},{"update_id":902,"message":{"from":{"first_name":"Nathan"},"chat":{"id":4242},"text":"and tag a release"}}]}

=== 8. A message stream never self-terminates ===
$ fm-procevent-telegram.sh terminal <a-result-with-updates>; echo $?
1
$ /home/nathan/.no-mistakes/worktrees/8d4bd6329c9b/01KZDDMAHY3TXPJ4SSGCEBB6J8/bin/fm-procevent.sh list
SOURCE                       ADAPTER      OWNER      PENDING
phone-line                   telegram     none       0

=== 9. Hard auth failure surfaces to the operator instead of spinning ===
$ fm-procevent-telegram.sh poll; echo $?
error: telegram rejected the bot token (HTTP 401); check /tmp/fm-tg-e2e.Ugl83t/home/.config/firstmate/telegram-token
exit=1
$ same failure under the runner, kept privately for a detached operator:
$ ls -l /tmp/fm-tg-e2e.Ugl83t/fmhome/state/procevent/phone-line.stderr
-rw------- 1 nathan nathan 117 Aug  7 00:13 /tmp/fm-tg-e2e.Ugl83t/fmhome/state/procevent/phone-line.stderr
$ cat /tmp/fm-tg-e2e.Ugl83t/fmhome/state/procevent/phone-line.stderr
error: telegram rejected the bot token (HTTP 401); check /tmp/fm-tg-e2e.Ugl83t/home/.config/firstmate/telegram-token
$ token leaked into that diagnostic? 0

=== 10. A conflict names both of its causes ===
error: telegram refused getUpdates with a conflict (HTTP 409); either a webhook is configured on this bot or another getUpdates poller is already running for it, so check both before re-arming this poll
exit=1

=== 11. Operator retires the line ===
$ /home/nathan/.no-mistakes/worktrees/8d4bd6329c9b/01KZDDMAHY3TXPJ4SSGCEBB6J8/bin/fm-procevent-telegram.sh retire phone-line
retired: phone-line
$ /home/nathan/.no-mistakes/worktrees/8d4bd6329c9b/01KZDDMAHY3TXPJ4SSGCEBB6J8/bin/fm-procevent.sh list
no sources registered
$ delivery marker cleared, cursor kept: telegram-cursor telegram-token 

End of operator journey.
Evidence: Reproducible demo script used to generate the transcript
#!/usr/bin/env bash
# End-to-end operator journey for the Telegram process-event adapter.
#
# Nothing here is a unit test: it drives the REAL generic runner
# (bin/fm-procevent.sh) with the REAL adapter (bin/fm-procevent-telegram.sh)
# against a stand-in Telegram Bot API, and prints what an operator would
# actually see at each step. The only fake is `curl`: a PATH stub that answers
# from a scripted sequence and records what it was asked for, so no network is
# used and no real bot token exists.
set -u

ROOT=${1:?repo root}
WORK=$(mktemp -d "${TMPDIR:-/tmp}/fm-tg-e2e.XXXXXX")
trap 'rm -rf -- "$WORK"' EXIT

export HOME="$WORK/home"
export FM_HOME="$WORK/fmhome"
export FM_PROCEVENT_CLAIM_ROOT="$WORK/claims"
export TG_STATE="$WORK/tg"
export FM_TELEGRAM_POLL_TIMEOUT=1
export FM_TELEGRAM_BACKOFF_SECONDS=1
mkdir -p "$HOME/.config/firstmate" "$FM_HOME/state" "$TG_STATE" "$WORK/bin"

TOKEN='7654321:AAFakeTokenNeverReal_e2e'
printf '%s\n' "$TOKEN" > "$HOME/.config/firstmate/telegram-token"
chmod 0600 "$HOME/.config/firstmate/telegram-token"

ADAPTER="$ROOT/bin/fm-procevent-telegram.sh"
RUNNER="$ROOT/bin/fm-procevent.sh"

cat > "$WORK/bin/curl" <<'STUB'
#!/usr/bin/env bash
set -u
printf '%s\n' "$*" >> "$TG_STATE/argv.log"
out=
while [ "$#" -gt 0 ]; do
  case "$1" in
    -o) out=$2; shift 2 ;;
    --data-urlencode) printf '%s\n' "$2" >> "$TG_STATE/fields.log"; shift 2 ;;
    *) shift ;;
  esac
done
cat >> "$TG_STATE/config.log"
n=0
[ -f "$TG_STATE/calls" ] && read -r n < "$TG_STATE/calls"
n=$((n + 1))
printf '%s\n' "$n" > "$TG_STATE/calls"
step=$(sed -n "${n}p" "$TG_STATE/script")
# Past the end of the script the API keeps answering the same way it last did,
# so a scripted condition (a rejected token) persists instead of evaporating.
[ -n "$step" ] || step=$(tail -n 1 "$TG_STATE/script")
[ -n "$step" ] || step='200 empty'
code=${step%% *}
kind=${step#* }
case "$kind" in
  reply)   body='{"ok":true,"result":[{"update_id":901,"message":{"from":{"first_name":"Nathan"},"chat":{"id":4242},"text":"ship it, merge the PR"}}]}' ;;
  reply2)  body='{"ok":true,"result":[{"update_id":901,"message":{"from":{"first_name":"Nathan"},"chat":{"id":4242},"text":"ship it, merge the PR"}},{"update_id":902,"message":{"from":{"first_name":"Nathan"},"chat":{"id":4242},"text":"and tag a release"}}]}' ;;
  empty)   body='{"ok":true,"result":[]}' ;;
  denied)  body='{"ok":false,"error_code":401,"description":"Unauthorized"}' ;;
  webhook) body='{"ok":false,"error_code":409,"description":"Conflict: terminated by other getUpdates request"}' ;;
  *)       body='{"ok":false,"description":"server trouble"}' ;;
esac
[ -n "$out" ] && printf '%s\n' "$body" > "$out"
printf '%s' "$code"
STUB
chmod +x "$WORK/bin/curl"
export PATH="$WORK/bin:$PATH"

api() { printf '%s\n' "$@" > "$TG_STATE/script"; : > "$TG_STATE/calls"; }
step() { printf '\n\033[1m=== %s ===\033[0m\n' "$*"; }
run() { printf '$ %s\n' "$*"; "$@"; }

printf 'operator config:\n'
run ls -l "$HOME/.config/firstmate"

step "1. Operator arms the phone line"
run "$ADAPTER" arm phone-line
run "$RUNNER" list

step "2. Runner supervises the poll; the operator sends a message from their phone"
api '200 empty' '200 empty' '200 reply'
run "$RUNNER" start phone-line
run "$RUNNER" list

step "3. What the poll actually asked Telegram for (offset = cursor+1, batch limit, long poll)"
run cat "$TG_STATE/fields.log"

step "4. The captured result the agent is woken with"
result=$(ls "$FM_HOME/state/procevent-inbox"/phone-line.*.result | head -n1)
printf '$ cat %s\n' "${result#"$FM_HOME"/}"
cat "$result"
printf '$ wake queue entry\n'
awk -F '\t' '{print $5}' "$FM_HOME/state/.wake-queue"

step "5. Token safety: it never reaches an argv or any adapter output"
printf '$ grep -c %s <curl argv seen by the process listing>\n' "<token>"
grep -c -- "$TOKEN" "$TG_STATE/argv.log" || true
printf '$ token present in the stdin config curl actually read?\n'
grep -c -- "$TOKEN" "$TG_STATE/config.log" || true
printf '$ token anywhere in the captured result or wake queue?\n'
grep -rc -- "$TOKEN" "$FM_HOME/state" 2>/dev/null | grep -v ':0$' || printf 'no matches\n'

step "6. Handler finishes the work: ack first, then handled (the documented order)"
run "$ADAPTER" ack 901
printf '$ cat ~/.config/firstmate/telegram-cursor\n'
cat "$HOME/.config/firstmate/telegram-cursor"
run "$RUNNER" handled phone-line 1
printf '$ ack is idempotent and never rewinds\n'
run "$ADAPTER" ack 901
run "$ADAPTER" ack 5
printf '$ cursor after the rewind attempt: '; cat "$HOME/.config/firstmate/telegram-cursor"

step "7. Poll restarts: the already-delivered message is not re-woken, the new one is"
api '200 reply' '200 reply2'
rm -f "$TG_STATE/fields.log"
run "$RUNNER" start phone-line
printf '$ offsets requested this round (cursor advanced to 902 territory):\n'
cat "$TG_STATE/fields.log"
printf '$ results now captured:\n'
ls -1 "$FM_HOME/state/procevent-inbox"/phone-line.*.result | sed "s#$FM_HOME/##"
printf '$ second captured result:\n'
cat "$(ls "$FM_HOME/state/procevent-inbox"/phone-line.*.result | tail -n1)"
"$ADAPTER" ack 902 >/dev/null
"$RUNNER" handled phone-line 2 >/dev/null

step "8. A message stream never self-terminates"
printf '$ %s terminal <a-result-with-updates>; echo $?\n' "$(basename "$ADAPTER")"
"$ADAPTER" terminal "$result"; printf '%s\n' "$?"
run "$RUNNER" list

step "9. Hard auth failure surfaces to the operator instead of spinning"
api '401 denied'
printf '$ fm-procevent-telegram.sh poll; echo $?\n'
"$ADAPTER" poll; printf 'exit=%s\n' "$?"
printf '$ same failure under the runner, kept privately for a detached operator:\n'
api '401 denied'
"$RUNNER" start phone-line >/dev/null 2>&1
run ls -l "$FM_HOME/state/procevent/phone-line.stderr"
run cat "$FM_HOME/state/procevent/phone-line.stderr"
printf '$ token leaked into that diagnostic? '
grep -c -- "$TOKEN" "$FM_HOME/state/procevent/phone-line.stderr" || true

step "10. A conflict names both of its causes"
api '409 webhook'
"$ADAPTER" poll; printf 'exit=%s\n' "$?"

step "11. Operator retires the line"
run "$ADAPTER" retire phone-line
run "$RUNNER" list
printf '$ delivery marker cleared, cursor kept: '
ls "$HOME/.config/firstmate" | tr '\n' ' '; printf '\n'

printf '\n\033[1mEnd of operator journey.\033[0m\n'

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

⚠️ **Review** - 3 infos
  • ⚠️ bin/fm-procevent-telegram.sh:50 - The documented "duplicate-result window" is not a single restart - it repeats every watcher cycle for the whole handling window. bin/fm-watch.sh:791 calls fm-procevent.sh reconcile each ~15s cycle (POLL=${FM_POLL:-15}), and cmd_reconcile restarts any registered source whose claim is released (bin/fm-procevent.sh:433-437) with no cooldown and no gate on a pending unhandled result. Since terminal always exits 1 and only ack moves the cursor, each restart re-polls the same offset=cursor+1, gets the same batch back, and fm_procevent_capture + publish_pending mint a fresh &lt;id&gt;.&lt;seq&gt;.result and a fresh wake key procevent:&lt;id&gt;:&lt;seq&gt; (bin/fm-procevent.sh:161-178). Concrete sequence: operator sends one message; poll captures update 901 as seq 1 and wakes the agent; the agent spends 5 minutes handling before it can ack; meanwhile ~20 more results and ~20 more wakes for that same update accumulate in state/procevent-inbox and the wake queue, each requiring its own fm-procevent.sh handled call. Worse, those duplicates are minted before the ack, so their update ids are above the cursor - exactly the case SKILL.md:55 says "get fully re-acted on with nothing marking them a repeat" - so the already-seen rule does not cover any wake announced before the ack lands. The sibling adapters do not have this shape: lavish's poll blocks until genuinely new feedback, and remote-reply advances its own cursor at capture. Poll cannot fix this locally without writing the cursor, which the intent forbids; the earliest shared boundary is the runner - have reconcile skip restarting a source that already has an unhandled pending result for it. Flagging rather than fixing because the header treats this window as a deliberate accepted tradeoff.
  • ⚠️ bin/fm-procevent-telegram.sh:163 - The header (lines 26-29) and the intent both justify the non-zero exit on 401/403/409 as making the runner "surface the blocker rather than spinning on it", but under the only path that actually starts an armed source nothing is surfaced. cmd_start runs the child as &#34;${ARGV[@]}&#34; 2&gt;/dev/null (bin/fm-procevent.sh:279), discarding the error: telegram rejected the bot token (HTTP 401)... / HTTP 409 message, and the one remaining signal - printf &#39;no-result: %s (exit %s)&#39; at bin/fm-procevent.sh:316 - goes to a detached runner whose output is thrown away (isolate_runner, bin/fm-procevent.sh:200: &gt;/dev/null 2&gt;&amp;1 &amp;). There is no procevent log. Concrete sequence: operator writes a typo'd token to ~/.config/firstmate/telegram-token and runs arm; every ~15s reconcile spawns a poll that dies instantly with exit 1, the registration stays armed forever, and the operator sees nothing anywhere - the silent spin the design set out to avoid, just at reconcile cadence instead of inside the poll loop. The adapter's own contract (exit non-zero) is met; the gap is in the shared runner, which would need to persist or print a hard-failure signal for a detached child. Raising it because the stated acceptance rationale does not hold end to end.
  • ℹ️ bin/fm-procevent-telegram.sh:152 - --max-time &#34;$((POLL_TIMEOUT + 10))&#34; is the one numeric input in this file that is not base-10 normalized; read_cursor uses $((10#$cursor)) (line 103), cmd_ack uses $((10#$id)) (line 201), and retry_after_delay uses $((10#$value)) (line 126). The validation at line 135 accepts any digit string, including a leading zero. With FM_TELEGRAM_POLL_TIMEOUT=08, bash fails the arithmetic expansion (08: value too great for base), curl never runs, rc is non-zero, and the loop hits sleep &#34;$BACKOFF&#34;; continue forever - an invisible 5s spin that never contacts Telegram and never exits. Use $((10#$POLL_TIMEOUT + 10)) to match the rest of the file.
  • ℹ️ .agents/skills/process-event-sources/SKILL.md:55 - "first call the telegram adapter's ack for the highest update id you fully handled, which confirms those updates at Telegram itself" misstates the mechanism. cmd_ack makes no network call - it only writes ~/.config/firstmate/telegram-cursor under a lock. Confirmation (and server-side deletion) happens later, when the next poll sends offset=cursor+1 (bin/fm-procevent-telegram.sh:140,153). The adapter's own header is accurate ("The poll only READS the cursor"), and the operational conclusion in this paragraph - ack first, then handled - is unaffected, but an agent reading this could reasonably expect ack to be a fallible network step.

🔧 Fix: gate duplicate telegram emissions, keep failed child stderr
5 issues (3 warnings, 2 infos) still open:

  • ⚠️ bin/fm-procevent-telegram.sh:249 - The marker is committed before the batch is handed over, which converts a crash window from "one benign duplicate" into "a silently dropped operator message" - the inverse of the ordering principle the rest of this design is built on (SKILL.md justifies ack-before-handled precisely because its crash window degrades to a no-op). Concrete sequence: poll gets update 901, write_emitted 901 commits, cat streams the JSON into the runner's bounder, and the runner then dies before fm_procevent_capture at bin/fm-procevent.sh:355 - a die &#34;cannot durably capture the result&#34; on a full or read-only state dir, a die &#34;cannot bound source output&#34; at line 338, a process-group stop, a reboot. No result, no wake. The next poll reads cursor 0, marker 901, gets [901] back, computes highest=901 which is not greater than 901, and sleeps forever. The reply sits unseen in the chat - the exact failure the adapter exists to prevent - until an unrelated later message arrives. Moving write_emitted &#34;$highest&#34; to after cat &#34;$POLL_BODY&#34; succeeds keeps every anti-duplicate property (a restarted poll still reads the marker and declines) and reduces the crash window to one duplicate result. The in-code rationale at lines 246-248 argues the opposite, but a write_emitted that dies is a persistently unwritable ~/.config/firstmate: with the marker first that means nothing is ever delivered, with it second it means the pre-fix duplicate behavior, so marker-after-handoff is the weaker failure in both directions.
  • ⚠️ bin/fm-procevent-telegram.sh:312 - Nothing ever clears ~/.config/firstmate/telegram-emitted, so the operator's normal recovery lever no longer works. Concrete sequence: operator arms telegram-captain, update 901 is emitted (marker 901) and captured, the agent starts handling it and dies before ack, so cursor stays 0. The operator does the documented thing - fm-procevent.sh retire telegram-captain (which clears the registration, runner file and stderr file at bin/fm-procevent.sh:612-615) and re-arms. The fresh poll reads cursor 0 and marker 901, Telegram redelivers [901], highest=901 is not above the marker, and it waits indefinitely. Message 901 is stranded with no operator-visible way out: arm does not touch the marker, there is no reset subcommand, and neither the script header nor SKILL.md names the file to delete. Before this commit, retire plus re-arm always re-captured everything above the cursor. Either have cmd_arm reset the marker to the current cursor (arming already implies no poll is running, since a second poller is the HTTP 409 case), or add an explicit reset and name it in the header.
  • ⚠️ bin/fm-procevent.sh:329 - The new plumbing makes cmd_start's completion depend on the child's stderr reaching EOF, which is a behavior change every adapter shares, not an additive one. Before, the child's stderr was /dev/null - an open file no descendant could ever gate on. Now stderr is a pipe read by an inner perl; that perl holds the outer fd3 (inherited, never closed) and, more decisively, the subshell cannot run its exit until the inner pipeline finishes. So any descendant that outlives the child while still holding fd2 keeps the stderr bounder alive, which keeps the subshell alive, which keeps the outer pipe open, which means the output bounder at line 331 never sees EOF and cmd_start hangs forever - where previously it returned as soon as the child's stdout writers closed. I checked the shipped adapters and found no confirmed trigger today: the telegram poll only forks curl and sleep, and lavish-axi's detached server spawns with stdio: &#34;ignore&#34; or a log fd rather than inheriting stderr; remote-reply execs ssh, whose behavior here depends on the operator's own ControlMaster/ControlPersist config. But register &lt;adapter&gt; &lt;id&gt; -- &lt;argv&gt; is a generic boundary, so the hang is reachable by construction. Redirecting the child's stderr straight to ${err:-/dev/null} as a plain file and bounding that file after the child exits keeps the diagnostic and restores the original completion condition exactly.
  • ℹ️ bin/fm-procevent.sh:615 - cmd_retire is the only registration-removal path that also removes &lt;id&gt;.stderr. Two siblings drop the registration and leave it: retire_owned_terminal_source removes only $registration, reachable whenever a child exits non-zero but still wrote output (line 346 lets that through, line 342 keeps the stderr record, then the adapter classifies the result terminal); and reconcile's stop path at lines 477-481 removes the source file plus staging and runner files but not the stderr file. The result is a private 0600 diagnostic sitting in $REG for a source that no longer exists. It is inert - every registry consumer in this file, fm-procevent-lib.sh and fm-teardown.sh globs *.source/*.runner/*.claim - so this is tidiness, not a correctness break. Add the same rm -f -- &#34;$(stderr_file &#34;$id&#34;)&#34; to both paths.
  • ℹ️ docs/configuration.md:525 - FM_TELEGRAM_BACKOFF_SECONDS now paces a third thing the doc does not mention. It still reads "pause after a transient network or server error before re-polling", but bin/fm-procevent-telegram.sh:259 also uses it as the wait between re-polls while an already-emitted batch stays unacknowledged - the common steady state for the whole time the handler is working, since Telegram answers that request immediately rather than long-polling. At the default 5 that is 12 getUpdates calls a minute for the duration of handling; at 0, which normalize_count accepts and which the current wording invites as "retry errors immediately", it is an unbounded request loop against the Bot API with no pause at all. Update the description to cover the already-emitted wait, and consider flooring that particular sleep at 1 second independently of the error backoff.

🔧 Fix: redirect child stderr to file, record telegram handoff after delivery
3 infos still open:

  • ℹ️ bin/fm-procevent.sh:94 - usage() still prints sed -n &#39;2,69p&#39;, but this commit's header edit is net +1 line (the two-line "nothing reads it back / next run replaces it" note became the three-line "nothing waits on it / completion is decided by the child's own output alone / dropping the registration drops it too" note). The header's last comment line is now 70, so bin/fm-procevent.sh --help ends mid-sentence at line 69: "...This runner proves capture before publication and bounded re-announcement until handled, and nothing" - the closing "about the source side of the handoff." is dropped. The existing help assertion at tests/fm-procevent.test.sh:1136 only checks for "Durability boundary" (line 68), so it does not catch this. Bump the range to 2,70p. The telegram adapter's own 2,87p is correct (its last comment line is 87).
  • ℹ️ bin/fm-procevent-telegram.sh:264 - Noting the accepted residual, not asking for a change. The ordering fix is right and the reason is now in the code, but cat &#34;$POLL_BODY&#34; returning 0 means the bytes reached the runner's bounding reader, not that fm_procevent_capture (bin/fm-procevent.sh:365) committed them. A runner death in that gap - die &#34;cannot durably capture the result&#34; on a full or read-only state dir, die &#34;cannot bound source output&#34;, a process-group stop - still leaves the marker set with no captured result, so the batch is not re-offered until a newer update arrives or the operator runs retire and re-arms. The adapter cannot observe the runner's durable capture through the supervision contract, so this is as close to "after durable handoff" as it can get inside the stated bound; the header states it plainly and retire is the documented lever. On record because the operator gets no signal that a reply is stranded.
  • ℹ️ .agents/skills/process-event-sources/SKILL.md:35 - The marker is cleared only by bin/fm-procevent-telegram.sh retire, but bin/fm-procevent.sh retire telegram-captain is a first-class documented command (its own help at lines 47-49 calls it "the supported explicit path") that drops the registration without clearing ~/.config/firstmate/telegram-emitted. An operator who takes that route and re-arms gets a poll that silently strands every update at or below the marker - the exact recovery gap this round closed. The convention is on the author's side: lavish and remote-reply also wrap retire, and SKILL.md:60 already says "Retire any other finished source with the adapter's retire". The gap is that the Telegram arming paragraph at SKILL.md:35-37 names only arm, so nothing at the point of use says which retire to reach for. Naming bin/fm-procevent-telegram.sh retire there, and why it is the one that recovers a stranded reply, closes it mechanically.
✅ **Test** - passed

✅ No issues found.

  • bin/fm-test-run.sh tests/fm-procevent-telegram.test.sh tests/fm-procevent.test.sh - both scripts pass, 0 failures (adapter behavior: offset=cursor+1 with absent cursor as 0, batch limit, poll never writes the cursor, ack atomic/idempotent/non-rewinding under a lock, 401/403 and 409 hard exits, 429 honouring bounded retry_after, terminal always exit 1, arm registering [<self>, poll], staged-file cleanup on process-group SIGTERM, --help contract)
  • Manual end-to-end operator journey driving the real bin/fm-procevent.sh runner with the real bin/fm-procevent-telegram.sh adapter against a stand-in Telegram Bot API (only curl stubbed, no network, no real token): fm-procevent-telegram.sh arm phone-line -> fm-procevent.sh start phone-line -> fm-procevent.sh list -> captured result + wake queue entry
  • Verified the request the poll actually issues: offset=1 on an absent cursor, limit=8, timeout=&lt;FM_TELEGRAM_POLL_TIMEOUT&gt;; and offset=902 after ack 901
  • Verified the documented handler order end-to-end: fm-procevent-telegram.sh ack 901 (cursor file written to 901) then fm-procevent.sh handled phone-line 1; repeat ack 901 and rewinding ack 5 both succeed as no-ops leaving the cursor at 901
  • Verified restart dedupe: a restarted poll does not re-hand the already-delivered batch and only exits 0 once update 902 arrives, producing phone-line.2.result
  • Verified fm-procevent-telegram.sh terminal &lt;result&gt; exits 1 and the source stays registered
  • Verified hard failures: HTTP 401 poll exits 1 with a token-free message, and under the runner the reason is kept privately at state/procevent/phone-line.stderr (mode 0600); HTTP 409 exits 1 naming both a configured webhook and another running poller
  • Verified token containment: 0 occurrences of the token in curl's argv log, in the captured result, in the wake queue, and in the kept stderr diagnostic; it appears only in the stdin config curl reads
  • Verified fm-procevent-telegram.sh retire phone-line drops the registration and clears the delivery marker while keeping the cursor
⚠️ **Document** - 1 info
  • ⚠️ bin/fm-procevent.sh:94 - bin/fm-procevent.sh:94 usage() prints sed -n &#39;2,69p&#39;, but the change extended the header comment to line 70 (set -u is line 71). The last header line is dropped, so bin/fm-procevent.sh --help now ends mid-sentence: "...bounded re-announcement until handled, and nothing" - losing "about the source side of the handoff." That sentence is the runner's own statement of what it does NOT prove, and the script header is the authoritative owner of the runner's mechanics. Before this change the range was 2,63p against a header ending at line 63, so this is a regression introduced here (7 header lines added, range advanced by 6). Fix is 2,69p -> 2,70p. I left it because this phase may only edit documentation files and doc comments, and usage() is an executable line. The sibling bin/fm-procevent-telegram.sh is correct (2,87p, header ends at 87).
  • ℹ️ docs/scripts.md:65 - docs/scripts.md lists only fm-procevent-remote-reply.sh and the newly added fm-procevent-telegram.sh; the generic runner fm-procevent.sh and fm-procevent-lavish.sh are still absent from the inventory. This predates the change and the author explicitly scoped it out as a separate table-wide sweep, so I did not touch it. Recording it only as a follow-up candidate.

🔧 Fix: anchor procevent help extraction to header end marker
1 info still open:

  • ℹ️ bin/fm-procevent.sh:95 - Follow-up candidate, deliberately not done here. About 18 other bin/*.sh scripts still bound their --help extraction with an absolute line count (sed -n &#39;2,NNp&#39;), the exact pattern that silently truncated this runner's header. I swept them and none is currently truncating (the line after each range is not a comment), so there is no live defect to fix and nothing this change made stale - but each carries the same latent failure the next time its header grows. The repo already has a marker-bounded idiom in three scripts (sed -n &#39;2,/^set -u$/p&#39; | sed &#39;$d; ...&#39; in bin/fm-public-followup.sh, bin/fm-public-followup-emit.sh, bin/fm-session-start.sh), so a consolidation sweep onto one marker convention has a precedent to standardize on. I used an explicit sentinel comment here rather than that /^set -u$/ idiom because the approved instruction specified a sentinel; a future sweep should pick one of the two and apply it uniformly. Out of scope for this change.
✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

notno added 11 commits August 6, 2026 17:06
* docs(agents): trim section 2 layout tree to a thin pointer surface

Replace the exhaustive annotated layout tree with a compact top-level
summary plus explicit owner pointers (docs/configuration.md for layout
and config schemas, producing script headers for artifact fields).
Normative tree annotations without another owner survive as section 2
prose: the state/ dot-file never-edit rule, the procevent
supervision-required presence rule, the symlink edit-the-original rule,
and the read-script-headers-first rule.
AGENTS.md drops from 63,377 to 53,563 chars.

* no-mistakes: apply CI fixes
An operator reply from a phone now wakes firstmate through the existing
process-event runner instead of sitting unseen in a chat.

bin/fm-procevent-telegram.sh long-polls getUpdates from an offset one past
the recorded cursor and returns the first non-empty batch as a captured
result. The poll only reads the cursor: the handler advances it after fully
handling the messages, so a crash never advances past unhandled replies. A
restart before that advance re-captures the same updates, so the header
states plainly that handlers must treat any update id at or below the cursor
as already seen.

The bot token is read from a private file and passed to curl through a stdin
config, so it never reaches the argv, stdout, or stderr. Transient transport
and server errors retry with bounded backoff; HTTP 401 and 403 exit non-zero
so the runner surfaces a rejected token rather than spinning. A message
stream never self-terminates, so terminal always keeps the source armed and
retirement stays an explicit operator action.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant