fix(clawsweeper-status): support public queue projection

CHANGELOG.md

@@ -6,6 +6,7 @@ summary: Timeline of guardrail helper changes mirrored from Sweetistics and rela
 
 ## Unreleased
 
+- Fixed `clawsweeper-status` public queue parsing, preserved optional health fields without shifted columns, and added a separate publication-tail summary.
 - Added headless Sparkle signing through scoped 1Password references, with public-key validation, mode-0600 temporary files, and cleanup on success or failure.
 - Corrected GitHub secret provisioning to omit `--body` for stdin and added a skill validation guard against the literal-dash trap.
 - Taught the `clawsweeper-status` snapshot to report queue handoff health, the ready/admissible split, backoff and parked reasons, and shed-since-reset, so exact-review items parked on retry exhaustion are no longer invisible behind a `healthy` verdict.

skills/clawsweeper-status/SKILL.md

@@ -30,12 +30,13 @@ Useful options:
 
 Report these sections concisely:
 
-- `Workers`: workflow state, Codex jobs against configured capacity, exact-review queue and target occupancy, and active workflow groups.
+- `Workers`: workflow state, Codex jobs against configured capacity, exact-review queue and target occupancy when available, and active workflow groups. Public queue aggregates remain usable without private target occupancy; `occupancy unavailable` does not mean zero.
 - `Queue health`: lead with this whenever it is not `healthy`. Pending depth on its own is not a verdict — read the split the script prints underneath it:
   - `ready`/`admissible` near zero while pending is deep means the lane is deliberately holding items back, not stalled.
   - `Queue backoff: throttle_retry N` means GitHub is rate-limiting; the lane recovers on its own once quota returns.
   - `Queue parked (needs operator): review_retry_exhausted N` does **not** self-heal. Parked items retry at 5/10/20 minutes and then wait for a human. Always call these out explicitly.
   - `Shed since reset` climbing into the thousands means sustained overload, not a blip.
+- `Publication tail`: aggregate pending/ready/backoff/parked/active counts and capacity when available, plus retry and parked reasons. This is separate from the review backlog, not per-target activity; legacy responses without this lane report `unavailable`.
 - `Recently merged`: merged PR URLs plus one-line titles.
 - `Recently reviewed`: ClawSweeper/Codex review comment URLs plus one-line comment summary.
 - `Recently commented`: other recent ClawSweeper comment URLs plus one-line comment summary.
@@ -61,4 +62,5 @@ Do not browse the web for these checks. Use `gh` directly.
 - Count Codex usage from actual in-progress/queued jobs; use setup-action steps plus known lane names to identify Codex work.
 - Treat `pending` workflow runs as concurrency waiters, not queued Codex jobs.
 - Treat stale worker counts cautiously; compare the status-filtered `gh run list` results with the default recent-run list when numbers disagree.
+- Queue reads stay unauthenticated. Public oldest-pending age is useful without a private item key; missing health counts remain `unknown`, not zero.
 - Use full GitHub URLs in the final answer.

skills/clawsweeper-status/scripts/clawsweeper-status.sh

@@ -230,8 +230,7 @@ if command -v curl >/dev/null 2>&1 && \
     type == "object" and
     (.pending | type == "number") and
     (.dispatching | type == "number") and
-    (.leased | type == "number") and
-    (.target_stats | type == "array")
+    (.leased | type == "number")
   ' "$exact_queue_json" >/dev/null; then
   exact_queue_available=true
 else
@@ -381,53 +380,58 @@ if [ "$exact_queue_available" = true ]; then
     exact_active_display="${exact_active}/${exact_capacity}"
   fi
 
-  target_exact_active="$(jq -r --arg target "$target_repo" '[.target_stats[]?
-    | select(.target_repo == $target)
-    | ((.dispatching // 0) + (.leased // 0))][0] // 0' "$exact_queue_json")"
-  target_exact_pending="$(jq -r --arg target "$target_repo" '[.target_stats[]?
-    | select(.target_repo == $target)
-    | (.pending // 0)][0] // 0' "$exact_queue_json")"
-  target_exact_active_display="$target_exact_active"
-  if [ -n "$exact_target_capacity" ]; then
-    target_exact_active_display="${target_exact_active}/${exact_target_capacity}"
-  fi
-  printf -- "- Exact-review queue: %s active, %s pending (target %s: %s active, %s pending)\n" \
-    "$exact_active_display" "$exact_pending" "$target_repo" "$target_exact_active_display" "$target_exact_pending"
-
-  # Depth alone hides the failure modes that actually stall the lane: items can be
-  # ready, in throttle backoff, or parked after retry exhaustion, and only the last
-  # needs an operator. Surface that split plus handoff health.
-  queue_health="$(jq -r '
-    def reasons: to_entries | map("\(.key) \(.value)") | join(", ");
-    [
-      (.handoff_health.status // "unknown"),
-      (.handoff_health.reason // ""),
-      (.ready_pending // 0 | tostring),
-      (.admissible_pending // 0 | tostring),
-      ((.oldest_pending_key // "") | tostring),
-      ((.oldest_pending_age_seconds // 0) / 60 | floor | tostring),
-      ((.lanes.review.backoff_reasons // {}) | reasons),
-      ((.lanes.review.parked_reasons // {}) | reasons),
-      (.shed_since_reset // 0 | tostring)
-    ] | @tsv' "$exact_queue_json" 2>/dev/null)"
-  if [ -n "$queue_health" ]; then
-    IFS=$'\t' read -r qh_status qh_reason qh_ready qh_admissible qh_oldest_key qh_oldest_min \
-      qh_backoff qh_parked qh_shed <<<"$queue_health"
-    health_line="- Queue health: ${qh_status}"
-    [ -n "$qh_reason" ] && health_line="${health_line} (${qh_reason})"
-    health_line="${health_line} — ready ${qh_ready}, admissible ${qh_admissible}"
-    if [ -n "$qh_oldest_key" ]; then
-      health_line="${health_line}, oldest pending ${qh_oldest_key} ${qh_oldest_min}m"
-    fi
-    printf -- "%s\n" "$health_line"
-    [ -n "$qh_backoff" ] && printf -- "- Queue backoff: %s\n" "$qh_backoff"
-    if [ -n "$qh_parked" ]; then
-      printf -- "- Queue parked (needs operator): %s\n" "$qh_parked"
-    fi
-    if [ "${qh_shed:-0}" != "0" ]; then
-      printf -- "- Shed since reset: %s\n" "$qh_shed"
+  # Public aggregates omit private target occupancy; only an array can establish zero.
+  if jq -e '.target_stats | type == "array"' "$exact_queue_json" >/dev/null; then
+    target_exact_active="$(jq -r --arg target "$target_repo" '[.target_stats[]
+      | select(.target_repo == $target)
+      | ((.dispatching // 0) + (.leased // 0))][0] // 0' "$exact_queue_json")"
+    target_exact_pending="$(jq -r --arg target "$target_repo" '[.target_stats[]
+      | select(.target_repo == $target)
+      | (.pending // 0)][0] // 0' "$exact_queue_json")"
+    target_exact_active_display="$target_exact_active"
+    if [ -n "$exact_target_capacity" ]; then
+      target_exact_active_display="${target_exact_active}/${exact_target_capacity}"
     fi
+    target_exact_display="${target_exact_active_display} active, ${target_exact_pending} pending"
+  else
+    target_exact_display="occupancy unavailable"
   fi
+  printf -- "- Exact-review queue: %s active, %s pending (target %s: %s)\n" \
+    "$exact_active_display" "$exact_pending" "$target_repo" "$target_exact_display"
+
+  # Render directly: Bash read collapses empty TSV fields and shifts optional health data.
+  jq -r '
+    def count: if type == "number" then tostring else "unknown" end;
+    def text: if type == "string" then . else "" end;
+    def reason_line($label):
+      if type == "object" then
+        to_entries | map(select(.value | type == "number" and . > 0))
+        | map("\(.key) \(.value)") | join(", ")
+        | select(length > 0) | "- \($label): \(.)"
+      else empty end;
+
+    ("- Queue health: \((.handoff_health.status | text | select(length > 0)) // "unknown")" +
+      (.handoff_health.reason | text | if length > 0 then " (\(.))" else "" end) +
+      " — ready \(.ready_pending | count), admissible \(.admissible_pending | count)" +
+      (if (.oldest_pending_age_seconds | type) == "number" then
+        ", oldest pending " +
+        (.oldest_pending_key | text | if length > 0 then . + " " else "" end) +
+        "\(.oldest_pending_age_seconds / 60 | floor)m"
+      elif (.oldest_pending_key | text | length) > 0 then
+        ", oldest pending \(.oldest_pending_key) (age unknown)"
+      else "" end)),
+    (.lanes.review.backoff_reasons | reason_line("Queue backoff")),
+    (.lanes.review.parked_reasons | reason_line("Queue parked (needs operator)")),
+    (.shed_since_reset | select(type == "number" and . != 0) | "- Shed since reset: \(.)"),
+    (if (.lanes.publication | type) == "object" then
+      .lanes.publication |
+      ("- Publication tail: \(.pending | count) pending, \(.ready | count) ready, " +
+        "\(.backoff | count) backoff, \(.parked | count) parked, \(.active | count)" +
+        (if (.capacity | type) == "number" then "/\(.capacity)" else "" end) + " active"),
+      (.backoff_reasons | reason_line("Publication backoff")),
+      (.parked_reasons | reason_line("Publication parked (needs operator)"))
+    else "- Publication tail: unavailable" end)
+  ' "$exact_queue_json"
 else
   printf -- "- Exact-review queue: unavailable\n"
 fi

skills/clawsweeper-status/scripts/clawsweeper-status.test.sh

@@ -82,22 +82,90 @@ chmod +x "$tmpdir/gh"
 cat >"$tmpdir/curl" <<'EOF'
 #!/usr/bin/env bash
 set -euo pipefail
+printf '%s\n' "$*" >>"${CURL_TEST_LOG:?}"
 if [ "${CURL_TEST_MODE:-}" = "fail" ]; then
   exit 22
-elif [ "${CURL_TEST_MODE:-}" = "absent" ]; then
-  printf '%s\n' '{"pending":2,"dispatching":3,"leased":5,"target_stats":[{"target_repo":"test/other","pending":1,"dispatching":2,"leased":4}]}'
-  exit 0
 fi
-printf '%s\n' '{"pending":2,"dispatching":3,"leased":5,"target_stats":[{"target_repo":"test/target","pending":1,"dispatching":2,"leased":4}]}'
+cat "${CURL_TEST_FIXTURE:?}"
 EOF
 chmod +x "$tmpdir/curl"
 
 export GH_TEST_LOG="$tmpdir/gh.log"
-PATH="$tmpdir:$PATH" "$script_dir/clawsweeper-status.sh" \
-  --repo test/target \
-  --clawsweeper-repo test/sweeper \
-  --limit 8 \
-  --run-limit 12 >"$tmpdir/output"
+export CURL_TEST_LOG="$tmpdir/curl.log"
+public_fixture="$script_dir/fixtures/public-exact-review-queue.json"
+printf '%s\n' '{"pending":2,"dispatching":3,"leased":5,"target_stats":[{"target_repo":"test/target","pending":1,"dispatching":2,"leased":4}]}' >"$tmpdir/legacy.json"
+jq '. + {target_stats: [{target_repo: "test/target", pending: 1, dispatching: 2, leased: 4}], oldest_pending_key: "test/target#42"}' \
+  "$public_fixture" >"$tmpdir/full.json"
+
+assert_contains() {
+  if ! grep -Fq -- "$1" "$2"; then
+    printf 'FAIL [%s]: expected %s in %s\n' "$test_case" "$1" "$2" >&2
+    cat "$2" >&2
+    exit 1
+  fi
+}
+
+assert_not_contains() {
+  if grep -Fq -- "$1" "$2"; then
+    printf 'FAIL [%s]: unexpected %s in %s\n' "$test_case" "$1" "$2" >&2
+    cat "$2" >&2
+    exit 1
+  fi
+}
+
+run_case() {
+  test_case="$1"
+  : >"$GH_TEST_LOG"
+  : >"$CURL_TEST_LOG"
+  CURL_TEST_FIXTURE="$2" CURL_TEST_MODE="${3:-}" PATH="$tmpdir:$PATH" \
+    bash "$script_dir/clawsweeper-status.sh" \
+    --repo test/target \
+    --clawsweeper-repo test/sweeper \
+    --limit 8 \
+    --run-limit 12 >"$tmpdir/output" 2>"$tmpdir/error" || {
+      printf 'FAIL [%s]: status script failed\n' "$test_case" >&2
+      cat "$tmpdir/error" >&2
+      exit 1
+    }
+  # Exact arguments also reject authentication headers, credentials, or extra requests.
+  expected_request="--fail --silent --show-error --connect-timeout 3 --max-time 8 ${CLAWSWEEPER_EXACT_REVIEW_QUEUE_URL:-https://clawsweeper.openclaw.ai}/api/exact-review-queue"
+  if [ "$(cat "$CURL_TEST_LOG")" != "$expected_request" ]; then
+    printf 'FAIL [%s]: queue request must remain unauthenticated and bounded\n' "$test_case" >&2
+    exit 1
+  fi
+  assert_not_contains 'run view 23' "$GH_TEST_LOG"
+  if grep -Eq 'actions/runs($| )|per_page=100|pulls\?state=closed' "$GH_TEST_LOG"; then
+    printf 'FAIL [%s]: broad GitHub payload query detected\n' "$test_case" >&2
+    exit 1
+  fi
+}
+
+assert_queue_jobs() {
+  assert_contains '- Active Codex jobs: 8/128 running, 2 queued' "$tmpdir/output"
+  assert_not_contains 'run view 25' "$GH_TEST_LOG"
+  assert_contains 'run view 26' "$GH_TEST_LOG"
+}
+
+assert_unavailable() {
+  assert_contains '- Exact-review queue: unavailable' "$tmpdir/output"
+  assert_contains '- Active Codex jobs: 4/128 running, 2 queued' "$tmpdir/output"
+  assert_contains 'run view 25' "$GH_TEST_LOG"
+  assert_contains 'run view 26' "$GH_TEST_LOG"
+  assert_not_contains '- Queue health:' "$tmpdir/output"
+  assert_not_contains '- Publication tail:' "$tmpdir/output"
+}
+
+assert_health() {
+  assert_contains '- Queue backoff: throttle_retry 5, review_retry 2' "$tmpdir/output"
+  assert_contains '- Queue parked (needs operator): review_retry_exhausted 3' "$tmpdir/output"
+  assert_contains '- Shed since reset: 17' "$tmpdir/output"
+  assert_contains '- Publication tail: 11 pending, 7 ready, 4 backoff, 2 parked, 1/6 active' "$tmpdir/output"
+  assert_contains '- Publication backoff: publication_retry 4' "$tmpdir/output"
+  assert_contains '- Publication parked (needs operator): dispatch_rejected 2' "$tmpdir/output"
+  assert_not_contains 'throttle_retry 0' "$tmpdir/output"
+}
+
+run_case legacy "$tmpdir/legacy.json"
 
 grep -Fq -- '- Active workflow runs: 6' "$tmpdir/output"
 grep -Fq -- '- Queued/waiting workflow runs: 2' "$tmpdir/output"
@@ -122,27 +190,112 @@ if grep -Fq 'run view 25' "$GH_TEST_LOG"; then
 fi
 grep -Fq 'run view 26' "$GH_TEST_LOG"
 
-CURL_TEST_MODE=absent PATH="$tmpdir:$PATH" "$script_dir/clawsweeper-status.sh" \
-  --repo test/target \
-  --clawsweeper-repo test/sweeper \
-  --limit 8 \
-  --run-limit 12 >"$tmpdir/output-absent"
-grep -Fq -- '- Exact-review queue: 8/28 active, 2 pending (target test/target: 0/24 active, 0 pending)' "$tmpdir/output-absent"
-
-CURL_TEST_MODE=fail PATH="$tmpdir:$PATH" "$script_dir/clawsweeper-status.sh" \
-  --repo test/target \
-  --clawsweeper-repo test/sweeper \
-  --limit 8 \
-  --run-limit 12 >"$tmpdir/output-failed"
-grep -Fq -- '- Exact-review queue: unavailable' "$tmpdir/output-failed"
-
-if grep -Fq 'run view 23' "$GH_TEST_LOG"; then
-  echo "workflow concurrency waiter was probed as a job-bearing run" >&2
-  exit 1
-fi
-if grep -Eq 'actions/runs($| )|per_page=100|pulls\?state=closed' "$GH_TEST_LOG"; then
-  echo "broad GitHub payload query detected" >&2
-  exit 1
-fi
+run_case public "$public_fixture"
+assert_contains '- Exact-review queue: 8/28 active, 9 pending (target test/target: occupancy unavailable)' "$tmpdir/output"
+assert_queue_jobs
+assert_contains '- Queue health: warning (handoff_delayed) — ready 2, admissible 1, oldest pending 12m' "$tmpdir/output"
+assert_health
+
+run_case full "$tmpdir/full.json"
+assert_contains '- Exact-review queue: 8/28 active, 9 pending (target test/target: 6/24 active, 1 pending)' "$tmpdir/output"
+assert_queue_jobs
+assert_contains '- Queue health: warning (handoff_delayed) — ready 2, admissible 1, oldest pending test/target#42 12m' "$tmpdir/output"
+assert_health
+
+run_case legacy-health "$tmpdir/legacy.json"
+assert_queue_jobs
+assert_contains '- Queue health: unknown — ready unknown, admissible unknown' "$tmpdir/output"
+assert_contains '- Publication tail: unavailable' "$tmpdir/output"
+assert_not_contains 'oldest pending' "$tmpdir/output"
+assert_not_contains '- Queue backoff:' "$tmpdir/output"
+assert_not_contains '- Queue parked' "$tmpdir/output"
+assert_not_contains '- Shed since reset:' "$tmpdir/output"
+
+for target_stats in null '{}' '"private"'; do
+  jq --argjson rows "$target_stats" '.target_stats = $rows' "$public_fixture" >"$tmpdir/queue.json"
+  run_case "target-stats-$target_stats" "$tmpdir/queue.json"
+  assert_contains '(target test/target: occupancy unavailable)' "$tmpdir/output"
+  assert_queue_jobs
+  assert_health
+done
+
+for target_stats in '[]' '[{"target_repo":"test/other","pending":1,"dispatching":2,"leased":4}]'; do
+  jq --argjson rows "$target_stats" '.target_stats = $rows' "$tmpdir/legacy.json" >"$tmpdir/queue.json"
+  run_case target-absent-from-array "$tmpdir/queue.json"
+  assert_contains '- Exact-review queue: 8/28 active, 2 pending (target test/target: 0/24 active, 0 pending)' "$tmpdir/output"
+  assert_queue_jobs
+done
+
+# Empty whitespace-separated columns must never shift reasons, ages, or counts.
+for optional_text in '""' null; do
+  jq --argjson text "$optional_text" '
+    .handoff_health.reason = $text | .oldest_pending_key = $text
+  ' "$tmpdir/full.json" >"$tmpdir/queue.json"
+  run_case "empty-text-$optional_text" "$tmpdir/queue.json"
+  assert_contains '- Queue health: warning — ready 2, admissible 1, oldest pending 12m' "$tmpdir/output"
+  assert_health
+done
+
+for empty_reasons in backoff_reasons parked_reasons; do
+  jq --arg field "$empty_reasons" '
+    .handoff_health.reason = "" | .oldest_pending_key = "" |
+    .lanes.review[$field] = {} | .lanes.publication[$field] = {}
+  ' "$tmpdir/full.json" >"$tmpdir/queue.json"
+  run_case "empty-$empty_reasons" "$tmpdir/queue.json"
+  assert_contains '- Queue health: warning — ready 2, admissible 1, oldest pending 12m' "$tmpdir/output"
+  assert_contains '- Shed since reset: 17' "$tmpdir/output"
+  if [ "$empty_reasons" = backoff_reasons ]; then
+    assert_not_contains '- Queue backoff:' "$tmpdir/output"
+    assert_not_contains '- Publication backoff:' "$tmpdir/output"
+    assert_contains '- Queue parked (needs operator): review_retry_exhausted 3' "$tmpdir/output"
+    assert_contains '- Publication parked (needs operator): dispatch_rejected 2' "$tmpdir/output"
+  else
+    assert_not_contains '- Queue parked' "$tmpdir/output"
+    assert_not_contains '- Publication parked' "$tmpdir/output"
+    assert_contains '- Queue backoff: throttle_retry 5, review_retry 2' "$tmpdir/output"
+    assert_contains '- Publication backoff: publication_retry 4' "$tmpdir/output"
+  fi
+done
+
+jq '
+  .handoff_health = {status: "healthy", reason: ""} | .oldest_pending_key = "" |
+  .ready_pending = 0 | .admissible_pending = 0 | .oldest_pending_age_seconds = null |
+  .shed_since_reset = 0 |
+  .lanes.review.backoff_reasons = {} | .lanes.review.parked_reasons = {} |
+  .lanes.publication = {pending: 0, ready: 0, backoff: 0, parked: 0, active: 0}
+' "$tmpdir/full.json" >"$tmpdir/queue.json"
+run_case empty-optionals "$tmpdir/queue.json"
+assert_contains '- Queue health: healthy — ready 0, admissible 0' "$tmpdir/output"
+assert_contains '- Publication tail: 0 pending, 0 ready, 0 backoff, 0 parked, 0 active' "$tmpdir/output"
+assert_not_contains 'oldest pending' "$tmpdir/output"
+assert_not_contains '- Queue backoff:' "$tmpdir/output"
+assert_not_contains '- Queue parked' "$tmpdir/output"
+assert_not_contains '- Publication backoff:' "$tmpdir/output"
+assert_not_contains '- Publication parked' "$tmpdir/output"
+assert_not_contains '- Shed since reset:' "$tmpdir/output"
+
+jq '.lanes.publication = {}' "$tmpdir/legacy.json" >"$tmpdir/queue.json"
+run_case publication-unknown "$tmpdir/queue.json"
+assert_contains '- Publication tail: unknown pending, unknown ready, unknown backoff, unknown parked, unknown active' "$tmpdir/output"
+
+run_case unavailable-fetch "$public_fixture" fail
+assert_unavailable
+printf '%s\n' '{malformed' >"$tmpdir/queue.json"
+run_case malformed-json "$tmpdir/queue.json"
+assert_unavailable
+for field in pending dispatching leased; do
+  for invalid in missing null '"5"'; do
+    if [ "$invalid" = missing ]; then
+      jq --arg field "$field" 'del(.[$field])' "$tmpdir/full.json" >"$tmpdir/queue.json"
+    else
+      jq --arg field "$field" --argjson value "$invalid" '.[$field] = $value' "$tmpdir/full.json" >"$tmpdir/queue.json"
+    fi
+    run_case "invalid-$field-$invalid" "$tmpdir/queue.json"
+    assert_unavailable
+  done
+done
+printf '%s\n' '[]' >"$tmpdir/queue.json"
+run_case invalid-top-level "$tmpdir/queue.json"
+assert_unavailable
 
 echo "clawsweeper-status tests passed"