Skip to content

[scanner] repeat-failure: copilot route emits the shared command record + re-runnable harness #14746

Closed

Description

@hivecommons-hive

Part of #14559 (per-backend split of the backend-specific routes). Epic: #14550. Target: v6 (open the PR with --base v6).

Scope (C1, C3, C14): copilot runs one long-lived Bash session per shell id, so the neutral BASH_ENV route never sees its commands. Add a producer (route e.g. copilot_events) that reads the Copilot session log ~/.copilot/session-state/<id>/events.jsonl (per-agent COPILOT_HOME, provisioned by setupCopilotHome in src/crux/agent/interactive_home.go) and emits one shared command record per shell tool.execution_complete event: command from the matching tool.execution_start / toolRequests arguments, result from data.success (Failed = !success; set ExitCode only if the event carries a numeric exit code — check). Reuse the event struct and parsing in renderCopilotTranscript (src/crux/agent/kick_transcript_copilot.go) and the per-session discovery in newestCopilotTranscriptActivity (src/crux/agent/busy_visibility.go); emit exactly one record per tool call across re-reads (track offset / toolCallId). Shell-activity signal for C12: shell tool.execution_start events.

Only write into copilot's settings files where Hive already writes them (the advisor hooks dir under the per-agent .copilot); with repeat_failure.enabled off Hive adds nothing and removes what it added at the next launch (C14 / C-AC12).

Shared pieces (v6 @ 6cec2b9): record type RepeatFailureRecord (Route, ExitCode *int, Failed) and writer AppendRepeatFailureRecord / RepeatFailureRecordPath(home) in src/crux/agent/repeat_failure_record.go; switch via Manager.repeatFailureEnabled() and route-constant pattern RepeatFailureRouteBashEnv = "bash_env" in src/crux/agent/repeat_failure_bash_env.go. Coverage table repeatFailureCoverageTable in src/crux/agent/repeat_failure_coverage.go; tests TestRepeatFailureCoverageRowsAreHonest / TestRepeatFailureCoverageDocsMatchCode; docs table "Repeated-failure check" in src/docs/backend-support-tiers.md (~line 550, rendered from RepeatFailureCoverageMarkdownRows()).

Harness (C4): a committed, re-runnable harness (stub model provider that makes the real copilot CLI run false or ls /nonexistent once, or a recorded real-run events.jsonl fixture) with the copilot --version output captured, proving one failing record. Update the copilot row in repeatFailureCoverageTable (Covered, Route, Evidence incl. --version, EvidencePath, ShellActivity) and the matching docs row.

If it cannot be covered (C5): leave the row "not covered", put what was tried in its Reason, and keep the docs row in sync.

Acceptance: C-AC1 for copilot when covered, else C-AC9; the C-AC11 coverage test stays green.

Decision taken by hive: split #14559 per backend so each lands as one reviewable PR; no ordering dependency between backends.

— hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88

Activity

  1. added
    good first issueDenotes an issue ready for a new contributor, according to the "help wanted" guidelines.
    help wantedDenotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.
    agent/scannerCreated or modified by the scanner agent
    on Oct 28, 2026
  2. hivecommons-hive commented on Oct 28, 2026

    @hivecommons-hive
    ContributorAuthor

    Folded finding. This was filed as a separate issue, but 2 open agent-filed issues (plus 1 closed as completed in the last 14 days) already cite src/docs/backend-support-tiers.md, so it was folded into the oldest of them instead of opening another. Treat this issue as the tracker for that component: prefer one structural fix that covers every folded variant over a PR per variant.

    [scanner] repeat-failure: openhands route emits the shared command record + re-runnable harness

    Part of #14559 (per-backend split of the backend-specific routes). Epic: #14550. Target: v6 (open the PR with --base v6).

    Scope (C1, C3, C14): openhands keeps one interactive /bin/bash open in tmux, so the neutral BASH_ENV route never sees its commands. Planned routes: a PostToolUse hook (~/.openhands/hooks.json; check the hook input fields) or a reader of its saved events (exit_code), emitting one shared command record per shell command (route e.g. openhands_hook / openhands_events).

    Hook wiring location TBD — not found on v6 at 2026-10-27: openhands has no Hive hook wiring or launch case on v6 (it falls to the default branch of backendLaunchCmdWithBobSession in src/crux/agent/manager_launch.go; only config.CLIBackends in src/crux/config/agent_runtime.go and the coverage row name it), and Hive does not write ~/.openhands/hooks.json today. So prefer a Hive-side reader of the saved events, or a launch-env / Hive-owned-file route; writing hooks.json is only allowed if Hive already writes that file for openhands. With repeat_failure.enabled off Hive adds nothing and removes what it added at the next launch (C14 / C-AC12).

    Shared pieces (v6 @ 6cec2b9): record type RepeatFailureRecord (Route, ExitCode *int, Failed) and writer AppendRepeatFailureRecord / RepeatFailureRecordPath(home) in src/crux/agent/repeat_failure_record.go; switch via Manager.repeatFailureEnabled() in src/crux/agent/repeat_failure_bash_env.go. Coverage table repeatFailureCoverageTable in src/crux/agent/repeat_failure_coverage.go; tests TestRepeatFailureCoverageRowsAreHonest / TestRepeatFailureCoverageDocsMatchCode; docs table "Repeated-failure check" in src/docs/backend-support-tiers.md (~line 550).

    Harness (C4): a committed, re-runnable harness (stub model provider driving the real openhands CLI, or a recorded real-run fixture) that records the openhands --version the harness installs (openhands is not in the image), showing one failing command recorded. Update the openhands row in repeatFailureCoverageTable (Covered, Route, Evidence incl. --version, EvidencePath, ShellActivity) and the matching docs row.

    If it cannot be covered (C5): leave the row "not covered", put what was tried in its Reason, and keep the docs row in sync.

    Acceptance: C-AC1 for openhands when covered, else C-AC9; the C-AC11 coverage test stays green.

    Decision taken by hive: split #14559 per backend so each lands as one reviewable PR; no ordering dependency between backends.

    — hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88

  3. hivecommons-hive commented on Oct 28, 2026

    @hivecommons-hive
    ContributorAuthor

    The folded openhands finding above was filed separately as #14748. This issue covers copilot only.

    — hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88

  4. added
    stage/010-actionableTriaged; eligible for the hive to pick up
    stage/020-claimedClaimed by the scanner or a relay contributor
    and removed
    stage/010-actionableTriaged; eligible for the hive to pick up
    good first issueDenotes an issue ready for a new contributor, according to the "help wanted" guidelines.
    on Oct 28, 2026
  5. hivecommons-hive commented on Oct 28, 2026

    @hivecommons-hive
    ContributorAuthor

    🔒 Claimed by agent scanner (agent on hive hosted-available-oke-11-placeholder-r05x). The claim lasts 2h0m0s without activity and renews while the work continues. Reply /unclaim to release, or /claim to take it over if you outrank the holder.

  6. hivecommons-hive commented on Oct 28, 2026

    @hivecommons-hive
    ContributorAuthor

    🔓 Claim by agent scanner released (holding agent relaunched).

  7. 2 remaining items

  8. hivecommons-hive commented on Oct 28, 2026

    @hivecommons-hive
    ContributorAuthor

    🔓 Claim by @thatguywob released (lease revoked).

  9. hivecommons-hive commented on Oct 28, 2026

    @hivecommons-hive
    ContributorAuthor

    🔒 Claimed by agent scanner (agent on hive hosted-available-oke-11-placeholder-r05x). The claim lasts 2h0m0s without activity and renews while the work continues. Reply /unclaim to release, or /claim to take it over if you outrank the holder.

  10. github-actions commented on Oct 28, 2026

    @github-actions
    Contributor

    Closed by #14801 (merged to v6).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent/scannerCreated or modified by the scanner agenthelp wantedDenotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.hive/claimed-by-scanner

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions