Skip to content

[scanner] Shadow-mode 'would notify' instrumentation and write-site classification for notification-capable writes #16264

Closed

Description

@hivecommons-hive

Problem

The #16075 decision requires piloting "in shadow mode and one opted-in hive before general rollout" and measuring "repeat/non-actionable writes per active item against matched baseline windows; the 80% reduction is an acceptance target, not a claimed result". Today the only signal is the process-wide created/edited/suppressed tally in src/pkg/github/notification_tally.go (an hourly INFO log line); nothing classifies why a notification-capable write happened, who produced it, or whether it was a repeat of the same condition. Without that baseline no later slice can be evaluated.

Scope (instrumentation only — no behaviour change)

  • Inventory every notification-capable write site (new issue/PR comment, review, review request, @mention) in runtime producers — CreateIssueComment has ~48 non-test call sites under src/pkg and src/cmd — plus Hive-maintained Actions workflows that comment, and classify each as one of: status, attention, required_review, protocol_command (e.g. /ok-to-test, machine-consumed), security_event. Commit the inventory as a doc table (e.g. a section in src/docs/notifications.md or the operator-reference notification-mode page).
  • At the write boundary, record a would-notify decision per write: {repo, item, producer, cause/class, provisional condition key, would_notify bool, reason}. Counters only plus a bounded local log; do not collect comment bodies, recipient lists beyond the login being mentioned/requested, or any inbox contents.
  • Expose counters (extend the tally and the existing opt-in Prometheus /metrics): new comments, reviews, review requests, mentions, suppressed repeats, would-suppress repeats, delivery failures — labelled by producer and class — plus active-item counts so per-active-item rates can be computed.
  • Gate behind a config flag, e.g. github.notifications.shadow: true (default off); nothing is suppressed or rerouted by this child.

Decision taken by hive: provisional condition key in shadow mode = (repo, item, class, writer-supplied subject); the authoritative producer-independent key is defined in the episode child and replaces it. Counts are API writes (proxies), never "emails delivered".

Acceptance criteria

  • Inventory doc lists every write site with its class; a unit test fails if a new call site to the comment-create primitive is added without a class tag (or equivalent guard).
  • Shadow flag off ⇒ byte-for-byte identical GitHub writes to today (test).
  • Shadow flag on ⇒ counters and would-notify records emitted; no GitHub write differs (test).
  • Machine-consumed protocol commands and required review submissions are classified and never marked would-suppress.
  • Docs explain how to compare matched baseline windows per active item, and that 80% is a target.
  • No new dependency or paid service.

Likely files

src/pkg/github/notification_tally.go, src/pkg/github/client.go (notificationsQuiet and comment primitives), src/pkg/github/pr_status_comment.go, src/pkg/config/config.go, metrics registration, src/docs/notifications.md, .github/workflows/* that post comments.

Evidence

RFC #16075 "Rollout and acceptance" step 1 and §5 ("classify each existing write site"); maintainer decision on #16075.

Independent of the status-writer child; can proceed in parallel.

Part of #16075

— hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88

Activity

  1. added
    help wantedDenotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.
    agent/scannerCreated or modified by the scanner agent
    on Nov 2, 2026
  2. hivecommons-hive commented on Nov 2, 2026

    @hivecommons-hive
    ContributorAuthor

    🔒 Claimed by @mrbobbytables (contributor on hive hosted-available-oke-11-placeholder-r05x). The claim lasts 30m0s without activity and renews while the work continues. Reply /unclaim to release, or /claim to take it over if you outrank the holder.

  3. hivecommons-hive commented on Nov 2, 2026

    @hivecommons-hive
    ContributorAuthor

    🔓 Claim by @mrbobbytables released (lease revoked).

  4. hivecommons-hive commented on Nov 2, 2026

    @hivecommons-hive
    ContributorAuthor

    🔒 Claimed by agent scanner (agent on hive hosted-available-oke-11-placeholder-r05x). The claim lasts 2h0m0s without activity and renews while the work continues. Reply /unclaim to release, or /claim to take it over if you outrank the holder.

  5. hivecommons-hive commented on Nov 2, 2026

    @hivecommons-hive
    ContributorAuthor

    🔓 Claim by agent scanner released (holding agent relaunched).

  6. added
    hive/likely-doneHive verified that a merged PR references or claims this issue; pending confirmation
    and removed on Nov 2, 2026
  7. hivecommons-hive commented on Nov 2, 2026

    @hivecommons-hive
    ContributorAuthor

    Fixed by #16321 (#16321, merged). Closing because Hive's PR claim metadata says this PR was the full fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent/scannerCreated or modified by the scanner agenthelp wantedDenotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.hive/likely-doneHive verified that a merged PR references or claims this issue; pending confirmation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions