Skip to content

✨ governor: release-feed watcher kicks agents when a tracked upstream dependency releases - #18507

Merged
kubestellar-prow[bot] merged 2 commits into
v5from
feat/18494-release-feed-watcher
Nov 17, 2026
Merged

kubestellar-prow[bot] merged 2 commits into
v5from
feat/18494-release-feed-watcher

Conversation

@clubanderson

Copy link
Copy Markdown
Member

Summary

Adds a governor release-feed watcher. An agent that tracks upstream releases (dependency bumps) is now kicked within one poll interval of a new version, instead of finding it on its own timer.

  • Config: governor.release_feeds.agents.<agent>.sources[] with kinds github_release, github_tag (owner/repo), pypi (package JSON API) and index (generic URL, JSON field or plain text). It also takes interval_s and cooldown_s at block level, plus a per-agent cooldown_s, and enabled as an off switch. Named defaults (15m interval, 1h cooldown, 1m floor) can be overridden with HIVE_RELEASE_FEED_INTERVAL / HIVE_RELEASE_FEED_COOLDOWN. Validation rejects malformed sources.
  • Polling (pkg/releasefeed): each shared feed is fetched once per poll with If-None-Match / If-Modified-Since. Requests use a client whose proxy is crux/upstreamproxy.Default(), the spoke's upstream-proxy rules. State (validators, last-seen versions per agent, pending releases, last kick) is written atomically to /data/release-feeds.json. The first answer is a baseline and an unchanged feed kicks nothing.
  • Kick: a new version is queued through the K1 kick queue (crux.KickQueuer.EnqueueKick, falling back to SendKick on a runtime without it). It passes the same gates as governor.ci_failure_webhook_kick: agent known, enabled and unpaused in config or at runtime, and AgentEligibleForCELKick. A refused kick keeps the releases pending. The new built-in ${NEW_RELEASES} renders the block (package, old, new, URL). When the agent's template does not use it, a short targeted kick carries the block instead. The reason goes on the governor kick-history entry through the new KickRecord.Reason, which is persisted in the snapshot.
  • Credentials: auth_var names an existing per-agent or hive-level variable, resolved through ResolveRegistryForAgent and sent as a bearer token. The token is never logged or written to state. If it does not resolve, the feed is skipped. Index URL queries are redacted to a digest in keys, logs and links.
  • Docs: new src/docs/release-feeds.md. Updated env-vars, documentation map, agent-configuration (template variable) and the runtime-and-quota operator reference. Settings-parity allowlist entries are added (the keys are YAML-only).

Tests

Fake feeds (httptest) cover every kind, conditional 304s, auth headers, errors, and confirm the body cap and redaction. Watcher tests cover baseline/no-change/kick, cooldown folding, gated and failed kicks staying pending, restart persistence, auth_var, shared feeds and pruning, and Run/cancel. Also tested: config precedence and validation, scheduler ${NEW_RELEASES} substitution, RecordKickWithReason, and the cmd/hive kicker gates, message selection and token resolution.

Fixes #18494

🤖 Generated with Claude Code

@kubestellar-prow kubestellar-prow Bot added dco-signoff: yes Indicates the PR's author has signed the DCO. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Nov 16, 2026
@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

PR Title Relation Files
#18536 ✨ feature: choose remote repository write identity same files src/cmd/hive/main.go
src/docs/agent-configuration.md
src/pkg/dashboard/settings_parity_allowlist.txt
#18592 🐛 Betty closes the loop on deep dives; research agents survive config reloads same files src/cmd/hive/main.go
#18576 ✨ Owner-only Send to on model-owned stage-tracker rows same files src/cmd/hive/main.go
src/pkg/config/config.go
src/pkg/dashboard/settings_parity_allowlist.txt
#18583 ✨ Governor: opt-in advisory report in every repo (default off) same files src/docs/env-vars.md
src/pkg/config/config.go
#18567 ✨ Workflow models resolve agent dispatch targets and a hive-side profile mapping same files src/pkg/config/config.go
src/pkg/dashboard/settings_parity_allowlist.txt
#18577 ✨ Mail-in: text bodies as notes, video, 35 MiB SIZE, streamed and chunked relay same files src/docs/env-vars.md
#18571 ✨ Hand workflow-model states to their agent profiles (claim first, dispatch once) same files src/pkg/config/config.go
src/pkg/dashboard/settings_parity_allowlist.txt
#18593 ✨ Betty answers questions about the hive from its live state same files src/cmd/hive/main.go
#18586 ✨ Sandboxed agents open issues and keep topic claims through the request spool same files src/cmd/hive/main.go

@github-actions

Copy link
Copy Markdown
Contributor

Mutation testing

gremlins found 9 surviving mutant(s) in changed files: the code ran under tests, but changing it did not fail any test.

File Line Mutation
src/pkg/releasefeed/fetch.go 198 CONDITIONALS_BOUNDARY
src/pkg/releasefeed/fetch.go 205 CONDITIONALS_BOUNDARY
src/pkg/releasefeed/fetch.go 238 CONDITIONALS_NEGATION
src/pkg/releasefeed/fetch.go 331 CONDITIONALS_BOUNDARY
src/pkg/releasefeed/store.go 129 CONDITIONALS_NEGATION
src/pkg/releasefeed/watcher.go 78 CONDITIONALS_NEGATION
src/pkg/releasefeed/watcher.go 164 CONDITIONALS_BOUNDARY
src/pkg/releasefeed/watcher.go 341 CONDITIONALS_BOUNDARY
src/pkg/releasefeed/watcher.go 417 CONDITIONALS_NEGATION

7 mutant(s) in changed files are not reached by any test (NOT COVERED). A maintainer may apply the mutation/acknowledged label to accept this.

File Line Mutation
src/pkg/config/release_feeds.go 32 ARITHMETIC_BASE
src/pkg/releasefeed/egress.go 13 ARITHMETIC_BASE
src/pkg/releasefeed/egress.go 14 ARITHMETIC_BASE
src/pkg/releasefeed/egress.go 15 ARITHMETIC_BASE
src/pkg/releasefeed/egress.go 16 ARITHMETIC_BASE
src/pkg/releasefeed/watcher.go 97 CONDITIONALS_BOUNDARY
src/pkg/releasefeed/watcher.go 97 CONDITIONALS_NEGATION

Verdict: lived (#17349).

@hivecommons-hive

hivecommons-hive Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

@clubanderson this PR is held at stage/071-needs-author and the hive will not merge it yet.

Why:

  • the branch conflicts with the base and needs a rebase

Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path.

hive/awaiting-ci: rebased onto v5 (one import-list conflict in src/cmd/hive/boot.go, kept both imports). CI pending — sweep will merge when green.


🐝 Hive Agent: scanner | Instance: hosted-available-oke-11-placeholder-r05x | SHA: 1689e6352

— hive: agent=scanner backend=claude model=claude-sonnet-5-5 claude=2.1.325

@github-actions github-actions Bot removed the conflicts-with-open-pr Open PR has a merge conflict with another open PR label Nov 16, 2026
@hivecommons-hive
hivecommons-hive Bot force-pushed the feat/18494-release-feed-watcher branch from 0a9c7d6 to 563ce87 Compare November 16, 2026 21:29
@github-actions github-actions Bot added the conflicts-with-open-pr Open PR has a merge conflict with another open PR label Nov 16, 2026
@clubanderson

Copy link
Copy Markdown
Member Author

cc @joerunde: this is part of the hive changes for the spyre-inference deps agent and sandboxed reviewer setup.

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Scanner triage @563ce87: test (rest 4/4) fails on TestRunLoopWithAppliesMidCycleReloadWithoutClobbering (cmd/hive), 3 of 3 runs, so it is not a one-off flake. coverage and test fail only because that shard failed. The shared-baseline helper could not read PR checks (GitHub 504), so shared vs PR-local is not yet established. The PR touches the cmd/hive kicker wiring, so check that test against the diff. Not pushing to a human-authored branch.


🐝 Hive Agent: scanner | Instance: hosted-available-oke-11-placeholder-r05x | SHA: unknown

— hive: agent=scanner backend=claude model=claude-sonnet-5-5 claude=2.1.325

@github-actions github-actions Bot removed the conflicts-with-open-pr Open PR has a merge conflict with another open PR label Nov 16, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Security review at head 563ce87 (diff read; checkout not needed for these two points).

1. Bearer token can be sent over plain HTTP / downgraded on redirect (src/pkg/releasefeed/fetch.go Fetch, ~L170 Authorization: Bearer; endpoint() accepts http for index; egress.go NewEgressClient has no CheckRedirect; config validation release_feeds.go only requires an http:// or https:// prefix).
auth_var resolves any per-agent or hive-level variable and attaches it to whatever index URL is configured. With an http:// URL the credential goes in cleartext. Even with an https:// URL, net/http keeps Authorization on a same-host redirect, including https→http, so a hostile or misconfigured feed can downgrade and expose it. The NewEgressClient doc comment claims the token "never leaves its host", which does not cover scheme downgrade.

Requested fix: refuse to send a token unless the request is https, and refuse scheme downgrades on redirect.

--- a/src/pkg/releasefeed/fetch.go
+++ b/src/pkg/releasefeed/fetch.go
@@
-	if token != "" {
+	if token != "" {
+		if req.URL.Scheme != "https" {
+			return FetchResult{}, fmt.Errorf("fetch %s: refusing to send auth_var credential over %s", src.Key(), req.URL.Scheme)
+		}
 		req.Header.Set("Authorization", "Bearer "+token)
 	}
--- a/src/pkg/releasefeed/egress.go
+++ b/src/pkg/releasefeed/egress.go
@@
-	return &http.Client{Transport: tr, Timeout: egressRequestTimeout}
+	return &http.Client{Transport: tr, Timeout: egressRequestTimeout,
+		CheckRedirect: func(req *http.Request, via []*http.Request) error {
+			if len(via) >= 10 {
+				return errors.New("stopped after 10 redirects")
+			}
+			if via[0].Header.Get("Authorization") != "" && req.URL.Scheme != "https" {
+				return errors.New("refusing https->http redirect with credential")
+			}
+			return nil
+		}}

(add "errors" import; also add a test with an http:// index source plus auth_var asserting no request is sent.)

2 (advisory, out-of-scope-ish). Upstream index version text reaches the kick prompt after whitespace stripping and a 128-byte cap (fetch.go cleanVersion). That blunts but does not eliminate prompt injection, e.g. a long hyphenated instruction string. Consider restricting versions to [A-Za-z0-9._+~:-]. Mentions are neutralized in release_feed_watcher.go Kick, which is good.

— hive: agent=adjudicator-5 backend=claude model=claude-sonnet-5-5 claude=2.1.325

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Review follow-ups (below review.backlog.min_severity: P2)

Non-blocking findings below the backlog filing line are listed here instead of as issues, one comment per PR, updated in place. Tick them off as they are fixed; set review.backlog.min_severity: P3 to file every finding.

  • P3 low security — Upstream version text reaches kick prompt with only whitespace/length filtering (src/pkg/releasefeed/fetch.go:300): cleanVersion strips whitespace and caps at 128 bytes but allows arbitrary characters, so a hostile index feed can still inject a short instruction-like string into the agent prompt. Restrict to a ver…

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Baseline triage: native-queue-policy is stale-branch (25 behind v5) and the PR is DIRTY with conflicts. Human-authored, so not pushing; please merge v5 in.


🐝 Hive Agent: scanner | Instance: hosted-available-oke-11-placeholder-r05x | SHA: unknown

— hive: agent=scanner backend=claude model=claude-sonnet-5-5 claude=2.1.325

@hivecommons-hive

Copy link
Copy Markdown
Contributor

scanner: this PR is currently dirty (merge conflict with the base) and carries a changes-requested review at 563ce87. It is authored by a human, so I am not pushing to the branch. Please rebase onto the base branch; deferring until then.


🐝 Hive Agent: scanner | Instance: hosted-available-oke-11-placeholder-r05x | SHA: unknown

— hive: agent=scanner backend=claude model=claude-sonnet-5-5 claude=2.1.325

@clubanderson
clubanderson force-pushed the feat/18494-release-feed-watcher branch from 563ce87 to 646b6c9 Compare November 17, 2026 00:31
@github-actions github-actions Bot added the conflicts-with-open-pr Open PR has a merge conflict with another open PR label Nov 17, 2026
@clubanderson clubanderson added the lgtm Indicates that a PR is ready to be merged. label Nov 17, 2026
@clubanderson clubanderson added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 17, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by:

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

… dependency releases

Add governor.release_feeds: per agent, GitHub release/tag, PyPI or generic
index feeds. The watcher polls them with If-None-Match/If-Modified-Since
through the spoke's upstream-proxy rules, persists last-seen versions in
/data/release-feeds.json, and on a new version queues a kick through the
kick queue with a ${NEW_RELEASES} block, recording the reason on the
governor kick-history entry. Kicks pass the CI-failure kick gates and are
bounded by a per-agent cooldown. Private indexes authenticate with
auth_var, an existing per-agent variable that is never logged.

Fixes #18494

Signed-off-by: Andy Anderson <andy@clubanderson.com>
A config reload overwrites b.cfg in place under cfgReloadMu, so the
watcher goroutine reading it raced the reload (caught by the race
detector in TestRunLoopWithAppliesMidCycleReloadWithoutClobbering).
Read b.sweepConfig, the immutable snapshot the loop republishes after
every reload, instead.

Signed-off-by: Andy Anderson <andy@clubanderson.com>
@hivecommons-hive
hivecommons-hive Bot force-pushed the feat/18494-release-feed-watcher branch from 2e33c68 to 1689e63 Compare November 17, 2026 01:21
@kubestellar-prow kubestellar-prow Bot removed the lgtm Indicates that a PR is ready to be merged. label Nov 17, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

New changes are detected. LGTM label has been removed.

@clubanderson clubanderson added the lgtm Indicates that a PR is ready to be merged. label Nov 17, 2026
@kubestellar-prow
kubestellar-prow Bot merged commit 0670cc0 into v5 Nov 17, 2026
97 of 99 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the feat/18494-release-feed-watcher branch November 17, 2026 02:16
joerunde pushed a commit to joerunde/spyre-inference that referenced this pull request Nov 17, 2026
Checks in the hive `deps` agent's prompt, definition and dependency
targets under `.hive/`, where hive's repo agent discovery looks for
them. The flow is unchanged: bump one dependency group, compile it, open
an issue if that fails, open one PR on success, fix our side when CI is
red, file dependency bugs upstream, recommend a refresh only when a
newer release may fix the bug, never run the test suite and never wait
for CI.

Hive features this now uses, all merged in hive v5:

- `hive-topic claim` from the sandbox, so each group keeps its one-PR
claim (hivecommons/hive#18586).
- `hive-open-issue` from the sandbox for dependency bugs (on
torch-spyre) and compile failures (here). This replaces the `known_bad`
ledger PR. `known_bad` stays in `targets.yaml` as a cap maintainers can
set by hand (hivecommons/hive#18586).
- Fix kicks on the agent's own red bump PR commit on top of it, and
sandbox PRs are credited to `deps`, so red CI kicks `deps` and not
`scanner` (hivecommons/hive#18587). PRs opened before that fix still get
the fix as a patch in the report.
- `${NEW_RELEASES}` from the release-feed watcher
(hivecommons/hive#18507).
- `.hive/agents/` discovery (hivecommons/hive#18505).

Files:

- `.hive/agents/deps.md`: the kick template (`prompt_source`).
- `.hive/agents/deps.yaml`: optional presentation-only definition.
- `.hive/deps/targets.yaml`: dependency groups. Only `deps-torch-spyre`
is enabled.
- `.gitignore`: keeps hive's per-run scratch under `.hive/` ignored and
tracks only `.hive/agents/` and `.hive/deps/`.

This overlaps torch-spyre#1626, which puts the prompt at
`.github/hive/prompts/deps.md` and ignores all of `.hive/`. Its prompt
content is merged into `.hive/agents/deps.md` here. If this lands first,
torch-spyre#1626 can drop its prompt and `.gitignore` hunks.

## Setup after merge (hive owner)

1. **Approve the prompt source.** Settings, Security, Prompt sources. On
the `torch-spyre/spyre-inference` row, click **Approve…** and confirm.
Leave `allow_github_prompt` and `github_prompt_allowlist` alone.
2. **Point the agent at the prompt.** Agents, `deps`, General tab,
**Prompt from GitHub repo**: Owner `torch-spyre`, Repo
`spyre-inference`, File path `.hive/agents/deps.md`, Ref `main`. Save.
Keep the current kick template as the cold-start fallback, and replace
its text with this file so the fallback runs the same flow.
3. **Variables tab** (static, template scope):
- `DEPS_LOCAL_CHECK`: `uv sync --frozen --inexact && uv run --no-sync
python -c 'import torch_spyre, spyre_inference;
print(torch_spyre.__version__)'`
   - `DEPS_MAX_REFRESHES`: `2`
   - `DEPS_UPSTREAM_REPO`: `torch-spyre/torch-spyre`
4. **Cadences tab.** idle, quiet and busy `24h`, surge `pause`.
5. **Optional, release-feed kicks** (YAML only, no Settings control
yet): `governor.release_feeds.agents.deps.sources: [{kind: github_tag,
repo: torch-spyre/torch-spyre}]`, with `cooldown_s: 21600`. Keep the
daily cadence for CI follow-up.
6. **Import tab.** `deps.yaml` will show under "Declared in your repos".
Leave it pending or reject it, because approving it would create a
second `deps` agent.
7. Unpause `deps`.

cc @joerunde

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: Andy Anderson <andy@clubanderson.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. dco-signoff: yes Indicates the PR's author has signed the DCO. lgtm Indicates that a PR is ready to be merged. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

✨ Governor release-feed watcher: kick agents when a tracked upstream dependency releases

1 participant