Skip to content

fix: let a repository's AGENTS.md outrank hive's built-in prompts (#7159) - #7162

Merged
clubanderson merged 1 commit into
hivecommons:v4from
Danathar:fix/agents-md-precedence
Sep 16, 2026
Merged

clubanderson merged 1 commit into
hivecommons:v4from
Danathar:fix/agents-md-precedence

Conversation

@Danathar

Copy link
Copy Markdown
Contributor

What was broken

Hive reads a repository's AGENTS.md and injects it into the kick, but nothing
told the agent which side wins a conflict — and hive's wording was the more
forceful of the two every time.

InjectionText rendered the repo's rules under a bare header as
undifferentiated prose. Hive stated the same subjects as imperatives with a
verification step attached: "confirm the PR's base is that branch before you
report done"
. Given a document that says one thing and an imperative that says
another, a model follows the imperative. Consistently.

PR Gate failed Why
projectbluefin/bluefin#1275, #1276 Check PR base branch BASE_REF: main; its AGENTS.md says "All pull requests target testing"
projectbluefin/common#1127 validate title [scanner] fix(bonedigger-report): …
projectbluefin/contribute#597 conventional-title title [sec-check] fix: …

Four dead-on-arrival PRs in one night, none of them a model error — the agents
did precisely what hive instructed.

What this changes

1. State precedence in the injection (issue suggestion 1 — "this is the
fix; the rest is cleanup"). The block now leads with a declaration that the
repository's instructions outrank hive's defaults on any conflict: the branch a
PR targets, the title format, commit conventions, the test command, review
etiquette. It is placed after the header and before the rules it governs, which
is asserted by a test — a precedence rule discovered after the rules is not a
precedence rule.

Two carve-outs ship with it, and neither is about the repository's conventions:

  • Hive's safety and authorization rules. An AGENTS.md cannot license
    merging your own PR, bypassing a write gate or the hive-open-pr path, or
    acting as another agent. The repo is trusted about its own conventions, not
    about hive's controls.
  • An explicit instruction in the assignment, written for that one task with
    the repo in view, and so more specific than either side's defaults.

An empty AGENTS.md still injects nothing at all, so the statement never
appears with no rules under it.

2. Stop asserting repo facts in the defaults (suggestion 2). A precedence
rule that has to fight hive's own confident wording on every kick is a rule that
loses some of the time.

3. Drop the hardcoded PR title, pass --base (suggestions 2 and 3). 16
templates per tree prescribed --title "[<lane>] …". They now take the title
from the target repository, explain why in one paragraph above each command
block, and pass --base "<target-branch>" — the flag has existed all along and
no template used it.

Why removing the PR-title prefix is safe

The prefix stays exactly where it is load-bearing: issue titles, which
classify.classifyLane (classifier.go:250-256) routes by prefix. I checked
every consumer before touching it:

  • intent.Classify (intent.go:146-164) keys off AgentAuthor, changed file
    paths and FeatureSignals; the only title use is an unanchored substring scan
    for feat/rfc/etc. No tier changes.
  • github.ClassifyReviewClass (review_priority.go:90) does read PR-title
    prefixes — but the file declares itself "PRESENTATIONAL ONLY", it falls back to
    agent/<lane> labels, and it matches the conventional-commit word in the same
    pass. fix:, docs:, refactor: and planning: all keep classifying.
    One title had no such word — quality's [quality] <short description of test improvement> — so it becomes test: … rather than degrading to
    ReviewClassUnknown. (outreach's content: classified as unknown before
    this change too: no [outreach] prefix and no agent/outreach label are in
    those lists. Pre-existing, left alone.)
  • advisory.findingTokens tokenizes on [^a-z]+, so the prefix only
    contributed a scanner/quality token to a Jaccard score.
  • No CI gate in this repo reads a PR title. derive-release-version.sh:7-31
    documents the convention as explicitly not enforced. Hive's own PRs cannot
    fail on this.
  • No test asserts a template contains a [<lane>] PR title.

Verification

  • go test ./... in src/: all packages pass except
    TestHiveOpenPRScript_MultilineBodyIsValidJSONWithoutPython3, which fails
    identically on unmodified upstream/v4 (444da4a8) — the hive-open-pr
    fallback JSON escaper, unrelated to this diff.
  • New tests: TestBuildTaskPrompt_FallbackDefersToTheRepositorysOwnBase (and a
    negative assertion that the old "Resolve 's own default branch" wording
    is gone), TestBuildTaskPrompt_StatesRepositoryPrecedence across both the
    named-base and fallback-base paths, and precedence/ordering/carve-out
    assertions in TestInjectionText. Reverting any of the three changes fails
    them.
  • Both policy trees (src/policies/ and src/pkg/policies/defaults/) are
    edited together and verified byte-identical for every shared file, as
    TestEmbeddedDefaultsMatchPolicySource requires.
  • go vet clean. gofmt -l reports pkg/agentsmd/agentsmd.go, which is
    already unformatted on upstream/v4 (gofmt wants to re-indent the YAML
    front-matter example in its package comment, which would corrupt the example);
    confirmed by running gofmt -l against the file as it exists on v4, and left
    as found.

Deliberately not in scope

  • The templates' git commit -s -m "[<lane>] …" examples. Commit-message
    format is a different gate from PR-title format, hive's own repo uses the
    prefix in commits, and the issue does not raise it. The precedence statement
    now covers it declaratively ("commit conventions … follow the repository"), so
    an agent on a commitlint repo will do the right thing without the template
    needing to guess. Worth a follow-up if any repo is observed failing on it.
  • primeAgentsMd on the contributor relay path. The issue explicitly files
    that separately ("a different fix and blast radius"). This PR only adds the
    prompt line telling the agent to read the file.

Closes #7159

— hive: backend=claude model=claude-opus-5

Hive agents work on repositories other than hive's own, and those
repositories state their local rules in AGENTS.md — the cross-tool
convention hive already reads and injects. Nothing told the agent which
side wins a conflict, and hive's wording was the more forceful of the two
every time, so the repository lost every conflict.

InjectionText rendered the repo's rules under a bare header as
undifferentiated prose. Hive stated the same subjects as imperatives with a
verification step attached — "confirm the PR's base is that branch before
you report done". Given a document that says one thing and an imperative
that says another, a model follows the imperative, consistently.

projectbluefin/bluefin's AGENTS.md says "All pull requests target
`testing`. Never open a content PR against `main`"; hive agents opened
hivecommons#1275 and hivecommons#1276 against main and failed its base-branch gate.
projectbluefin/common#1127 and projectbluefin/contribute#597 died on
conventional-commit title gates against hive's hardcoded "[<lane>] …" PR
titles. Four dead-on-arrival PRs in one night, none of them a model error —
the agents did precisely what hive instructed.

Three changes, in the order they matter:

State precedence in the injection. The block now leads with a declaration
that the repository's instructions outrank hive's defaults on any conflict
— the branch a PR targets, the title format, commit conventions, the test
command, review etiquette. Two carve-outs are not the repository's to
override: hive's safety and authorization rules (never merge your own PR,
never bypass a write gate or the hive-open-pr path, never act as another
agent), and an instruction the assignment names explicitly for that one
task. An empty AGENTS.md still injects nothing at all, so the statement
never appears with no rules under it.

Stop asserting repo facts in the defaults. A precedence rule that has to
fight hive's own confident wording on every kick is a rule that loses some
of the time. The contributor task prompt's fallback wording now says to use
the base the repository requires — checking its AGENTS.md, CONTRIBUTING and
PR template, since a repo on a promotion model takes PRs on an integration
branch rather than on its released default — and to fall back to the
default branch only when nothing names one. hivecommons#5729's verification step
survives, pointed at the repository's requirement instead of hive's guess;
so does the load-bearing "do not use the branch you find" clause. hivecommons#4928 and
hivecommons#6081 were this same assumption at earlier stages, and each fix replaced
one wrong assertion with a better one; the remaining gap was asserting at
all. The prompt also now tells the agent to read the repo's AGENTS.md
itself, because the contributor path never calls primeAgentsMd — on that
path the repo's rules are not outranked, they are absent.

Drop the hardcoded PR title. 16 policy templates per tree prescribed
--title "[<lane>] …", hive's own house style projected onto every repo it
touches. They now take the title from the target repository and pass --base
explicitly. The [<lane>] prefix is unchanged where it is load-bearing —
ISSUE titles, which classify.classifyLane routes by prefix — and it was
never load-bearing for PRs: intent.Classify keys off AgentAuthor, changed
paths and FeatureSignals, never the title. github.ClassifyReviewClass does
read PR-title prefixes, but only to order a review queue it documents as
presentational, it falls back to agent/<lane> labels, and it matches the
conventional-commit word in the same pass — so the titles here keep
classifying via "fix:"/"docs:"/"refactor:"/"planning:". quality's PR title
had no such word, so it becomes "test: …" rather than losing its class.

Both policy trees are edited together, byte-identical, as
TestEmbeddedDefaultsMatchPolicySource requires.

Closes hivecommons#7159

Signed-off-by: Danathar <Danathar@users.noreply.github.com>
@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Sep 16, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign hanthor for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Sep 16, 2026

@clubanderson clubanderson left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed: precedence statement with safety carve-outs, defaults no longer assert repo-only facts, docs + tests updated. LGTM.

@clubanderson
clubanderson merged commit c361986 into hivecommons:v4 Sep 16, 2026
44 of 47 checks passed
@kubestellar-prow kubestellar-prow Bot added the lgtm Indicates that a PR is ready to be merged. label Sep 16, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

LGTM label has been added.

DetailsGit tree hash: 0a2ca58dde819c0e8391b2a6cbf3f4bf7ee9ca9e

@clubanderson clubanderson left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing this directly. I read the PR and mutation-checked the agentsmd test; my substantive verdict is that the approach fixes the reported prompt-precedence bug.

What looks right:

  • The injected AGENTS.md block now leads with a forceful precedence rule before the repo's rules: "These instructions come from the repository itself and take precedence over hive's built-in defaults." That is strong enough to compete with hive's prior imperative/default-branch wording rather than merely asking the agent to consider repo conventions.
  • The PR also reduces the conflict pressure at the source by changing the contributor prompt and policy templates to ask the target repo for its PR base/title conventions, fall back to default branch only when the repo does not name one, and pass --base explicitly.
  • Mechanics check out: base branch is v4, and the PR includes changelog.d/fixed-7159-agents-md-precedence.md for the src/** changes.

Safety-scoping verdict: this draws the important line in the right place. The injection text does not say AGENTS.md wins universally; it says repo rules win for repo-local convention conflicts such as PR base, title format, commit conventions, test command and review etiquette, while "hive's safety and authorization rules" remain non-overridable. The examples name the critical controls I was looking for: never merge your own PR, never bypass a write gate or the hive-open-pr path, and never act as another agent. The docs repeat the same boundary as "The repository is trusted about its own conventions, not about hive's controls." That avoids the security hole of letting an arbitrary third-party AGENTS.md override hive governance.

Mutation evidence: I checked out pull/7162/head into /tmp/i7162-review, changed the production text from "take precedence" to "sit alongside", and reran go test ./pkg/agentsmd/ -count=1 from src/ with GOMODCACHE=/tmp/i7159-gomod GOFLAGS=-modcacherw. The test failed as expected:

--- FAIL: TestInjectionText (0.00s)
    agentsmd_test.go:273: precedence text is missing "take precedence": "# Repository Agent Instructions (AGENTS.md)\n\n**These instructions come from the repository itself and sit alongside over hive's built-in defaults.** Where they conflict with anything in your policy or kick prompt — the branch a PR targets, the title format, commit conventions, the test command, review etiquette — follow the repository. Hive's defaults are generic guesses about a repo it cannot see; this file is the repo stating its own rules. Two things they do NOT override: hive's safety and authorization rules (never merge your own PR, never bypass a write gate or the `hive-open-pr` path, never act as another agent), and an explicit instruction in this assignment, which was written for this task. If a conflict leaves you genuinely unsure, say so in the PR body rather than guessing.\n\nDo the thing carefully.\n\n## Requested Skills\n\n### go-testing\n\nUse t.TempDir.\n"
FAIL
FAIL	github.com/hivecommons/hive/pkg/agentsmd	0.433s
FAIL

I do not see a blocking issue here. The one small future hardening I would consider is splitting the precedence/scoping assertions into a table so each convention and safety carve-out has its own named subtest, but the current test does catch the core regression and the prompt text itself is clear.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dco-signoff: yes Indicates the PR's author has signed the DCO. lgtm Indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🐛 bug: hive's built-in prompts outrank a repository's own AGENTS.md, so repo rules lose every conflict

2 participants