fix(project-context): admit commands agents get wrong - #2703
Draft
alexeyv wants to merge 1 commit into
Draft
Conversation
Use unaided session history to decide which command invocations earn a guide line. Keep aided success from serving as removal evidence.
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
📝 WalkthroughWalkthroughThe project-context theory, documentation, skill, and reference templates now use unaided session history and review corrections to identify command errors and runtime pitfalls. Refresh, record, audit, admission, exclusion, and retirement rules now apply evidence-based criteria. ChangesProject Context Evidence Rules
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/bmm-skills/plan/bmad-project-context/references/template.md`:
- Line 8: The project-context output and brainstorming loader use different
default files, so verified facts are not consistently loaded downstream. Update
the bmad-project-context generation flow and the bmad-brainstorming
customize.toml default loader path to share the same contract: include AGENTS.md
in the loader defaults, or also write the generated context to
project-context.md; preserve existing context-loading behavior.
In `@src/bmm-skills/plan/bmad-project-context/SKILL.md`:
- Line 33: Update the wording in SKILL.md to use the hyphenated compound
modifier “agent-specific” instead of “agent specific,” without changing the
surrounding guidance.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 9473b8a7-c92c-4287-be27-a7bc0001b6ae
📒 Files selected for processing (6)
docs/explanation/project-context-theory.mddocs/explanation/project-context.mddocs/how-to/project-context.mdsrc/bmm-skills/plan/bmad-project-context/SKILL.mdsrc/bmm-skills/plan/bmad-project-context/references/best-practices.mdsrc/bmm-skills/plan/bmad-project-context/references/template.md
Comment on lines
+14
to
+18
| Written context earns its cost when it carries something the agent cannot derive by reading the repository, or a command that unaided session history shows agents still get wrong. | ||
|
|
||
| Two results from different literatures locate the same boundary. Separating code reasoning from documentation memorization across repository-level tasks, **code access delivers the dominant gains over documentation access** — a document describing how the system works loses to the source it describes. Running the inverse experiment — generating requirements *from* code — models prove unreliable at producing anything not already implemented. Current behavior is recoverable from source. **Intent, rationale, and what was deliberately rejected are not.** | ||
|
|
||
| So anything derivable is read live and never stored. A stored copy is a stale duplicate of something the agent reads more accurately first-hand, and it is charged on every single call. | ||
| So derivable facts are read live and never stored. The exception is evidence of behavior: if unaided session history shows agents choose the wrong command, the exact invocation changes behavior despite being derivable. Everything else stays at its source of truth, where it cannot drift from a stored copy. |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Repo files matching project context docs/skill:"
git ls-files | rg '(^docs/|src/.*)project-context|src/.*/bmad-project-context'
echo
echo "Locate references to review corrections:"
rg -n -i "review.?correct|review.?correction|review corrections|correction" .
echo
echo "Locate canonical terms around evidence/session history:"
rg -n "unaided session|interview the gaps|supplied unaided|session history|review" docs src/bmm-skills/plan/bmad-project-contextRepository: bmad-code-org/BMAD-METHOD
Length of output: 50381
🏁 Script executed:
#!/bin/bash
set -euo pipefail
for f in \
docs/explanation/project-context-theory.md \
docs/explanation/project-context.md \
docs/how-to/project-context.md \
src/bmm-skills/plan/bmad-project-context/SKILL.md \
src/bmm-skills/plan/bmad-project-context/references/best-practices.md \
src/bmm-skills/plan/bmad-project-context/references/template.md
do
echo "===== $f ====="
wc -l "$f"
done
echo
echo "Relevant file sections:"
sed -n '1,110p' docs/explanation/project-context-theory.md
echo "---- docs/explanation/project-context.md ----"
sed -n '1,80p' docs/explanation/project-context.md
echo "---- docs/how-to/project-context.md ----"
sed -n '1,80p' docs/how-to/project-context.md
echo "---- SKILL.md step 2-4 and refresh/record ----"
sed -n '1,130p' src/bmm-skills/plan/bmad-project-context/SKILL.md
echo "---- best-practices.md admission/exclusion/retirement ----"
sed -n '1,70p' src/bmm-skills/plan/bmad-project-context/references/best-practices.md
echo "---- template Running section ----"
sed -n '1,80p' src/bmm-skills/plan/bmad-project-context/references/template.mdRepository: bmad-code-org/BMAD-METHOD
Length of output: 36962
Apply the same evidence rule to review corrections.
docs/how-to/project-context.md and src/bmm-skills/plan/bmad-project-context/SKILL.md let users supply review corrections, but the admission, section, retention, and refresh rules still treat only unaided session history as command-error evidence. Define whether corrections are admissible, apply the decision consistently across docs/, SKILL.md, best-practices.md, and template.md, and state an explicit retirement rule for correction-admitted commands if aided success alone does not remove them.
📍 Affects 5 files
docs/explanation/project-context-theory.md#L14-L18(this comment)docs/explanation/project-context-theory.md#L49-L50docs/explanation/project-context-theory.md#L77-L83docs/explanation/project-context.md#L8-L22docs/how-to/project-context.md#L37-L43docs/how-to/project-context.md#L61-L61src/bmm-skills/plan/bmad-project-context/SKILL.md#L45-L53src/bmm-skills/plan/bmad-project-context/SKILL.md#L81-L81src/bmm-skills/plan/bmad-project-context/SKILL.md#L93-L101src/bmm-skills/plan/bmad-project-context/references/best-practices.md#L7-L16src/bmm-skills/plan/bmad-project-context/references/best-practices.md#L29-L29src/bmm-skills/plan/bmad-project-context/references/best-practices.md#L38-L39
Source: Path instructions
| 2. **Policy** — what the org requires. | ||
| 3. **Where things are** — entry points, and pointers to children and linked files. | ||
| 4. **Running and verifying** — only what `package.json`, a `Makefile`, or CI config does not already say. | ||
| 4. **Running and verifying** — commands unaided session history shows agents get wrong, plus what `package.json`, a `Makefile`, or CI config does not already say. |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n -C 4 'persistent_facts|project-context\.md|AGENTS\.md' \
src/core-skills src/bmm-skills docsRepository: bmad-code-org/BMAD-METHOD
Length of output: 50386
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf 'Files matching AGENTS/project-context/customize:\n'
fd -a '(^|/)AGENTS\.md$|project-context\.md$|customize\.toml$|SKILL\.md$|module-help\.csv$' . | sed 's#^\./##' | sort | head -300
printf '\nRelevant templates and customize values:\n'
for f in \
src/bmm-skills/plan/bmad-project-context/references/template.md \
src/bmm-skills/plan/bmad-project-context/SKILL.md \
src/core-skills/bmad-brainstorming/customize.toml \
src/bmm-skills/ship/bmad-code-review/SKILL.md
do
if [ -f "$f" ]; then
echo "--- $f"
wc -l "$f"
sed -n '1,180p' "$f"
fi
done
printf '\nTargeted search for AGENTS.md references in src/docs:\n'
rg -n --glob '!node_modules' --glob '!dist' --glob '!build' \
'\bAGENTS\.md\b|bmad-project-context|persistent_facts\s*=|file:\{project-root\}/\*\*/project-context\.md' \
src docs | head -500Repository: bmad-code-org/BMAD-METHOD
Length of output: 50385
Preserve the downstream context-loading contract.
bmad-project-context writes the verified block to AGENTS.md, but src/core-skills/bmad-brainstorming/customize.toml only defaults to file:{project-root}/**/project-context.md. Add AGENTS.md to this loader’s default path if brainstorming should carry the same facts, or keep the generated context in project-context.md as well.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/bmm-skills/plan/bmad-project-context/references/template.md` at line 8,
The project-context output and brainstorming loader use different default files,
so verified facts are not consistently loaded downstream. Update the
bmad-project-context generation flow and the bmad-brainstorming customize.toml
default loader path to share the same contract: include AGENTS.md in the loader
defaults, or also write the generated context to project-context.md; preserve
existing context-loading behavior.
| ### 1. Assess and report | ||
|
|
||
| Read `AGENTS.md`, harness or agent specific rule files, docs folders, and any notes carrying lessons. Report what exists and how it measures up, per `best-practices.md`. | ||
| Read `AGENTS.md`, harness or agent specific rule files, docs folders, notes carrying lessons, and any agent session history or review corrections the user supplied. Report what exists and how it measures up, per `best-practices.md`. |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Use the hyphenated modifier.
Change agent specific to agent-specific.
Based on the static analysis hint, the compound modifier needs a hyphen.
🧰 Tools
🪛 LanguageTool
[grammar] ~33-~33: Use a hyphen to join words.
Context: ...port Read AGENTS.md, harness or agent specific rule files, docs folders, notes...
(QB_NEW_EN_HYPHEN)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/bmm-skills/plan/bmad-project-context/SKILL.md` at line 33, Update the
wording in SKILL.md to use the hyphenated compound modifier “agent-specific”
instead of “agent specific,” without changing the surrounding guidance.
Source: Linters/SAST tools
alexeyv
marked this pull request as draft
August 9, 2026 22:37
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Teach
bmad-project-contextto admit exact commands when unaided session history shows agents getting them wrong, even when the command is derivable from repository files.Why
The previous derivability rule treated a command's presence in configuration as proof that agents would choose it correctly. Actual session history can show otherwise. Later clean sessions are aided by the instruction, so their success alone cannot prove the line is unnecessary.
How
Testing
HUSKY=0 npm ci && npm run quality