diff --git a/AGENTS.md b/AGENTS.md index 55c8219e5a..e955f8b36a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -168,3 +168,47 @@ them alone proves succession. attributed a "timeout defaults to null" sentence to it that appears only in `#1891`'s PR body — both caught by grepping the file instead of trusting the summary that introduced them. + +## Test-gate regressions and stale-PR merges + +- A red `tests`, coverage, or `interrogate` gate on your pull request is not proof that your + diff caused it. Full-suite execution on a push to `main` is not guaranteed: the workflows + that run `pytest tests` on push are `paths:`-filtered, so a pairing broken outside their + declared paths reaches `main` with no full-suite run. The breakage then surfaces on the + next pull request whose review dispatch does run the suite, and fails it regardless of + that request's own diff. This procedure covers the suite gates only; a red Semgrep, + CodeQL, Strix, or Scorecard check is a different diagnosis. +- Reproduce a suspect failure on a clean baseline before repairing it. Run + `git worktree add /tmp/baseline --detach`, then `cd /tmp/baseline` + and run `python3 -m pytest tests -q`; that takes roughly four minutes and needs no + virtualenv. You must `cd` into the worktree: over thirty test files read repository files + through working-directory-relative paths such as `Path(".github/workflows/...")`, so + pointing pytest at the baseline directory from your own checkout silently tests your tree + and reports a green baseline that proves nothing. Baseline the pull request's actual base + or merge-base rather than `origin/main` once `main` has moved past it. If the failure + reproduces on the baseline it is pre-existing: repair it as its own pull request and name + the change that introduced it. +- When you change a workflow file or a `scripts/ci/` module, grep the whole `tests/` tree + for every literal you touched — event-type strings, cron expressions, environment-variable + names, tuple members, pinned digests — not only the obviously named sibling test. A change + can satisfy one oracle and still leave a second, independent one stale. +- Read a stale pull request's own changes with a three-dot diff — + `git diff ...` — or with `gh pr diff`, which is already three-dot. A two-dot + `git diff ` renders everything the base gained since the fork point as though + this branch deleted it, so an untouched branch reads as a mass revert. +- Content-hash pins exist under `tests/`; find them before editing a workflow. Run + `grep -rn 'hash-object' tests/` — today that is the `git hash-object` pin of + `.github/workflows/opencode-review-dispatch.yml`. Any byte change to a pinned file makes + its constant stale and fails a required gate for every open pull request, reverts included, + because a revert restores the original bytes while the pin stays on the reverted value. + Recompute only with `git hash-object `, and only for a constant you have confirmed is + a blob pin. Nearly every other forty-hex literal under `tests/` is something else — a + pinned action SHA, a vendored-revision pin, a synthetic fixture head, or an assertion that + a SHA appears in a document — and pointing `hash-object` at any of those produces a wrong + value that breaks what it replaces. A second contract re-derives the dispatch pin by + regular expression from the first, so keep the assignment on one line and correct it in one + place. +- Production code under `scripts/ci/` branches on `GITHUB_ACTIONS`, and pytest inherits that + variable in CI, so a failure class exists that cannot reproduce locally. Before calling a + scheduler change clean, run the affected tests both ways, including + `GITHUB_ACTIONS=true python3 -m pytest `. diff --git a/CLAUDE.md b/CLAUDE.md index e5ceba7140..f1b2cbb482 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -196,3 +196,26 @@ repeatable compile command. longer than it is. Querying `status=success` and `status=failure` directly cuts through the churn to the most recent real conclusion of each kind. Those are historical signals about pipeline liveness only — they never substitute for exact-current-head evidence on the PR you are acting on. +- **Do not assume `interrogate` skips private helpers.** `[tool.interrogate]` here sets no + `ignore-*` flags and the tool defaults them off, so a docstring-less `_helper` or `__helper` in + `scripts/ci/` counts against the 100% gate — it is the stricter docstring check, not the laxer + one. Sibling repositories configure this differently (`contextual-orchestrator` enables six + `ignore-*` flags and does skip them), so read the target repo's `pyproject.toml` rather than + carrying a docstring habit across repositories. Note also that `ignore-private` would cover only + double-underscore names; single-underscore needs `ignore-semiprivate`. +- **A stale PR's conflict scope is a snapshot, not a property of the PR.** Any advance of the base + between measuring the conflicts and resolving them invalidates the list, and base advances land in + the same directories conflicts do (`.github/workflows/`, `scripts/ci/`, `docs/doctoring/`). Scope + grows as often as it shrinks — a branch that merged cleanly can become conflicted with no change + to the branch at all — so re-run the merge yourself immediately before resolving and treat any + earlier measurement, including your own from minutes ago, as expired. Resolving against a stale + smaller scope silently leaves conflicts unhandled. +- **No test parses fenced code blocks.** The doc-contract tests match exact prose in specific files; + none of them check Markdown structure, and `ARCHITECTURE.md` (five mermaid diagrams) is read by no + test at all. A conflict resolution that splits a fenced block into two fragments therefore ships + green, rendering the diagram source as a plain code block. After resolving a conflict in a + document containing fenced blocks, re-read the whole enclosing section rather than the diff hunk, + and confirm each block has one opening fence carrying its language tag and one matching closing + fence. Do not check by counting fences — a split leaves four where there were two, so an even + count proves nothing. The damage can also arrive inherited, from an earlier commit on the same + branch or from the autofix flow's conflict-marker resolution.