docs: the containment tier was reported by a guess taken before the run - #348
Merged
Conversation
Records #345 and #347, which are user-visible and unreleased. The Windows containment work they came out of is deliberately NOT written up here: no Windows job has been confirmed to run its end-to-end test, and an entry claiming a mechanism nobody has watched work is the kind of sentence this file exists not to contain. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XdXrbksFKirm7yW6EDunur
There was a problem hiding this comment.
Pull request overview
Adds a new unreleased CHANGELOG entry documenting the previously fixed bug where multiple artifacts reported a pre-run containment probe rather than the run’s achieved tier, and explains why the discrepancy mattered (including audit evidence implications).
Changes:
- Add a detailed “Fixed” entry describing the containment-tier reporting defect and its impact across stdout,
--json, and the run record. - Clarify Linux-specific behavior (probe and achieved tier cannot disagree) and why this masked the issue on Linux.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| had quietly stopped being true*: containment on other platforms has | ||
| steps that fail after a probe says yes. | ||
|
|
||
| Two artifacts describing one run differently is worse than either being |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Records #345 and #347 in the CHANGELOG. Both are user-visible and unreleased.
The entry says what was wrong, why it mattered, and what holds now: three of the four places that report a containment tier were reading a pre-run probe rather than what the run achieved, one definition (
achieved_tier) now serves both the certifying and the reporting path, and Linux output is unchanged because on Linux the two cannot disagree.What this entry deliberately does not say
Nothing about Windows containment. That work is what surfaced the defect, and it is merged — the filters, the per-run identity, the audit lane, the E2E. But no Windows job has been confirmed to run that end-to-end test. The push to
mainshould have triggered it; I could not read job-level results, and the jobs endpoint returns zero for every run in this repo, so that is a tool limitation rather than evidence either way.An entry claiming a mechanism nobody has watched work is the kind of sentence this file exists not to contain. It ships when someone opens the run and confirms
windows build + E2Ewent green.Verification
diff size 25 <= 400, nothing silencedGenerated by Claude Code