docs: three stale claims that contradict what the code and the scorecard say - #111
Merged
Conversation
…ard say Each of these is a documented fact that HEAD disproves. No behavior changes -- docs and workflow comments only; cla.yml's triggers are untouched. 1. Secure_Development_Standards.md:378 -- ASVS V3 marked OUT of scope Reason given: "No browser-delivered UI (PySide6 desktop + APIs); documented exclusion. Re-scope if a web/admin UI is added." That trigger fired and nobody pulled it: BACKLOG #103 retired the desktop console on 2026-07-13 and made the browser console at /ui the SOLE operator UI. The assessment of record already disagrees with this row -- it scores V3 in full (31 requirements: 18 Pass / 9 Partial / 0 Fail / 4 N/A) and claims no exclusion anywhere. So this was stale prose contradicting the scorecard, not a scope decision awaiting a ruling. SECURITY-LOOSENING.md:396 had already mapped serve_web_console to V3, so the tree contradicted itself. Also corrects the :394 roll-up: 12 -> 13 chapters active, and "Documented exclusions: V3, V17" -> "Documented exclusion: V17". 2. EARLY-ADOPTER-GUIDE.md:636 -- "there is no Prometheus exporter" There is. messagefoundry/api/metrics.py IS the Prometheus exporter, and api/app.py:4147 registers GET /metrics returning Prometheus exposition (text/plain), gated by monitoring:read exactly like /stats, PHI-safe by construction (BACKLOG #21 -- labels are operator-assigned identifiers only, never a message field). prometheus_client is a hard import, not optional. This is the one with real cost: an adopter reading it wires up log parsing and /stats polling for a capability that already ships. /metrics and /metrics/history are now listed first. 3. security.yml:16 and cla.yml:19 -- "this private repo" The repo is PUBLIC (gh repo view: visibility=PUBLIC). security.yml's claim was that CodeQL needs GitHub Advanced Security here; CodeQL runs and passes as a required context on every PR, which is its own disproof. cla.yml's is the one that matters: it justifies omitting the `closed` trigger as "moot on this private repo", and its own re-add condition is "if the repo ever takes fork PRs from non-allowlisted contributors" -- which a public repo can do today. Corrected to say the ALLOWLIST, not visibility, is what makes the on-close behavior unnecessary. Triggers unchanged; this flags a decision, it does not make one. Left alone deliberately: release.yml:84 (past-tense history of a step REMOVED at the cutover), release.yml:271 and scorecard.yml:9 (both already state the repo IS public). Secure_Build_Scorecard_MEFOR.md:64 also mentions "documented V3/V17 exclusions" but is a dated 2026-07-14 reconciliation record citing a superseded assessment -- editing it would revise history. Verified: 117 tests green across test_security_doc_drift, test_security_static, test_security_posture and test_dast_claims; both edited workflows re-parsed and their triggers confirmed unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Aug 1, 2026
Merged
wshallwshall
added a commit
that referenced
this pull request
Aug 1, 2026
…superseded assessment (#114) `Secure_Build_Scorecard_MEFOR.md` named ASVS-L3-ASSESSMENT-2026-07-16 as "the CURRENT canonical verdict-of-record" and instructed "Grade off this doc." Both claims were wrong. Four dated re-scores have landed since, and its Posture A/B counts rest on a posture split that no longer exists -- there is one scored posture now, not two. Repoints at the current verdict-of-record and marks the old entry superseded. Following this file's existing "cited, not restated" convention for the maintainer-internal corpus, no new counts are published here; the pre-existing published figures are retained only to mark them stale. Replaces PR #112, whose other half (the ASVS V3 exclusion row) was fixed independently by #111 while it was open.
wshallwshall
added a commit
that referenced
this pull request
Aug 1, 2026
§A.3's roll-up reads "In scope: 14 chapters active today (V1–V8, V11–V16)" — it said 13. V1–V8 is 8 and V11–V16 is 6. The table directly above it is the authority and agrees with 14: exactly 14 rows carry an in-scope "Yes" (V1–V8, V11–V16), V17 is the sole "No", and V9 and V10 carry their own qualified verdicts. This is the same off-by-one that was already in the line before #111 — it read "12 chapters active (V1, V2, V4–V8, V11–V16)", which is 13. #111 correctly brought V3 into scope but carried the arithmetic error forward rather than recomputing it, so the count moved 12→13 when the list moved 13→14. Found because the website mirrors this file as `docs/secure-development-standards.md` and publishes it as `assets/MessageFoundry-Secure-Development-Standards.pdf`. That copy was corrected to 14 in website PR #105, so the two have been disagreeing since #111 merged. This makes them agree, with the engine as the correct one. The "17 chapters" statements at :243 and :248 are right and untouched — that is the total in ASVS 5.0, not the in-scope count. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
wshallwshall
added a commit
that referenced
this pull request
Aug 2, 2026
…ments in the item itself The #340 filing was agreed to need cost evidence. The hand account it was to be built from ("#132 took four full CI cycles to land, none from a failure — green -> main moved -> rebase, x3, ~80 min of runner time for a change correct on the first pass") was wrong in every load-bearing figure, so this re-derives all of it from the Actions API instead of relaying it. What #132 actually did: nine CI runs / ten attempts across eleven head moves in a 3h28m window; two attempts failed on real defects (one on the head it opened at, so it was not correct on the first pass — it opened at one commit and merged at five); four runs cancelled in flight by cancel-in-progress; five head moves were rebases, four of them onto a main tip that had landed 1-14 min earlier. Cost 212.7 min run wall-clock / 442.5 min job wall-clock — quantities that differ by 2.08x, so the unit is load-bearing. No billable figure exists: /timing reports 0 billable ms (self-hosted). Two statements already in the item were false and are corrected: - "A hand-coordinated merge freeze ... *did* hold main still for a full window." It did not. main advanced four times between #119's first fully-green head and its merge, one of them 8m26s after the freeze was recorded in a work claim. - "#119 still failed" / "the bounds that actually killed #119" reads as the PR dying. #119 merged at 2026-08-02T01:45:00Z, 12h15m after arming. The caveat is stated level with the cost, per the standard this repo keeps violating. Notably the overnight serialization is NOT evidence hand coordination worked — it is entailed by strict + a 20-25 min suite — and strict does not always serialize (#110/#111 merged 24s apart; mechanism not established). Also measured: no workflow carries a merge_group: trigger, so zero of the 13 required contexts would report on a queue ref, which makes Proposed step 2 a precondition rather than a follow-up. Adds the measurement-cost linkage to #344: supersession-and-re-run produces multi-attempt runs, and the default actions/runs/{id}/jobs endpoint returns only the latest attempt, so failed earlier attempts vanish from any duration table built from it — at exactly the tight end where a margin is decided. #344's own figures are untouched; another session holds that item.
wshallwshall
added a commit
that referenced
this pull request
Aug 2, 2026
…ments in the item itself (#143) * backlog(#340): the merge-queue cost, re-derived — and two false statements in the item itself The #340 filing was agreed to need cost evidence. The hand account it was to be built from ("#132 took four full CI cycles to land, none from a failure — green -> main moved -> rebase, x3, ~80 min of runner time for a change correct on the first pass") was wrong in every load-bearing figure, so this re-derives all of it from the Actions API instead of relaying it. What #132 actually did: nine CI runs / ten attempts across eleven head moves in a 3h28m window; two attempts failed on real defects (one on the head it opened at, so it was not correct on the first pass — it opened at one commit and merged at five); four runs cancelled in flight by cancel-in-progress; five head moves were rebases, four of them onto a main tip that had landed 1-14 min earlier. Cost 212.7 min run wall-clock / 442.5 min job wall-clock — quantities that differ by 2.08x, so the unit is load-bearing. No billable figure exists: /timing reports 0 billable ms (self-hosted). Two statements already in the item were false and are corrected: - "A hand-coordinated merge freeze ... *did* hold main still for a full window." It did not. main advanced four times between #119's first fully-green head and its merge, one of them 8m26s after the freeze was recorded in a work claim. - "#119 still failed" / "the bounds that actually killed #119" reads as the PR dying. #119 merged at 2026-08-02T01:45:00Z, 12h15m after arming. The caveat is stated level with the cost, per the standard this repo keeps violating. Notably the overnight serialization is NOT evidence hand coordination worked — it is entailed by strict + a 20-25 min suite — and strict does not always serialize (#110/#111 merged 24s apart; mechanism not established). Also measured: no workflow carries a merge_group: trigger, so zero of the 13 required contexts would report on a queue ref, which makes Proposed step 2 a precondition rather than a follow-up. Adds the measurement-cost linkage to #344: supersession-and-re-run produces multi-attempt runs, and the default actions/runs/{id}/jobs endpoint returns only the latest attempt, so failed earlier attempts vanish from any duration table built from it — at exactly the tight end where a margin is decided. #344's own figures are untouched; another session holds that item. * backlog(#340): the protocol cost, and a measurement claim of mine that was imprecise Two amendments from peer review, both of which improve on what I wrote. 1. My measurement-cost paragraph conflated two different prunings. Corrected by the ci-margin-correction session, who hit the same trap from the other side and re-measured with ?filter=all. The precise statement: - filtering on JOB conclusion deletes job-cancelled/step-succeeded rows, which are the tightest by construction; - the default latest-attempt view hides FAILED earlier attempts. It does NOT move a step-success maximum -- so my implication that it changes the margin was wrong. What it hides is that the sample is RIGHT-CENSORED: the largest observable step is the largest that FIT under the cap. Keying on the step's own conclusion (not the job's) and reading ?filter=all are two separate fixes for two separate defects. Still no #344 figure is quoted here. 2. Adds the protocol cost, relayed independently by two sessions and assembled by sandbox-codec. It is the strongest argument in the item and is not a throughput argument: with no queue, sessions invent an ordering ritual, and the ritual is less reliable than the mechanism it replaces. The self-reported instance -- a session promising not to jump the queue while its own PR had auto-merge armed -- is already named as a failure mode in WORKTREES.md, which that session had read about this very freeze hours earlier. Also: re-read the open-PR set nine hours after the first measurement. 15 open, 10 armed, still 0 CLEAN. Membership churns; the condition has never lifted. Removes a "20-25 min" suite duration I restated twice without deriving it -- the What section already states it once, which is where it belongs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Four lines across four files. No behavior changes — docs and workflow comments only;
cla.yml's triggers are untouched and re-verified after the edit.Each is a documented fact that
HEADdisproves.1.
Secure_Development_Standards.md:378— ASVS V3 marked out of scopeThe row reads:
That trigger fired and nobody pulled it. BACKLOG #103 retired the desktop console on 2026-07-13 and made the browser console at
/uithe sole operator UI.The important part: the assessment of record already disagrees with this row. It scores V3 in full — 31 requirements, 18 Pass / 9 Partial / 0 Fail / 4 N/A — and makes no exclusion claim anywhere. So this was never a scope decision awaiting a ruling; it was stale prose contradicting the scorecard.
SECURITY-LOOSENING.md:396had already mappedserve_web_consoleto V3, so the tree contradicted itself.Also corrects the
:394roll-up: 12 → 13 chapters active, and "Documented exclusions: V3, V17" → "Documented exclusion: V17".2.
EARLY-ADOPTER-GUIDE.md:636— "there is no Prometheus exporter"There is one.
messagefoundry/api/metrics.pyis the Prometheus exporter — its module docstring says so.api/app.py:4147registersGET /metricsreturning Prometheus exposition (text/plain), gated bymonitoring:readexactly like/stats.prometheus_clientis a hard module-level import, not an optional extra.This is the one with real cost. An adopter reading that line wires up log parsing and
/statspolling for a capability that already ships./metricsand/metrics/historyare now listed first, with the auth and PHI properties stated.3.
security.yml:16andcla.yml:19— "this private repo"The repo is public (
gh repo view→visibility=PUBLIC).security.ymlclaimed CodeQL needs GitHub Advanced Security here. CodeQL runs and passes as a required context on every PR, which is its own disproof.cla.ymlis the one that matters. It justifies omitting theclosedtrigger as "moot on this private repo", and its own re-add condition is "if the repo ever takes fork PRs from non-allowlisted contributors" — which a public repo can do today. Corrected to say the allowlist, not repo visibility, is what makes the on-close behavior unnecessary. Triggers unchanged: this flags a decision for you, it does not make one.Deliberately left alone
release.yml:84— past-tense history of a step removed at the cutover. Correct as written.release.yml:271,scorecard.yml:9— both already state the repo is public.Secure_Build_Scorecard_MEFOR.md:64— also says "documented V3/V17 exclusions", but it is a dated 2026-07-14 reconciliation record citing a since-superseded assessment. Editing it would revise history rather than fix a claim.Verification
test_security_doc_drift,test_security_static,test_security_posture,test_dast_claims.cla.yml'spull_request_target.typesconfirmed still[opened, synchronize].