Repository navigation
v0.39.0: full-repository audit and its first-tier fixes - #1
Merged
Merged
Conversation
…26-09-27) Read-only audit of v0.38.0 recorded under docs/audits/ and indexed in docs/README.md; no code changed in this pass. Nine parallel line-by-line reviews (tools by area, skills, READMEs, the architecture and evidence record) plus a first-party re-run of every high and medium finding. Headline items, each reproduced in this checkout: - gates that report a measurement from missing evidence: a nonexistent --before baseline reads as zero words (length-growth on every section, exit 1), an unreadable .tex root lints as an empty document (exit 0), and a --git-ref baseline drops \input children deleted from the working tree; - the rewrite fidelity gate hard-rejects LaTeX en-dash ranges (0.5--1.2 reads as -1.2), turns CDM-like into the acronym CDM-, loses Unicode minus and exponent signs, and cannot see a negation moved between clauses; - rewrite_reward exits 2 whenever the optional voice_model.joblib is absent, so the de-ai and condense gates cannot run on a fresh profile; - five tools crash with a TypeError when --field is omitted although the shared help text promises auto-detection; - silent zero findings under a measured status: anchoring has no power at its own 30-document floor, salience and the per-bucket axes skip whole documents that fall into the unknown bucket, and --journals is ignored on the local full-text shortcut; - the quantile grid sits one order statistic high (p10 reads 0.133 at the 30-unit floor); condense_map budgets the body sentence when the abstract comes first and counts placeholders the length gate does not; - the L0 pattern misses paved/showcased and judges paragraph-initial connectors per physical line; a commented \cite raises a blocker; - TIER_WEIGHTS are never applied; final-review closes rounds without the length or residue gates the standard requires; - stale figures across tools/README, DISPOSITIONS and the EVALUATION hub (3.1% / 16.7% / AUC 0.246 / 493 papers / AUC 0.932 / "10 checks"). The record ends with a three-tier action list (fix, document, defer) and the behaviours checked and found sound. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DuasTxTXcSQsC6U2qYvUMM
…that diverged The shared --field option promised auto-detection while five tools divided the profile root by None and crashed, the axis tools never auto-detected, and two tools carried private loops. optional_field_dir resolves it once: an explicit field as given, a single field on its own, several with one stderr note and no profile. axis_main uses it on the read path, requires an existing field for --calibrate (exit 2 otherwise, and exit 2 when the calibrator read no records instead of writing an empty artifact), and grows extra_arguments / check / report_for hooks so an axis flag such as --glossary goes through argparse instead of being stripped from argv. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DuasTxTXcSQsC6U2qYvUMM
… the map's carve-out and counts, and the linter's line-bound rules
Audit items A1-A20, A28, F3, F4, F7, F9, F12, F14, F16-F20, F25, F27,
F28, F33, D6-D11, D14, D16, D19 and M2/M3 (docs/audits/full-review-2026-09-27.md).
tex_assembly / tex_macros / extract_sections
- a root that cannot be read raises instead of reading as an empty
document: the gate no longer scores a mistyped --before as growth in
every section, and the linter no longer exits 0 on a directory named
like a manuscript; only a child may stay in place unresolved;
- a --git-ref baseline resolves its \input children AT THE REF, so a
section file deleted from the working tree stays in the baseline;
- \input{fig.tikz} is read as written, a nested \input resolves against the
document root's directory first, and \includegraphics / \inputminted no
longer read as include calls;
- an argument-taking macro with a numeric body is left alone, and a
commented-out definition is neither harvested nor dropped;
- \[ ... \], $$ ... $$ and subequations are display mathematics in both
projections, so the two views make the same substitutions again.
length_gate / condense_map
- exact percentages (7% of 100 is 7, not 8); --require-shrink 0 means no
cut required, so a map with nothing to cut can still close the pass;
- comments are blanked before segmenting, so a commented-out heading opens
no section; dead front-matter code and duplicate imports removed;
- the abstract/conclusion carve-out applies to either side of a
restatement pair, and a body sentence is the canonical home when one
exists; a copy whose negations or numbers differ in either direction is
not a restatement; every removable-word count is the gate's prose count,
the budget denominator excludes heading words, any \...ref command keeps
a float alive, \caption* and nested captions are read, (Gaussian) is not
an acronym, and a case-folded verbose match cannot raise KeyError.
ai_ism_lint / validate_plugin
- paragraph-initial connectors are judged on a paragraph's first prose
line, openers where a sentence starts; paved/showcased join Tier A;
every em-dash on a line is a target with its own id; \label/\ref/\cite/
\url arguments are blanked before the lexical scan; three-parallel needs
whole words; an unparseable lexicon is exit 2; Markdown front matter is
metadata; several profiles without --field say so on stderr; the CLI is
built on cli_common and --summary is hidden;
- the validator's shape regex now reads "N checks" and scans tools/README,
and a relative-link pass reports targets that do not exist (it found the
16 local-path links in the 2026-09-04 audit note, now repository-relative).
extract_style / retrieve_exemplars
- multi-word openers can be observed; placeholders are not words and do
not drop a citation-led paragraph from the bank; words are Unicode
letters; bank ids carry the bucket; the profile directory is created
after the corpus check; skipped PDFs are counted in the summary; an
empty --topic ranks by section name in the keyword fallback.
Tests: 38 new (the extractor's moved to tests/test_extractor_statistics.py,
the original file being at the line budget); the suite is 567 tests in
28 files, recorded counts to be updated with the release notes.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DuasTxTXcSQsC6U2qYvUMM
Twelve defects from docs/audits/full-review-2026-09-27.md §3.4, each with a
regression test, no other file touched.
- F1: a `\cite` inside a `%` comment raised reference-missing-entry (exit 1).
cite_lines strips comments with tex_assembly.RE_TEX_COMMENT first.
- F2/F32: http.client.HTTPException (IncompleteRead, BadStatusLine) is not an
OSError and escaped fetch as a traceback, i.e. exit 1 read as a blocker
verdict. fetch now catches (HTTPException, OSError); the redundant
URLError/TimeoutError members are gone, and main ends in one deliberately
broad `except Exception -> 2` as ai_ism_lint does, printing the type.
- F5: _parse_fields is a piece grammar: `#` concatenation, quoted values
scanned by brace depth with `\"` allowed, and text the grammar cannot
place raises ValueError (named with entry key and line) instead of
silently dropping every later field, DOI included.
- F6: a registry miss is never cached and a legacy `null` in the cache file
is dropped on load, so a late-indexed DOI is re-queried each round;
metadata-mismatch findings carry `observed.cached`.
- F8: _first() reads list-valued CrossRef/DataCite fields defensively
(`date-parts: [[]]` was an IndexError); a 200 body that is not the
expected JSON/Atom shape is RegistryUnavailable, so the entry is
unmeasured and the axis degraded rather than the run crashing.
- F10: first-author agreement is equality once normalized or a shared name
part of four or more letters; `Li`/`Lin`, `Ma`/`Mandelbaum`, `de Jong`/
`de Vries` are now reported while `van Waerbeke`/`Waerbeke` still agree.
- F11: RE_CITE recognises \nocite and biblatex \parencite, \textcite,
\autocite, \smartcite, \footcite, \supercite, \fullcite, capitalised
forms, every key group of the multicite \cites{a}{b}, and \nocite{*} as
citing every entry; natbib's \citetext{prose} no longer yields a key.
- F13: @string definitions are collected (in order, with `#`) and bare
field values expanded before comparison.
- F21: a missing-entry finding reports the citing command's line (in the
assembled document) instead of 1.
- F23: author lists split on ` and ` case-insensitively.
- F24: a DOI carried only in the `url` field (doi.org / dx.doi.org, URL-
decoded) is an identifier.
- F22 (retry/mailto) is left as a named known gap in fetch's docstring.
Exit contract unchanged (0 no blocker / 1 blocker / 2 invalid input or any
execution failure). Output changes a caller can see: `observed.cached` on
metadata-mismatch findings, the lookup-failed message now reads "the
registry gave no usable answer", and `--cache` files no longer hold nulls.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DuasTxTXcSQsC6U2qYvUMM
(cherry picked from commit d93176db349df170405cdbdfa5690de5ba9692b5)
Audit tier two (docs/audits/full-review-2026-09-27.md §6 and §8.2): - skills: the condense baseline is the whole include tree or a git ref, a target of 0 closes, the duplicate floor is 30 prose words, and the rewrite gate no longer needs the optional learned model; de-ai rebuilds every calibrated axis when the corpus changes, drops the no-op --summary, states the three zero-hit exemptions the tool applies, snapshots the document and runs the §5.3 loop-close gates before it stops; final-review verifies the bibliography before each round, snapshots the target, closes every round with the length and residue gates and lists them in its stable-round criterion; paper-review §O carries residue-absence and the delete-first ruling; the paper skill's Tier A table and grep agree with the linter (paved, showcased, connectors per paragraph, the cap per heading unit); - the standard is v3.9: the fourth narrow exit contract, verify_references; the cap is a section cap, step 5 is L2 anchoring and L4 voice; - RESPONSIBILITIES gains the residue-absence rule, the three zero-hit exemptions, the linter's real default axes, and rows for verify_references and label_findings; DISPOSITIONS quotes the v0.36.2 and v0.36.3 figures (3.37 per 1,000 / AUC 0.174, 1.59%, 41,644 passages, 530,504 pairs, 10.78% / 7.80%), counts four static residue rules, retitles the register row, and records the spaced-unit limit; the EVALUATION hub's L3 row is the 2026-09-05 retrain, its document-shape row dates the 493-paper scoring against the 507-document artifact, cohesion and hedging quote the v0.36.3 sizes, the v0.28.0 release gate and the two "open" decisions are closed; DEAI_SUBSYSTEM's paper-as-agent example is one the detector flags, its exemptions, pair breaks, weight names, float exclusion, exit-contract count, policy-asset and L1 wording match the code, and its validator list is complete; - README: 11 checks, 40 tools in one schema, the worked-example table reads the revision at 3 as examples/README does, the retrain row quotes the current stability figures, demo 2's command reproduces its counts, the paper-review row lists dimension P; README.zh-CN: the refuted estimator-noise model, the demo-2 clause, the bank sizes; - style-corpus: tier weights are recorded, not applied; the extractor warns below five files; the breadth and held-out directories are in the layout; style-profile: the discourse and anchoring calibrations and their artifacts, the negatives file the harvester writes, the rebuild notice; examples: the headline pair is restated in the abstract, dated to the v0.36.3 run; docs/README: the link check and the worked example; - tools/README: rows for length_gate, condense_map, ai_ism_lint, verify_references (its agent's text), cli_common, extract_style, tex_assembly, a new tex_macros row, deai_residue (13.3%), deai_register (AUC 0.174, three exemptions) and validate_plugin (11 checks); - the 2026-09-04 audit note's sixteen links are repository-relative. Recorded test counts are updated with the release notes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DuasTxTXcSQsC6U2qYvUMM
…eshold Audit 2026-09-27, B4/B6/B7/B8/B9/B12/B14/B15 and the hygiene rows B19/B20/B21/B23/B28 for docshape, docstructure, metrics, structure and features. - document_shape sweeps deai_metrics.section_units and drops the preamble and every `skip` unit on both the calibration and the detection side; acknowledgements, appendices and reference lists entered dispersion, role coupling and section arc as body prose (B12). The sweep and the substance filter are shared as shape_units/measurable_sections so deai_partition can score the same geometry (B11 lands with partition). - The document-shape strong flag uses the interpolated quantile the baseline quotes as strong_threshold and the LOO rate is computed with, not the add-one rank percentile: a document could be reported strong while quoting a threshold its own value was below (B9). - --strong-percentile must lie strictly in (0, 1); 1.5 was an IndexError with exit 1 (B8). --calibrate resolves its field through resolve_field and the read path through optional_field_dir; a missing --field was a TypeError traceback (B1). - A structure template finding is measured only against a reference for its own bucket plus a policy, degraded with a baseline that lacks the bucket, unmeasured without one; an `unknown` finding was measured with `templated_fraction: None` (B6). The auxiliary message states the reference fraction and calls a figure rare only below the policy gate (B14). RE_TRICOLON_WRAP requires a sentence-initial "These N <nouns>" closed by a summing verb; "represent these two quantities" counted, and entered template_score and the manifold (B15). - The distribution axis is degraded when sentence_stats.json holds no classified bucket, where pooled_cv is 0 and burstiness can never fire (B7). - deai_features checks the .npy before importing numpy, gates the UID block on deai_oracle.model_runtime_available() (None entries with a uid_status/uid_reason marker; features_vector raises with the reason), and the CLI prints the reason and returns 2 instead of a ModuleNotFoundError (B4). - Dead code: the callerless distribution_hits/structure_hits adapters (B19), docstructure's shadowed constants and unused imports (B20), the unread REPO_ROOT/DEFAULT_PROFILE_ROOT/argparse (B21); deai_metrics.main resolves the field through optional_field_dir and deai_structure reads the policy through deai_metrics.load_policy(section=...) (B23). The re-export contract test excludes module objects, which is what had made the dead imports "required". The misplaced __main__ block in test_deai_docstructure moves to the end (B28). The structure and docstructure baselines must be rebuilt: B12 and B15 change the values every document contributes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DuasTxTXcSQsC6U2qYvUMM (cherry picked from commit ba62d8eafd4efd4232de0c1d95668c342f14aeac)
…t, fix every heading Audit 2026-09-27, B1/B2/B3/B10/B11. - Candidate states are compared by their split-conformal p (higher = deeper inside the human band); raw manifold distance is only a tie-break between states scored on one calibration basis. The greedy step compared raw distances across a length-stratum switch, where the two manifolds' units differ, and stopped on p while advancing on distance. Every plan step records calibration_basis before and after, and the CLI marks a switch (B10). - A split is simulated on the original block text at the sentence boundary's own offset, line breaks kept. Re-joining es.sentences with spaces turned newlines into spaces, so a trailing `% note` commented out the rest of the paragraph and the ranking scored a half that had lost its prose (B2). The split halves now carry their real line ranges. - A block starting with any heading command (extract_sections.RE_HEADING_COMMAND) is fixed, not only `\section` (B3). - The measurable blocks are the ones document_shape scores, through the shared shape_units/measurable_sections filter (word floor, paragraphs per section, sections per document), so the starting distance is the one deai_docstructure reports for the same file (B11). - The field resolves through optional_field_dir; no profile or no manifold is exit 2 with a message naming the profile, and a missing file is exit 2, where a missing --field was a TypeError traceback (B1). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DuasTxTXcSQsC6U2qYvUMM (cherry picked from commit 940b651455c86e887f157252377ee51c12f2ddc9)
…orted, not measured Audit 2026-09-27, B1/B5/B13. - A conformal p is never below 1/(n+1). With n = 30 per class and k >= 2 classes measured, 1/31 = 0.032 exceeds alpha/k = 0.025, so no document could ever flag and the axis still reported `measured`. Detection now computes which classes the baseline can test (class_tests): an underpowered class is skipped and the axis is `degraded` with the reason; a document whose classes have no reference at all is `unmeasured`. The share is not re-spread over the remaining classes, which keeps the document-level rate <= alpha. Calibration records min_class_documents_for_share and underpowered_classes in the baseline and the CLI prints them as a warning (B5). - The sentence scan runs on the section units every per-bucket axis reads, preamble and `skip` units dropped, headings and floats blanked through deai_reference.without_headings and the bibliography blanked by a local pattern (deai_reference holds none and is not edited). On the raw LaTeX a heading fused with the first sentence below it and a thebibliography block anchored the last sentence with every year in the reference list (B13). - --calibrate resolves its field through resolve_field, the read path through optional_field_dir; a missing --field was a TypeError traceback (B1). The anchoring baseline must be rebuilt: its rates were measured on the raw text. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DuasTxTXcSQsC6U2qYvUMM (cherry picked from commit 8932ee2be319d8ac92bac66e285cf205fd9d7377)
…he small skill and record fixes - tools/README and DEAI_SUBSYSTEM describe the document-level axes as they now behave: an axis that cannot fire is degraded (metrics, structure, anchoring's Bonferroni share), one quantile rule per document-shape threshold, skip units dropped from the shape sweep, partition ranked by conformal p on the original text, the wrap-up beat, the UID features unmeasured without the runtime; - skills: the paper skill marks its ML and detection guidance [WGL] and targets the field's journals; calibrate is "not a policy source"; the condense and calibrate hints list only flags their bodies define; brainstorm no longer narrates removed flags; the exemplar retrieval step names --allow-fallback; the negative-label floor is stated where the residue gate is run; the proposal skill dispositions colon hits on run-ins; physics and figure-review share one build recipe; - the two design notes point at DISPOSITIONS instead of claiming status; - evaluation parts quote the v0.36.3 register sizes and floor multiples, the UID spread, four retrains, the closed estimator-noise entry, the v0.36.2 collocation bank and the v0.32.0 structure counts as such, and the figure-pinning test without a stale count; - the eval_findings rows name the axes the tool scores; - README.zh-CN reproduces demo 2 with the same flags as README.md, gains the worked-example section, and its roadmap is the English one. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DuasTxTXcSQsC6U2qYvUMM
…ated Version strings, the two versioned document headers and the CHANGELOG entry for the audit's first tier as landed so far (assembly, projection, gate, map, linter, validator, extractor, bibliography checker, document-level axes). The entry's test count and the remaining parts follow with their commits. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DuasTxTXcSQsC6U2qYvUMM
… counts what it compares
Audit 2026-09-27 §3.5 (E1, E2, E3, E4, E12, E30, H1, E9, E32).
The deterministic gate was hostage to two optional artifacts and misread
several ordinary LaTeX forms:
- H1/E9: a missing voice_model.joblib was exit 2, so a fresh clone or an
untrained profile could not run the gate the de-ai and condense skills
call mandatory. The field now resolves through cli_common.optional_field_dir
(one field auto-detects; none or several means no profile); a missing or
unusable bundle leaves the voice term unmeasured at weight 0 with one
stderr line naming which, and the exit status is 0/1 by eligibility. The
same rule is applied to the semantic-fidelity cosine when
sentence-transformers is absent: unmeasured, weight 0, floor not applied,
said on stderr. Exit 2 remains invalid input and crashes.
- E1/E3: the numeral tokenizer read the LaTeX en-dash range 0.5--1.2 as
{0.5, -1.2}, dropped the Unicode minus and the sign of an exponent, and
started tokens inside identifiers (M200 -> "00", M200c -> unit "c"). A sign
may not follow a sign character; U+2212 is a sign and folds to "-"; an
exponent is part of the number; a token starts after no letter, digit or
dot, with a leading-dot decimal kept as a form of its own.
- E2/E4: CDM-like no longer yields the acronym "CDM-"; projection
placeholders are stripped before the acronym scan; 3rd/1990s/5x yield no
unit; numerals inside inline math are collected so display and inline
forms agree (the math category already protects both).
- E12: numbers, negation, comparison and causal markers are compared as
bags, so a dropped second negation or a repeated value replaced by another
is caught; the report names count mismatches, and the residual limit (a
marker moved between clauses) is stated in the module and the report.
- E30: ldots/dots/citeyearpar/citetext (and rm/mathrm, the math-mode twins
of textrm) are formatting; {\rm Mpc} is protected as \mathrm{Mpc} when the
vocabulary lists the unit; an unusable bundle is named as present.
- E32: the exit-contract tests now reach the crash guard and cover 0/1/2.
33 tests added or rewritten in tests/test_rewrite_reward.py (562 in 27 files).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DuasTxTXcSQsC6U2qYvUMM
(cherry picked from commit 8e0371caced5ce20afce54ceb0e470ae08533877)
… is a draw Audit 2026-09-27 (docs/audits/full-review-2026-09-27.md), rows E5, E6, E7, E10, E11, E13, E14, E15, E18, E19, E28 and E33. eval_findings: a population of twenty documents with no body words was `measured` with a None density that `render` formatted with `.3f`; now `unmeasured` with the reason, and `_cell` prints a dash for a missing density. `median_words` is `statistics.median` (the upper middle element read 100/200/300/400 as 300). `MIN_DOCUMENTS` is imported from eval_docscale so both evaluators share one floor. eval_docscale: an empty human corpus reached `statistics.median([])` as a StatisticsError; now a SystemExit naming the empty corpus and the calibrate command. Every row is `unmeasured` under the shared `MIN_DOCUMENTS` floor (one document once printed a flag rate and an AUC), a flag is decided by the operating point's own alpha (a legacy in-sample-percentile point keeps its threshold decision instead of having its percentile read against the conformal alpha), each row names its operating point, tiers are discovered under docval/ the way eval_findings does, and the field resolves under this tool's name rather than through retrieve_exemplars' wrapper. label_findings: the field options sat on the root parser and every subparser, and argparse let the subparser defaults overwrite the root's parse, so `--field wgl --corpus-root X sample` ran with field=None silently; the root is `base_parser` only and that spelling is an error. `--n` is the per-(population, axis) quota with help text (the old formula made it a dead flag below 200 rows per population). Every finding of every document is collected first and each cell is drawn at random with a per-document cap (`_spread_sample`), the summary prints how many documents each cell came from, and the emitters run once per document for flags and controls alike. An emitter failure is counted per axis and printed with the shortfall line instead of reading as "the corpus holds no more". Pooled recall counts a passage several axes flagged once, by (source, start_line, end_line), and the printed line says so. `score --recheck <missing file>` exits naming the file. tests: test_label_findings.py's `unittest.main()` block moves to the end (it ran 6 of 16 tests when invoked directly). tests/test_eval_docscale.py is new; 33 tests added in all. The recorded suite size is for the integrator. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DuasTxTXcSQsC6U2qYvUMM (cherry picked from commit 8927937994ff23fb82f751d19268e9a8582d08f1)
…k that missed it is scope-aware
Audit 2026-09-27 (docs/audits/full-review-2026-09-27.md), rows E8, E16,
E21, E22, E31 and M5.
voice_dataset: `embedder_available()` probes the import, and that is what
the feature-cache fingerprint records; when the encode itself failed
(offline download, device fault) the rows carried corpus_cos = 0.0 and
were cached under the healthy fingerprint, then served as measured to
every later run. `embeddings_failed_at_run_time` names that state; the
cache is withheld and the finished language-model rows are written to the
resumable checkpoint with n_done = every row, so the next run retries the
embedder without refeaturizing, and a cache already holding empty
embeddings under a healthy fingerprint is recomputed rather than reused.
`_load_jsonl` skips blank lines (json.loads("") took the run down; the
legacy loader tolerated them).
voice_audit: the repeated grouped audit started at the primary split's
own seed, so twenty repeats were nineteen plus the split already
reported; `audit_split_seed` starts one past the seed the primary
actually used, which the trainer now passes.
train_voice_model: the field resolves through cli_common.resolve_field
(`profile_root / None` was a TypeError traceback), and a missing numpy /
scikit-learn / joblib is one stderr line and exit 2 instead of a
traceback.
E22 residue removed: hashlib, math, re, statistics, defaultdict, argparse
and the DEFAULT_PROFILE_ROOT copies in train_voice_model; hashlib, json,
re, Counter in voice_audit; the function-local `import re` and the
DEFAULT_PROFILE_ROOT copy in voice_dataset; argparse,
DEFAULT_PROFILE_ROOT and the never-called `list_fields` in
train_ai_ism_classifier. The re-export contract gains the two new names.
tests: the unbound-name check pooled every binding in the module, so a
function-local `import numpy as np` anywhere let a bare `np` pass in every
other function; it now reads CPython's symbol tables per scope, with
fixtures for the leaked-import case, the original time/CHECKPOINT_EVERY
case, closures, `global` bindings and a class attribute read from a
method. A numpy stand-in drives `build_features` through the failed
encode, the resumed run, and the stale cache. An unused-import check
covers the four files.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DuasTxTXcSQsC6U2qYvUMM
(cherry picked from commit 9ede3c3a390631406e10b8c75ad8a8f83f855434)
…d the query sets are [WGL] Audit 2026-09-27, D1-D5, D12, D13, D24, D26. `--journals` applies on the local full-text shortcut, which had built the "refereed" breadth corpus without any journal restriction. `--start-at` sizes each page from where the band began instead of requesting a negative page. A throttled full-text run reports TRUNCATED and exits 2 like the abstract sweep, instead of DONE and exit 0. `--updated-before` applies under `--fulltext`, and `--query-set`, `--out-name` and `--resume` are refused there rather than accepted and ignored. Candidate dedup and the output directory key on the bare arXiv id, so a version bump no longer fetches `...v2` beside `...v1`. A `\documentstyle` or comment-led file is a LaTeX source, and colliding tarball basenames are both kept. Omitting `--field` resolves through cli_common and is a message and exit 2, never `Path / None`. The built-in query sets, keyword filter and journal table are marked [WGL]; `--query` replaces them for any other field. The tests outgrew the 750-line budget and are split by subject: the parser, backoff, vintage, resume and held-out tests stay in test_fetch_arxiv_abstracts, the sweep paths are in test_fetch_arxiv_sweeps, and the offline harness both use is _fetch_harness. The `__main__` block sits at the end of each file, so a direct run executes every class. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DuasTxTXcSQsC6U2qYvUMM
…ling floor and the fetcher's flags The tools registry says what rewrite_reward, label_findings, eval_findings, eval_docscale, voice_dataset, train_voice_model and fetch_arxiv_abstracts do now: bag comparison of numbers and markers, no exit 2 for a missing optional artifact, the per-cell labelling quota and its document spread, the shared 20-document floor and the operating point named per row, the encoder failure that never becomes the cache, the scope-aware unbound-name check, and the fetcher's flags holding in both modes. The de-ai and condense skills carry the gate's residual limit and its optional `--field`; the calibrate skill samples `--n 24` per cell, explains the document spread, and warns that the fetcher's built-in query sets are [WGL]. The learned-model note records the two new protected-set properties and the tests that pin them; the responsibilities table gives eval_docscale its floor and alpha rule. The classifier's closing hint no longer names the linter's hidden `--summary` flag. The release notes gain the three paragraphs for these parts. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DuasTxTXcSQsC6U2qYvUMM
…nd a status that names what it skipped Audit items C1-C19, C21 and C22 of docs/audits/full-review-2026-09-27.md (C20 concerns deai_structure and is not touched here; the deai_voice part of C1 and C18 follows in the next commit). deai_reference - quantiles() stores the ceil(q * n)-th smallest value (nearest rank). The floor(q * n) it stored before sat one order statistic high: p90 of 100 values was the 91st smallest, and the 30-unit floor read its tails at P = 0.133 and 0.933. An artifact calibrated before this keeps the old grid until it is recalibrated. (C12) - usable_buckets() takes the unit the axis measures and refuses a bucket whose artifact records another one; unit_reason() names the refused buckets in the status. sections() pools a bucket's spans into one unit, as the reference side pools a paper's paragraphs per bucket. (C11) - unbucketed_reason() counts a document's units in the `unknown` bucket so a status can say how much of the document went unmeasured. (C13) - sentence_lines() locates a projected sentence in its raw unit. (C15) - calibrate() records text_key and text_fallback_rows per bucket and returns None, writing nothing, when no record contributed. (C8, C17) - the unreachable v_high == v_low branch and two stale docstrings. (C18, C19) deai_discourse / deai_collocation / deai_salience - each status function takes the text and reports degraded, with counts, when units sit in `unknown`, when a bucket records another unit, and (salience) when no bucket resolves above the gate: the floor alone read `measured` with zero findings on a flat reference. (C10, C11, C13) - collocation: --glossary goes through argparse (extra_arguments), is refused beside --calibrate (check) and picks its report (report_for); the definition cue is read only in the words after the pair; a pair joined twice in a sentence counts twice; the glossary status depends on the bank alone; sentence findings carry the sentence's own lines; calibrating with no passage writes nothing and exits 2. (C8, C14, C15, C21) - salience: a bank row without numeral_text is counted per bucket and warned about on stderr instead of calibrating silently on the [math] projection; the test that pinned the silent p90 of 0.0 now requires the warning. The finding path checks the unit through the shared helper. (C17, C22) - discourse: the degraded branch a finding could not take is gone. (C18) deai_register / deai_residue - \newcommand* is read. (C9) - body_lines() and body_only() take drop_skip; the edit-meta scan keeps the skip sections (an appendix TODO was invisible and the gate exited 0), and the label rule looks for a caption's object in paper_prose(), the whole paper minus preamble and bibliography. (C3, C5a) - RE_LABEL allows a space before the brace, as extract_sections does. (C4) - stem() reduces the -ation/-ated/-ating/-ate(s) family to one root. (C5b) - an unwritable --output is exit 2, not the gate's exit 1. (C6) - calibrate() on no passage returns None and writes no lexicon. (C8) - RE_BIBITEM, the saturated confidence formula, the stale seam comment, the AUC figure and the defined_terms direction. (C18, C19) deai_oracle / deai_provenance / deai_personal - the CUDA fallback handler survives an empty message; calibrate skips a blank or undecodable bank line, and a --field with no bank is exit 2 rather than a traceback. (C7) - --field resolves through cli_common.optional_field_dir. (C1, C21) - provenance reads reference.paragraphs(), so a title block is no ledger paragraph, and the text output no longer rebuilds the ledger. (C16, C18) - unused argparse imports and profile-root constants removed. (C18) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DuasTxTXcSQsC6U2qYvUMM (cherry picked from commit ee5303ff92a4fc53009888cedea9702100ab4bd0)
…and --field resolves like every other tool deai_features.features_vector now raises when the optional transformers and torch runtime is missing, since three of the bundle's features are token surprisal. voice_score, voice_axis_status and voice_findings probe deai_oracle.model_runtime_available() first: the score is None, the findings are empty, and the axis reports `unmeasured` with the runtime's reason, so the linter, the rewrite ranker and --scores see a status instead of a RuntimeError. --scores prints that status line and exits 0. Also here, audit C1 and C18 of docs/audits/full-review-2026-09-27.md: --field resolves through cli_common.optional_field_dir, so a missing --field is the tool's exit 2 rather than a TypeError; --scores lists paragraphs by source line from the shared sweep, so a score can be matched to a finding; unused imports and the profile-root constants are removed. tests/test_deai_voice.py is new and takes over the field-resolution case that sat in tests/test_deai_oracle.py. tests/test_train_voice_model.py pins the runtime as present in its bundle-status test: it asserted the bundle's status on a machine without torch, where the axis is now `unmeasured` whatever the bundle says. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DuasTxTXcSQsC6U2qYvUMM (cherry picked from commit 0a50a3c0434e0b577af8442dde732fc503decd46)
…he audit's landing record The v0.39.0 entry gains its paragraph for the per-bucket axes and closes with the suite as it stands (763 tests in 32 files; the validator's 11 checks pass). The recorded counts in both READMEs and the evaluation record are refreshed; the latency rows no longer quote the size of the suite they were timed on, which the registry check read as a claim. The audit note ends with a landing record: which items each commit covers, what §8.3 leaves open, and the baselines and artifacts that must be rebuilt or recalibrated before the EVALUATION figures move. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DuasTxTXcSQsC6U2qYvUMM
The voice commit cherry-picked as 3e13594 was amended once more in its worktree to add the test that reads the --scores lines by source line; this is that addition, so the branch carries the agent's final version. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DuasTxTXcSQsC6U2qYvUMM
…rt, and the records say what moved ai_ism_lint passes the text to the salience and discourse status functions and reads the glossary status from the bank alone, so a document whose units sit in `unknown`, a bucket recorded at another unit, and a glossary-only run report what the axes now know instead of a profile-only status. The tools registry and the responsibilities table describe the per-bucket axes as rebuilt: the grid at nearest rank, the unit read back and refused by name, a section pooled per bucket, the status that counts what it skipped, `--glossary` as an argparse option with its cue after the pair, residue reading the whole paper minus preamble and bibliography, the oracle's exit on a field with no bank, and the voice axis unmeasured without the runtime. The evaluation hub marks the salience, collocation, cohesion and hedging rates as measured on the earlier grid, and the collocation page records the 30–400-word bank band and the new artifact keys. The residue section of the subsystem note counts four static rules and one diff rule. The audit's landing record and the recorded suite size (764 tests in 32 files) are brought up to date. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DuasTxTXcSQsC6U2qYvUMM
…sions, invented examples Audit 2026-09-27 section 8.3, each item settled (the audit's section 10 records it): - F26: `robust` directly before estimator(s) / estimation / statistic(s) / regression is a method name and no longer counts toward the Tier B cap, across a line break too; `robust estimate` still counts. The paper skill says so beside its Tier B rule. - B17: the Mondrian strata edges are fitted on the training records only, as exchangeability requires; the calibration records no longer place them. - B18: a docstructure feature whose baseline cannot resolve its own tail (fewer than ceil(1 / min(p, 1 - p)) documents, 20 at 0.05 / 0.95) reports degraded and ordinary instead of strong. - E17: the paired leakage estimate counts the paper as the bank holds it: the paragraphs extract_style.bank_paragraphs admits (30-400 words, the unknown bucket excluded) and the terms deai_register.passage_terms reads, overlaid on the df table, so `halo` no longer matches inside `halos`. - E20: corpus_cos is a 0.0 placeholder without the embedder or the corpus centroid; a bundle that reads it now scores unmeasured (deai_voice.scoring_reason), and deai_features builds the feature only when both exist and says so on stderr otherwise. - B23: deai_features' CLI walks the shared paragraph sweep, placeholders are stripped by the shared helper, the second `_MATH_MARKER_RE` is renamed for what it measures, and anchoring classes each paragraph by the shared bucket instead of a third section classifier. - F22: CrossRef requests carry a polite-pool mailto from SCI_PAPER_MAILTO and retry once on 429 / 5xx, honouring a capped Retry-After. - D20: the exemplar embedding cache carries a sidecar fingerprint (bank sha1, row count, model) and is rebuilt when it disagrees. - B16: closed with its reason beside DEFAULT_BURSTINESS_RATIO. Found while reviewing the PR: - extract_style.words() split hyphenated compounds and apostrophes after the tokenizer change; a compound is one word again. - unit_reason named foreign-unit buckets the document never reaches and the axis never reads; it is scoped to both. - tex_macros read definitions from commented-out lines, so a commented \newcommand could beat the live one; definitions now come from the comment-blanked copy and are blanked at equal length in both copies. Examples in docstrings, comments and test fixtures that had been taken from a manuscript under review are replaced by invented ones. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…view The evaluation record, the changelogs, the standard, both READMEs, three skills and two audit records quoted a manuscript under review: a codename, phrases a reviewer marked, sentences from an absence sweep, values from an abstract, a macro, and words a register finding named. Each is replaced by an invented example of the same kind, or by the count the tool reported; no measurement changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… fact, the L0 mirror checked Audit 2026-09-27, the items the first tier and d42ef14 left open: - B22: deai_docshape.shape_units blanks headings and floats before the paragraph cut (a heading fused into its section's first block; a float holding a blank line split its paragraph in two). deai_partition keeps what the blanking took out as fixed, unscored blocks. - B24: the in-sample percentile fallback for a manifold without split-conformal calibration is gone from deai_docshape, deai_docstructure, deai_partition and eval_docscale. The axis reports degraded with a rebuild reason (deai_docstructure.uncalibrated_manifold_reason), and deai_partition prints that reason with status unmeasured instead of "not measurable". - Hedging runs on `intro` and `method` (deai_discourse AXES), as the 2026-09-27 re-measurement of separation and transfer supports. - H8: ai_ism_lint.validator_check holds the paper skill's Tier A table, opener row and Tier B sentence to the four L0 patterns in both directions; validate_plugin's skills check runs it (still 11 checks), and test discovery runs once per validator process (F29). - E23: extract_sections.bundle_documents is the one bundle loader both evaluators read. E24: register findings once per document, math density once per audit, one cross_validate pass, and the linter measures the document shape and the anchoring once per run. E25: voice_audit uses eval_docscale.rank_auc. E26: voice_audit says what it fits. E29: train_voice_model re-exports no private helper. - A22-A25: aliases that only forwarded are removed, condense_map and deai_residue take cli_common.report_options, and the tests share one LaTeX bundle writer (tests/_texbundle.py). - D17/D18: the dossier's no-sections hint joins the `unknown` table, and its header names the source files the run skipped. D21/D23/D25: unused constants and imports go, VALID_SECTIONS is CLASSIFIED_BUCKETS, and extract_md_negatives drops a local path and states that it overwrites. - The projection's placeholder vocabulary has one owner (PLAIN_INLINE, PLACEHOLDER_SHAPE); the arXiv API is queried over HTTPS with one user agent. - Tests: B30 (the conformal path end to end), D27 (a PDF through a stand-in pymupdf), D28/D29, F30/F31/F37/F38, and one for each behaviour above. 801 tests pass; the validator's 11 checks pass once the recorded counts follow in the documentation commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e audit's open skill items H14 (the author's decision: the skills follow the standard): standard §4 makes an advisory strong only when a measured effect exceeds a calibrated operating point. De-AI's Pass-2 density hits, paper-review's structural hits, physics' naming and unit checks and mainline's narrative findings have no such point; each is now an ordinary advisory ranked first. The standard is unchanged. The skill items of audit 2026-09-27 that the verification after the first tier found still open: - H15: an unlabelled schematic formula is a ranked-first advisory only when its dimensions close; otherwise P1 makes it an integrity blocker. - H16: proposal-polish proposes a firmer central hypothesis to the author and never applies it silently (§6 keeps stance and modality). - H17: mainline asks an eighth question, the one thesis line every contribution branch serves (standard §5.4). - H18/H22: under --orchestrated, paper-review also skips the dimension-D de-ai audit and the dimension-G figure-review call, which the parent runs; the partition suggestions come from de-ai Pass 3, not from --audit-only. - H37: paper-review's M.1 runs physics P1-P6 for its first and third passes instead of restating them, and dimension K says how it runs alone and under --orchestrated. - H39/H40: argument hints list only defined flags; brainstorm drops the no-op --min-frameworks, and names cc-enforcer, FACTS.md and cc-tree as optional. - H42: the residue gate joins the stopping semantics. - H32/H33: narration of removed flags and dated source notes leave the skills; the attribution and adoption boundaries live in ACKNOWLEDGMENTS.md. - F15: the Tier B cap is per heading unit, as the linter counts it. - H45-H51: repetition inside a skill is cut to one statement and a pointer. Repetition across skills stays (the author's decision), because each skill is loaded alone and does not follow pointers. - H21 needs no change: brainstorm's NEEDS-MORE-INFO is a provisional verdict that convergence forbids, which is what paper-review requires. Also: the paper skill's review grep finds every form the linter does (`delving`; `Moreover` mid-sentence), and calibrate's document-structure row says what an uncalibrated manifold does since B24. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s document items The `wgl` profile was rebuilt on the corpus-side fixes of this release, and each figure below was re-taken through the shipped path rather than carried forward: - both READMEs' asset rows: the exemplar bank (28,444 paragraphs from 18 of the 19 curated and 498 of the 500 reference papers), register passages and terms, pooled UID, the structure and salience bucket counts, the document-structure baseline (504 documents, strata edges 49 and 78) and the collocation bank (535,517 pairs); - the document-scale table and a new §9 notice. The 2026-08-25 notice, whose tables repeated §9.1-§9.8, gives way to it (R6); four changes between the two measurements are named and not separated; - the L3 retrain (45,153 records; §7.0 retrain equivalence), the held-out register row under the E17 estimator, hedging's §19 re-measurement, and the worked example in examples/README.md. Audit 2026-09-27 document items: G4, G6, G12, G28, G29, G30, G31, G34, H34, H35, E27, F15, I26-I29, I37, M3/I35 (the 2026-09-04 review's per-item dispositions), B24 and B25 where the docs described them, the L0 mirror in tools/README and the standard (H8), and R1-R10: each measurement keeps one home and the other places point to it. The suite size reads 801 tests in 32 files. CHANGELOG gains the audit remainder and the rebuild; v0.35.0-v0.35.1 move to CHANGELOG-ARCHIVE-v0.35.md because the live file had reached 806 lines. latency.json, which nothing read, is removed (M4/G19). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Section 10 of docs/audits/full-review-2026-09-27.md now covers all 318 items. The PR's own rows stay, with E32 in the rewrite_reward row and the two "this section's commit" cells naming the PR commits they meant. New rows list the items the PR fixed without naming them (each re-checked before merge), d42ef14's follow-ups including C20, the invented examples of bbd355a, and the code, skill and document commits of this merge. G26, M6, H21 and H23-H30 are closed with their reasons. The "artifacts to rebuild" paragraph becomes the record of the 2026-09-27 rebuild and re-take, and two measured properties of the held-out set are noted: its 200 bundles read as 203 documents, and 15 of them have an odd number of unescaped `$` once comments are removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e-merge review of 6aaf9f3) Since B22 kept headings and floats as landmark blocks, suggest() took the median overlap over every adjacent block pair, landmarks included. Each section heading and each float between paragraphs added near-zero pairs, so the floor fell: 0.0 on the new test's three-section document, against 0.078 over its paragraph pairs, which admits every merge and only splits whose halves share no content word. The floor now reads consecutive prose blocks per section, the blocks _state_statistics scores. Test: the floor equals the median over the document's paragraph pairs, with and without a float between two of them; it failed on the previous code. Suite 802; the recorded counts and the audit record's section 10 follow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A read-only audit of v0.38.0 (
docs/audits/full-review-2026-09-27.md: defects, logic, redundancy, drift) followed by the fixes for its first tier and the documentation corrections that go with them. 20 commits, 103 files; the release is v0.39.0 and the standard is v3.9 (a fourth narrow exit contract,verify_references.py). No consequence class changed, no paper PASS/FAIL or authorship claim was added, and missing dependencies or calibration still reportunmeasured/degraded.Gates and projection
length_gate --before <mistyped path>no longer reports growth in every section,ai_ism_lint dir.texno longer exits 0 clean, and a--git-refbaseline resolves its\inputchildren at the ref.\[ ... \],$$ ... $$andsubequationsare display mathematics in both projections; a macro definition inside a comment is not harvested.condense_mapcounts what the gate counts (carve-out on either side of a restatement pair, exact percentages,--require-shrink 0, placeholders and headings out of the counts).ai_ism_lintjudges paragraph-initial connectors per paragraph, addspaved/showcased, ids every em-dash, and blanks\label/\ref/\cite/\urlarguments.One field resolver.
cli_common.optional_field_dirandaxis_mainhooks replace five TypeError crashes on a missing--fieldand two private loops;--calibrateneeds an existing field and never writes an empty artifact.Axes report what they can measure
degraded; document shape uses one quantile rule per threshold; partition ranks by conformal p on the original text.structure_baseline,docstructure_baselineandanchoring_baselinemust be rebuilt.unknown, a flat salience reference and a bucket the baseline lacks aredegradedwith counts; residue sees an appendixTODOand a\section {X}label;--glossaryis a real argparse option. Per-bucket artifacts must be recalibrated for the new grid, and the EVALUATION rates measured on the old grid say so.unmeasuredwithout the surprisal runtime instead of a RuntimeError.Tools that now do what their flags say
rewrite_reward: no exit 2 for a missing profile,voice_model.joblibor sentence-transformers; numbers and negation/causal/comparison markers compared as bags; LaTeX en-dash ranges, Unicode minus and identifiers tokenized correctly.verify_references: twelve defects (comment-hidden\cite, biblatex and multicite commands,@stringexpansion,#concatenation, first-author substring matches, cached registry misses,http.clienterrors as a crash).label_findings/eval_findings/eval_docscale/ voice training: thin strata areunmeasured,--nis a per-cell quota with a document spread, pooled recall counts a passage once, a failed encode never becomes the feature cache, the unbound-name check is scope-aware.fetch_arxiv_abstracts:--journals,--start-at,--updated-beforeand the throttle exit hold in both modes, ids are bare,\documentstylesources are read, the built-in query sets are[WGL]and--queryserves other fields.Documentation. Skills, the standard, RESPONSIBILITIES, DISPOSITIONS, the EVALUATION hub and parts, both READMEs and the tools registry say what the code now does; the validator gained a relative-link check and reads
tools/README.md; the audit ends with a landing record mapping items to commits and listing what stays open (§8.3 deduplication, the baselines to rebuild, the grid re-measurement).Test plan
python -m unittest discover -s tests: 764 tests in 32 files, OK (16 skipped for optional dependencies); v0.38.0 had 529 in 27.python tools/validate_plugin.py: all 11 checks pass (manifests, skills, documentation boundaries, recorded counts, registries, syntax, runtime/CLI, linter exit semantics, tests and CI, residue contract, 750-line budget).--help, no field, and--calibratewithout a profile: documented exit codes, no tracebacks.structure_baseline,docstructure_baseline,anchoring_baselineand recalibrate the per-bucket artifacts on a machine with the corpus, then re-measure the EVALUATION rates marked as measured on the earlier grid.🤖 Generated with Claude Code
https://claude.ai/code/session_01DuasTxTXcSQsC6U2qYvUMM
Generated by Claude Code