fix: two rule #5 violations, a framework change hiding 137 routes, and a test that rewrote a tracked report - #1080
Merged
Hashevolution merged 11 commits intoAug 28, 2026
Conversation
PR #1079 merged as 6d6a079. Squash discarded the branch commits and the remote branch is gone, so the SHAs the letters cited as evidence β e19f239, a9d96f4, 8c6f726, dace68f β are now unreachable from anywhere. A later session following them would find nothing. All four references now point at the merged commit. Two substantive consequences, not just link rot: - Finding 4's letter said the third finding's scorer fix "sits in the same branch". It is on main now, so the sentence is false as written. Changed to "has landed in the meantime", which is what the reader needs to know and stays true afterwards. - Finding 1's frontmatter records that the original commit message contradicted itself on attribution. That commit no longer exists in the history, so this file is now the only record of it β noted as such rather than pointing at a dead SHA. Also folded into the README: the numbering table no longer cites commits that are gone (his own wording is what identifies each finding), and the send table names the combined 1-2-3 version the operator chose. Docs only. Quality delta: exempt (label: docs) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jttr1R2aJ3YjJkS1odiLC3
Records which version actually went out, so a later session does not re-send the per-finding drafts. Only finding 4 remains, and it is blocked on the Track 2c re-measurement. Docs only. Quality delta: exempt (label: docs) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jttr1R2aJ3YjJkS1odiLC3
β¦ne.py with a plan test_v06_module_size_gate was red on two real CLAUDE.md rule #5 violations, neither introduced by recent work: core/response_style.py 22,036 bytes (984 over β 1,556 over) core/reasoning/engine.py 21,464 bytes **response_style.py is split**, not grandfathered β the file's own GRANDFATHERED comment records an anti-creep rule, and splitting is the convention. The StylePreset dataclass and the three preset bodies (12 KB of the 22) move to core/response_style_presets.py; response_style.py keeps the resolver and re-exports every public name, so the eleven call sites across core/reasoning/*, answer_style_classifier, memory/extractor and routes/query are untouched. Import direction is one-way, so no cycle. response_style.py 4,383 bytes response_style_presets.py 19,065 bytes Verified equivalent rather than assumed: the pre-split module is loaded out of git alongside the new one and compared field-for-field. All three presets fingerprint identically (NATURAL 3,632 chars, TERSE 967, DETAILED 1,305), the five module constants match, and resolve_style agrees on eight inputs including the env-var path, an unknown id and whitespace/case variants. Re-exports are the same objects, not copies. 80 passed across the six suites that touch response_style. **engine.py is grandfathered with a split plan**, because splitting it here would be the wrong kind of change. It is core/reasoning, so rule #2 requires STEP 7 bench numbers and a Quality Delta Card β and bench.py needs a live server plus Ollama, neither of which exists in a session container. Shipping an unmeasured refactor of the hottest path to make a gate green is exactly what that rule exists to prevent. The entry names the seam an operator who can run the bench should use (the mode dispatch block, the way pipeline_synth was lifted) and notes that the structural test helpers already absorb either shape. One bug caught during the split: `@dataclass(frozen=True)` sat one line above the class and was left behind by the first cut, so StylePreset came out a plain class and every preset construction raised TypeError. Caught by importing, not by review. Quality delta: exempt (label: fix) β response_style is not core/retrieval, core/graph or core/reasoning, and no reasoning line changed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jttr1R2aJ3YjJkS1odiLC3
Same root cause as the pipeline_synth cluster, twice more. A module gets split into a package under the 20 KB gate; `inspect.getsource()` on a package returns only `__init__.py`; a structural test that greps the source reports the feature deleted when it had only moved. - `core.gemma_client` became a package (client / config / errors / response_parser). test_max_tokens_relax read it with getsource, so the num_predict and num_ctx defaults β which live in client.py β became invisible and it reported the caps as missing. - `core.reasoning.reflect` became a package. test_i18n_language_detection asked the package for `_is_korean`, which now lives in reflect/loop.py and is not re-exported. Pointed the parametrize entry at the real consumer rather than adding a re-export to production code to satisfy a test β the helper belongs where it is used. `_module_source` is now public as `module_source`, since this is the third time it has been needed and the next split will want it too. The old private name stays as an alias. 74 passed across the five suites touching these. Quality delta: exempt (label: fix) β test-side only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jttr1R2aJ3YjJkS1odiLC3
β¦ was a framework change
Seven suites failed with "'/query/' not found in {...}" while the
endpoint worked fine. The endpoints were never missing.
FastAPI used to flatten a router's routes into `app.routes` on
`include_router`, so a test could collect `{r.path for r in app.routes}`
and see everything. Under the installed versions (fastapi 0.141.1,
starlette 1.6.0) inclusion instead appends one
`fastapi.routing._IncludedRouter` wrapper per call. The wrapper routes
requests correctly but carries no `path`, so the old comprehension
silently dropped every included endpoint β 19 wrappers hiding ~137 paths
here, leaving only the 16 declared directly on the app.
Established rather than inferred. The cluster does not reproduce in a
session container because huggingface.co is blocked and the server
cannot boot, so `sentence_transformers` was stubbed to get past
VectorStore init; the import then succeeded with the CI route set
exactly. Driving the app with TestClient showed no 404 anywhere:
/healthz 200, /llm/active 422, /query/ 422, /workspace/info 401,
/templates/ 405 β every one handler-reached. The defect was in how the
tests looked, not in the app.
tests/_app_routes.py holds the knowledge in one place instead of nine:
`route_paths()` walks `original_router` recursively, so a router
included into a router is still reached. Five tests in
test_app_routes_helper.py pin it against a throwaway app β including the
prefix assumption, which is safe today because this server includes
every router without one, and which fails loudly if FastAPI starts
exposing composed paths through the wrapper.
Also in test_v06_llm_settings: `assertEqual(len(schema), 10)` went stale
when agent_enable_shell (#1042) and agent_allow_cloud (#1045) were added
deliberately, failing as "12 != 10" β accurate and useless, since it
cannot distinguish an addition from a disappearance. Replaced with the
explicit key set, which names whatever moved in either direction.
155 passed across the ten suites.
Quality delta: exempt (label: fix) β test-side only, no core/ change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Jttr1R2aJ3YjJkS1odiLC3
Running the suite left `reports/research-runs/cascade-consistency-probe.json` modified every time. The test called `probe.main()`, which hardcoded its output to that path β a committed record of a past measurement, not a scratch file. The committed copy still holds the pre-fix numbers (invalidated_leakage 3, 1/4 consistent, generated on Windows), so each run silently overwrote a historical record with current output, and a developer got a spurious diff after every test run. `main()` now takes an optional `out_path`, defaulting to None β the committed report, so running the script by hand is unchanged. The test passes a temp path. A second test pins both halves of that contract, so the hardcoded path cannot come back without failing. Caught on the way: `out_path.relative_to(ROOT)` in the closing print raises for any path outside the repo β the same trap found earlier in track2c_remeasure.py, and it fired the moment the test passed a temp dir. Falls back to the absolute path now. Verified: 3 passed, the working tree is clean after the run, and running the script directly still writes the committed report. Quality delta: exempt (label: fix) β no core/ change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jttr1R2aJ3YjJkS1odiLC3
β¦ites on the current markup **A real UI defect, found by a test that was reported as stale.** `test_admin_pw_eye` asserted the ποΈ emoji on the admin password toggle and failed with `'ποΈ' not found in ''` β the empty string is the point. The control moved to Korean text labels (νμ / μ¨κΉ) in admin.js, but the markup kept an empty button body, so it rendered blank until first clicked and only `aria-label` identified it. The CSS carries no `content` or background icon, so nothing filled it. admin.html now ships `νμ`, which matches the masked initial state and the JS flip. The rest is markup that moved while assertions stayed put: - **`test_a11y_aria_labels`** chased `id="session-btn"` and `data-action="clear-history"`, both removed by the v0.6.1 sidebar rework. Rather than re-point at replacements, the two are replaced by one scan: every button with no visible text must carry an accessible name. That keeps the actual contract, survives the next rename, and is wider β all four pages instead of a listed handful. Scanned first to be sure it was not papering over a live gap: 12 icon-only buttons, zero unlabelled. Mutation-checked that it bites. - **`test_frontend_event_delegation`** wanted `logout`, `rename-session`, `delete-session` and `toggle-session-panel` as data-actions. `logout` is deliberately not one β the role badge assigns `badge.onclick` in chat.js so delegation cannot double-fire alongside it, and index.html records that reason; a JS property assignment is not an inline handler, so the no-inline contract is intact. Session rename/delete moved to a selection-based popover that carries no per-row `data-sid`, so those are pinned in index.html instead. `intro.html` joins the migrated-pages set β verified to have zero inline handlers, not assumed. - Two more stale hardcoded counts, same treatment as the llm_settings schema: the workspace tab count (4, now 5 with "templates") becomes a name-set comparison, which says what appeared or vanished instead of "5 != 4"; and the password field's padding assertion follows its declaration into the extracted utility class. One test regex assumed `id` preceded `class` and would also have matched `admin-login-pw-toggle` by prefix; both fixed. 66 passed across the three suites (was 8 failing). Quality delta: exempt (label: fix) β one HTML label, no core/ change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jttr1R2aJ3YjJkS1odiLC3
β¦ce pattern Five failures, four distinct causes, none of them a live defect: - **`brain_icon_shimmer`** (2) sliced `js[idx:idx + 1500]` to read `appendTyping`. That function grew to ~12,000 characters, so the placeholder markup fell outside the window and the test reported it missing when it had only moved further down the same function. Every assertion it makes is satisfied by the current source. - **`reasoning_ui_animation`** expected `animation: james-spin` on the active thinking icon. It was renamed to `james-icon-active` (mobile.css:180). The `james-spin` keyframe still exists for other spinners, so the test had to follow the rename, not the keyframe. - **`threshold_labels`** matched the endpoints row by `style="display:flex;gap:4px"`. That became the `d-flex gap-4` utility classes; the μν¨ / κ°λ ₯ labels are unchanged. Now matched by class, loosely enough to survive the next extraction. - **`copy_and_conditional_download`** asserted the literal "π 볡μ¬". The three feedback controls became inline SVG with title and aria-label. Asserting the accessible name instead is both closer to what the button promises a user and stable across an icon swap. - **`workspace_cr_panel`** wanted `id="tab-cr"` with inline `display:none`; it now carries `class="d-none"`, declared in tokens.css. `tests/_js_source.py::function_body` replaces the fixed-window slice: it bounds at the next top-level function, so it tracks a function however it grows, and raises on a missing name so a rename fails loudly instead of asserting against an empty string. Thirteen more fixed-size slices exist across the suite; they are still inside their windows and are left alone rather than churned. Its own tests caught a bug in it before it shipped: the opening pattern did not allow `async function`, so `function_body(js, "beta")` missed an async declaration entirely. 62 passed across the six suites. Quality delta: exempt (label: fix) β test-side only, no core/ change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jttr1R2aJ3YjJkS1odiLC3
**The same defect as the admin password toggle, in a second place.** `test_chat_ux_n4_n5` wanted β¨ in the next-actions header and found nothing β because the emoji-to-SVG sweep removed the glyph but left `<span style="font-size:14px"></span>` behind. With the header's `gap:6px` that renders a blank box and a stray gap on every answer carrying suggestions. I had earlier called this failure "pre-existing and unrelated" and moved on; it was a real defect of exactly the kind found in admin.html. The empty span is gone. Also fixed, and recorded rather than resolved: **the vision model default is split three ways.** PR #1070 changed `config.py` from llava:13b to qwen2.5vl:7b for a "proven OCR win", but `model_resolver.py`'s vision list and `llm_settings.py` still say llava:13b. The kill-switch path reads config; the normal path β the one production actually takes β reads the resolver. So the OCR improvement may only apply when the kill-switch is on. I did not change it: picking a vision model is a behavioural decision and neither Ollama nor a test image exists here to measure OCR quality, so extending #1070's evidence-backed decision without evidence would be wrong. Written up in reports/research-runs/vision-model-default-split-20260828.md. The two vision tests hardcoded llava:13b while asserting on the kill-switch path, so they now read `config.MULTIMODAL_MODEL` and will not need editing on the next default change. The test covering the resolver path still says llava:13b, because that is still true. Four more moved-not-missing cases: - `chat_history_persistence` used an 800-char window on `clearHistory`; the v0.6.1 v8 confirm copy pushed `removeItem('james_session')` to ~950. Now uses `function_body`. - `vision_upload_wire` wanted `data-action="attach-image"`, renamed to `composer-attach`. - `graph_visual_rendering` addressed the inline `.nodeVal(function ...)` accessor; hub sizing moved into the named `_baseNodeVal` so trace-dim mode can swap and restore it. - The companion `nodeColor` test required a reference to `isHub` "to leave the hook open" β a branch where both arms returned the same value, kept alive only by the test, and dropped by that same extraction with no behavioural change. Re-adding a dead branch to make a test pass would be the wrong repair, so the assertion now states what the accessor does. 76 passed across the six suites. Quality delta: exempt (label: fix) β one span removed from chat.js, no core/ change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jttr1R2aJ3YjJkS1odiLC3
β¦ackage-split casualties `test_v06_claude_md_entry_pointer` was failing on a real defect in the file that steers every new session. The contract it enforces is that row 1 of `Where to look next` IS the cycle entry β a `docs/handovers/` document, marked so a fresh session can stop scanning there. The M9 joint-deposit record (a `reports/promo-assets/` archive) had been inserted above it, so row 1 pointed at a published record and the actual entry skeleton sat in row 2. A new session reading top-down would have taken the wrong door. Rows swapped; nothing removed. While there, the M9 row's "remaining open thread = Ali's four engineering findings" is now accurate: 1-3 were sent 2026-08-26, and 4 names the harness and where it must run. Three more `inspect.getsource()`-on-a-package cases, same cause as pipeline_synth / gemma_client / reflect.loop: - `test_reflect_meta_narration_strip` asked the reflect package for both strip helpers; they live in meta_narration.py. Still importable from the package β the import at the top of that test proves it. - `test_entity_type_extension` opened `core/graph_engine.py` by path; that is the `core/graph_engine/` package now. Both use `module_source`, which is why it was made public. One local-only artefact worth recording so the next session does not chase it: `test_reranker`'s two failures were caused by my own `sentence_transformers` stub lacking `CrossEncoder`, not by the repo β they do not appear in CI's failure list. Stub extended; 14 pass. The stub lives outside the repo (PYTHONPATH), so nothing here depends on it. 97 passed across the four suites. Quality delta: exempt (label: fix) β docs + test-side, no core/ change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jttr1R2aJ3YjJkS1odiLC3
β¦ a table-vs-text gate finding
Three more, and one of them is a live ingestion problem rather than a
test issue.
**`_looks_like_text` rejects transcribed tables.**
`test_file_processor_trusted` patched vision to return a markdown table
and then died on `Image.open("dummy.png")` β meaning the vision result
had been discarded and the code fell through to the OCR branch. The gate
has three guards written for OCR noise, and two of them punish table
syntax: pipes and `---` are non-word characters, dragging the word-char
ratio under 0.5, and each `|` is its own whitespace token, dragging the
real-word ratio under 0.4. So a table that vision transcribed correctly
is thrown away and the image is OCR'd again.
This is not confined to the toy fixture: a realistic Korean financial
table scores 0.38 against the 0.40 floor. Not fixed here β the repair
changes document-ingestion behaviour (exclude table punctuation from
both ratios, or stop applying an OCR-noise gate to vision output, which
is not OCR noise) and wants a decision plus a measurement. The test now
pins the current behaviour with the reasoning attached, so it fails
loudly when someone changes it. The success-path test uses prose, which
is what it was always trying to exercise.
**`test_inline_style_extraction`** matched `href="/static/tokens.css"`
exactly, but the links carry a cache-buster (`?v=v21-20260625-csp`), so
every page reported all three stylesheets missing. Query strings are now
allowed, still anchored on the filename.
That test also demanded every page be either extracted-from-inline or
deliberately-still-inline. intro.html is neither: it was born with
external CSS. Putting it in `_EXTRACTED` made it fail the β₯8 KB floor,
which encodes "~600 lines were moved out of this page" β never true
here, and intro.css is complete at 4.9 KB. Added
`_EXTERNAL_FROM_BIRTH`, which joins the cascade-order and accounted-for
checks but not the size floor.
Also removed an import left unused by the previous commit's change.
Local suite, replaying the CI command: **66 β 19 failures, no
regressions** β every FAILED entry on this branch also fails on main.
Quality delta: exempt (label: fix) β test-side only, no core/ change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Jttr1R2aJ3YjJkS1odiLC3
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Standing CI failures, worked cause by cause rather than silenced. Measured the same way both sides: the CI command replayed locally against
mainand against this branch.Paired local replay at the mid-point commit: 66 failed β 52 failed (14 fixed, 0 regressions). Six further commits of UI / CSS / docs re-anchoring followed. CI on the final head (
169715b): 6 failed, 5 errors, 4,362 passed β see What is still red below.The largest cluster was a framework change, not our code. Seven suites failed with
'/query/' not found in {...}while the endpoint worked. FastAPI used to flatten a router's routes intoapp.routesoninclude_router; under the installed versions (fastapi 0.141.1 / starlette 1.6.0) it appends one_IncludedRouterwrapper per call instead. The wrapper routes correctly but carries nopath, so the usual comprehension silently dropped every included endpoint β 19 wrappers hiding ~137 paths, leaving the 16 declared directly on the app.tests/_app_routes.pyholds the unwrapping in one place instead of nine.Two real CLAUDE.md rule #5 violations, neither introduced recently:
core/response_style.pycore/reasoning/engine.pyresponse_style.pyis split rather than grandfathered because the gate's own comment records an anti-creep rule. TheStylePresetdataclass and the three preset bodies moved tocore/response_style_presets.py; the resolver and every public name stay put, so the eleven call sites are untouched.engine.pyis not split here on purpose. It iscore/reasoning, so rule #2 requires STEP 7 bench numbers and a Quality Delta Card β andbench.pyneeds a live server plus Ollama, neither of which exists in a session container. Shipping an unmeasured refactor of the hottest path to make a gate green is what that rule exists to prevent. The grandfather entry names the seam for an operator who can run the bench.Three more package-split casualties.
gemma_clientandreflectbecame packages;inspect.getsource()on a package returns only__init__.py, so structural tests reported features deleted when they had only moved. This is the third occurrence, so the helper is now public asmodule_source.A test was rewriting a tracked measurement. Every suite run left
reports/research-runs/cascade-consistency-probe.jsonmodified β the test calledprobe.main(), which hardcoded that path. The committed copy still holds the pre-fix numbers (leakage 3, 1/4 consistent, generated on Windows), so each run overwrote a historical record.main()now takes an optionalout_path; running the script by hand is unchanged.Two real UI defects, found by re-anchoring the tests rather than the tests being wrong.
admin.html's password toggle rendered as an empty<button></button>(no label, no icon);chat.js's next-actions header carried a second empty decorative span. Both are the same defect class, and both are fixed rather than asserted around.Verification
Local runs replay the CI command exactly β same
--ignorelist, same env, same flags. One deviation, needed only here:sentence_transformersis stubbed viaPYTHONPATH, becausehuggingface.cois blocked in a session container and the server cannot boot without it. CI downloads the real model, so CI is unaffected.main(6d6a079), local replay169715b, CIEvery fixed item corresponds to a change here; the diff of FAILED sets shows no new entries.
The route cluster was established, not inferred. With the stub in place the import reproduced CI's route set exactly, and driving the app with
TestClientshowed no 404 anywhere:/healthz200,/llm/active422,/query/422,/workspace/info401,/templates/405 β all handler-reached. The defect was in how the tests looked.The split was proved equivalent, not assumed. The pre-split module is loaded out of git alongside the new one and compared field by field: all three presets fingerprint identically (NATURAL 3,632 chars, TERSE 967, DETAILED 1,305), the five constants match, and
resolve_styleagrees on eight inputs including the env path, an unknown id and whitespace/case variants. Re-exports are the same objects, not copies.ruff check .clean;banditclean. Working tree clean after a full suite run (it was not before).One measurement of my own was invalid and was redone. A first tally compared a
tail-truncated branch output (22 items) againstmain's full output (49) and claimed 27 fixed. Spot-checking three supposedly-fixed tests showed them failing identically on both sides. Re-measured properly: 14 fixed, 0 regressions β the number above.Two bugs caught while working, both by running rather than reading:
@dataclass(frozen=True)sat one line above its class and was left behind by the first cut, makingStylePreseta plain class β every preset construction raisedTypeError.out_path.relative_to(ROOT)raises for any path outside the repo, and fired the moment a test passed a temp dir. Same trap found earlier intrack2c_remeasure.py.What is still red
test_lrb_v021_cross_model(1)test_measurement_critical_surfaces(3)test_mobile_responsive(1)!importantagainst a budget of 25.test_native_done_reason(1) +test_entity_name_markdown_strip(5 errors)pytest-timeouterrors on a filetests/conftest.pyalready names as intermittent on cold runners; the 1 is its documented cascade, from the timeout leakingpatch("llm.router.RouterWrapper"). A single flake accounts for all six.Quality Delta vs baseline
Quality delta: exempt (label: fix)
core/changes areresponse_style.pyand the newresponse_style_presets.pyβ notcore/retrieval,core/graphorcore/reasoning, and no behaviour changed (proved above). Everything else is test-side, measurement-side and docs.Out of scope
core/reasoning/engine.pyβ blocked on rule wiki_reset.py crashes on Windows CP949 console (UnicodeEncodeError)Β #2, as above. Plan recorded in the grandfather entry._looks_like_textrejecting transcribed markdown tables β a realistic Korean financial table scores 0.38 against a 0.40 floor, so vision output is discarded and the image is re-OCR'd. Pinned by a test that records the behaviour; not fixed, because changing the gate is a retrieval-quality change that wants measurement.config.pytoqwen2.5vl:7bwhilemodel_resolver.py:152andllm_settings.py:51still sayllava:13b. Recorded inreports/research-runs/vision-model-default-split-20260828.md.π€ Generated with Claude Code
https://claude.ai/code/session_01Jttr1R2aJ3YjJkS1odiLC3
Generated by Claude Code