test: investigate Flask JSON scheduling sensitivity without changing official results

[tappe9]

Summary

The bounded investigation, final self-review and exact-tree normal CI are complete. All 13 jobs in CI 35311164081, attempt 1 passed, including all nine implementations, shared checks, smoke, the existing Axum diagnostic and the fail-closed required aggregate. No merge or official publication has been performed; #70 remains open for acceptance.

• Add a reusable, explicitly non-publishing Flask diagnostic with two predeclared phases, strict raw parsing, unchanged production images, external readiness, scoped cleanup, complete progress evidence and no selective retries.
• Record supported findings, provenance, evidence limits and reproduction commands in docs/FLASK-DIAGNOSTIC.md, linked from the Flask guide.
• Add regression coverage, including a red/green-proved guard rejecting image drift before loading the next cell.
• Remove the temporary branch-only investigation workflow. The final tree has no extra permanent workflow or automatic diagnostic run.

Exact reviewed and tested source

Identity	SHA
Base main	9b92c74cba34bd5a4bfc389075b9bef135131034
PR head	1ee3a87cda97a280e54745e425579ca5ee3222b8
GitHub-tested merge commit	e846e6b97447dce75abb6abc8dbc7049d134dad9
Identical local, PR-head and tested-merge tree	e0298668618fa31124a823502cc1886d8c8264be

Only four final paths differ from main:

1. benchmark/flask_diagnostic.py
2. tests/test_benchmark_flask_diagnostic.py
3. docs/FLASK-DIAGNOSTIC.md
4. docs/FLASK.md (a link to the diagnostic guide)

Applications, dependency pins, Compose, registry/cohort/profile definitions, existing workflows, results/**, README numeric blocks and published history remain unchanged. The separate stale-cohort documentation cleanup remains #71. Scratch notes and Superpowers plans remain ignored and untracked as required by AGENTS.md.

Findings — diagnostic values, not official improvements

The original JSON/DB inversion reproduced. Changing only the Python switch interval to 1 ms or disabling waitress.queue warnings did not improve JSON in the first experiment.

The second experiment used a same-job baseline / single-logical-CPU affinity / baseline comparison:

Arm	JSON requests/s	JSON mean ms	DB requests/s
baseline before	101.553	491.030	1190.344
one-CPU affinity	1882.871	26.508	1204.494
baseline after	102.701	485.357	1206.691

The quota remained one CPU, memory 512 MiB, one Waitress worker and DB pool maximum 10. All three cells used the same image and retained 50 connections, a five-second warm-up and three ten-second windows per endpoint. Affinity changed CPU placement, not the aggregate CPU quota or exclusive core reservation.

The observations, bracketing baselines and per-thread/cgroup counters support a CPU-placement-sensitive scheduling/CPU-quota interaction in this pinned stack. They do not identify a particular GIL/kernel/Waitress function or prove throttling alone as the cause. There is one affinity treatment cell on one host, and counter spans include warm-up and observation overhead. The approximately 18x within-job contrast is not a production speedup or a result to generalize to other hosts.

The next decision is common CPU-allocation semantics, not a Flask-only optimization. Keeping the existing quota-only baseline with a clear explanation is valid. Any common-methodology change requires separate review across the cohort; any later official run/publication requires explicit authorization.

Retained real-container evidence

Plans were recorded before their respective experiments in #70. No additional Flask hypothesis measurement or selective retry was performed during this continuation; normal PR CI retained its existing non-publishing smoke and Axum checks.

Evidence	Run / attempt	Measured source	Artifact ID
Original official publication	34925168324 / 1	c8e033ea176366644f5bed5ded12cab79e6fcbeb	10380696186
Initial hypothesis matrix	35299848616 / 1	cbebd385b0f78503d797fcc73167f1ecce1b8611	10529950752
Scheduling probe	35300974388 / 1	a153a103f445af8cd571687fb095cc0d2f4cb9a3	10529473210

All three downloaded ZIP SHA-256 values match GitHub metadata. The original 72 measured windows pass benchmark.report.audit_raw, and the original selected JSON matches the published file at the audit main byte-for-byte. Both diagnostic archives were independently replayed: 54 measured windows plus 18 warm-ups, including raw oha, memory, whole-run selection, observations, plan/source/tree and container identities. No transport errors or non-200 responses were present. See the guide and verified outcome comment for digests and telemetry.

These experiments ran on the measured sources above, before final formatting/documentation and the image-drift guard. They are not claimed as new full Flask diagnostic executions of the final PR head. The observed images already matched; the guard has separate red/green and normal-CI evidence. Artifacts have finite retention, not guaranteed permanent availability.

Validation and review

• Final diagnostic tests: 15 passed, including failure/cleanup, invalid output, hook isolation, path safety, non-publication and image drift.
• Image-drift regression failed before the guard and passed afterward; the next cell is cleaned without measured load or replacement output.
• Complete benchmark suite: 201 tests each on Python 3.10.21, Python 3.14.7 and Python 3.14.7 with -O in the exact-tree shared CI job.
• Shared-contract tests: 38; workflow tests: 21; all passed in shared CI.
• Site/history/Pages Python checks and 40 Node/browser tests passed; Node tests had zero failures or skips.
• Registry, README generation, scoped Ruff formatting/lint, whitespace and real PostgreSQL startup/reset/cleanup checks passed.
• All nine implementation acceptance and shared-contract jobs passed, including Java and Flask; the retained Axum load diagnostic passed.
• Existing non-publishing smoke and all final cleanup steps passed.
• Fail-closed required aggregate and the complete workflow finished successfully.
• Original and diagnostic raw evidence independently replayed as described above.
• Final local/remote blob and tree identities matched; protected inputs and published outputs have zero diff against main.
• Final self-review recorded against this exact head; no unresolved review threads were present at final verification.

Local full benchmark/site targets did not complete within tool execution deadlines and local Docker acceptance was unavailable; these are not claimed as local passes. The complete repository/container evidence is the successful normal PR CI above, not older main CI. Review is self-review, not independent approval; no self-approval review was submitted.

Boundaries

No official workflow dispatch, publication, Pages deployment, cohort activation, API/runtime tuning, release/tag, permissions or protection changes. #52 remains deferred and #59 remains the next justified official-run verification. This PR preserves existing results and explains an observation; it does not authorize a new methodology or close #70 merely because a diagnostic has attractive throughput.

Refs #70.

[tappe9]

Final source/evidence self-review

Reviewed head 1ee3a87cda97a280e54745e425579ca5ee3222b8, tree e0298668618fa31124a823502cc1886d8c8264be, against freshly verified main 9b92c74cba34bd5a4bfc389075b9bef135131034. GitHub's tested merge commit e846e6b97447dce75abb6abc8dbc7049d134dad9 has the identical tree. This is self-review, not an independent approval.

Scope and integrity

• The final comparison contains exactly the diagnostic module, its tests, the durable diagnostic guide and a link from the Flask guide. No application, dependency pin, Compose resource setting, cohort/profile, existing workflow, README result or published-history file changed.
• The branch-only investigation workflow is absent from the final tree. Scratch investigation and Superpowers planning paths remain ignored and untracked.
• Fixed phases preserve every attempted sample and stop on invalid metrics or partial failure. Cleanup precedes final successful output; official report validation rejects this diagnostic format.
• The new image-drift guard rejects a changed image before the next cell's contract/load, and still cleans that cell. Its regression failed before the guard and passes with the guard. Original retained experiments already used matching images within each job.
• Hook code changes only its declared diagnostic factor. Baseline has no startup hook, and CPU affinity does not increase the quota or reserve a physical core.

Evidence and interpretation

The original 72 measured windows and both diagnostic archives (54 measured windows plus 18 warm-ups) were replayed against raw data, selected whole-run values, memory and provenance. ZIP digests and source/tree identities match retained GitHub evidence. These are original-run audits, not new measurements of this final head.

The guide correctly limits the inference to CPU-placement-sensitive scheduling/CPU-quota behavior in the tested stack. It does not claim a GIL-only cause, a particular Waitress/kernel defect, a universal 18x gain or a production optimization. One affinity treatment cell on one host and outside-window collection overhead remain explicit limitations. No additional experiment is justified solely to obtain a preferred number.

Executed correctness evidence

In the shared job of CI 35311164081, the complete benchmark suite passed 201 tests each on Python 3.10.21, Python 3.14.7 and Python 3.14.7 with -O. The job also passed 38 shared-contract tests, 21 workflow tests, registry/README checks, Ruff, real PostgreSQL startup/reset/cleanup, site/history/Pages Python checks and 40 Node/browser tests with zero failures or skips. This includes the final diagnostic's 15 tests.

All nine implementation acceptance/contract jobs are successful in the current job snapshot, including Flask and the existing Axum load diagnostic. The smoke job and final required aggregate are still pending completion at this review timestamp; this review does not substitute for those gates. Local full-suite timeouts and unavailable local Docker are not claimed as local passes.

No additional source change is requested by this self-review. Complete the exact-run smoke/required verification before removing draft status. Keep #70 open until the reviewed work is accepted; do not merge or run official publication as part of this review.

[tappe9]

Superseded by #77. The follow-up alternatives screening supported the smaller Flask + Gunicorn gthread production change, which has now been merged. The investigation evidence in this PR remains useful historical context, but the permanent diagnostic subsystem is no longer needed for the selected fix.