Repository navigation
P1: Make local validation and review evidence reproducible #259
Description
Scope clarification from the full audit (2026-09-13)
Closure records the no-CI policy retirement and the specific prior local validation evidence from PR #276. It is not certification that every Make target, every test tier or clean-environment setup is complete. Newly source-confirmed defects (missing make test target file, omitted security/performance tiers, out-of-lock browser dependency and sandbox gaps) are explicitly tracked in #281. Agent and safe-tool completion are #282/#283. Keep this historical policy milestone closed while those concrete acceptance tasks remain open; do not copy its result counts as a new run.
Parent roadmap: #252
Current policy (2026-09-13)
No CI checks. Everything is validated locally. This includes code review,
formatting, lint, type checks, migrations, all test tiers, security scans,
artifact checks and mobile validation. Do not add hosted jobs, required CI
statuses, synthetic success checks or an Ollama reviewer. GitHub is for source,
PRs, issues and publication, not validation.
Record commands, environment, results, limitations and reviewed head/base SHAs.
Builders merge only with completed local evidence and GitHub mergeability;
reviewer agents never merge. Real staging/provider/device acceptance remains
required where applicable, driven by authorized local operator tools.
Historical evidence and older comments do not override this policy.
Scope
Replace the obsolete hosted AI/CI gate task with a reproducible local validation
and review process. No AI provider, GitHub secret, workflow or required status
is needed for code review. Do not restore the retired gate.
Implementation plan
- Remove/disable backend static CI, mobile analysis CI, hosted AI review and legacy
E2E workflows. Keep Pages publication separate from validation. - Align agent instructions, current guides, roadmap children and README. Remove
the retired CI badge and all active prerequisites for hosted checks. - Guard the workflow inventory with local regression tests.
- Record final-source local review and test evidence, then use normal PR merging
only when GitHub says mergeable. Reviewer agents must not merge. - Verify no stale required checks remain. If future settings conflict, report them
for a policy-aligned administrative correction; do not bypass or fake success.
Acceptance criteria
- No hosted CI/review/test workflow remains active or is required.
- Local policy tests reject reintroduction of hosted validation.
- Current roadmap/docs/agent rules consistently require local evidence.
- A real implementation PR records local validation and reviewed head/base SHAs.
- Normal merge and local-main synchronization are verified without CI prerequisites.
Validation
Run all applicable validation locally, including make test-all. Inspect live
workflow/protection state read-only. The latest protection checks returned 404
and an empty ruleset list; no protection changes are needed.
Historical context
The original AI-gate implementation and provider diagnosis in older comments are
superseded, not rollout instructions. PR #272 and #275 are merged. The earlier
90-error scoped-mypy result in a borrowed virtualenv was not a confirmed source
blocker: a clean locked environment passed. Keep actual full-API typing debt and
current environment verification separate.
Completed
PR #276 merged as 21a4a80 after recorded local
review and validation. GitHub reported mergeable with no status checks.
Local main is synchronized. All 21 open roadmap issues were reconciled with
this policy; remaining security/release work is not closed by this ticket.
See PR #276 for commands, source SHA, actual outcomes and limitations.
Activity
tomqwu commentedon Sep 9, 2026
Prerequisite discovered while validating draft PR #272: the current required CI command
poetry run mypy api/utils api/core api/schemasreports 90 errors from imported api/models.py and api/security.py in the local locked-version Python 3.11 environment. The same result was reproduced against untouched main at 214e3f3. Fullmypy apireports 1096 errors on both versions. Resolve/verify this CI baseline before enabling required-check enforcement; do not suppress failures merely to turn the gate green. No workflow or protection setting was changed in #272.tomqwu commentedon Sep 9, 2026
Update: the actual GitHub CI strict-mypy step on PR #272 passed. The 90-error result above is local and reproduced on the local untouched-main snapshot, but it is NOT a confirmed GitHub CI blocker. Investigating cache/environment differences; preserve this distinction when planning enforcement. CI test jobs are still running, and Codex review jobs were skipped.
tomqwu commentedon Sep 9, 2026
Resolved the local strict-mypy discrepancy: a fresh worktree-local Python 3.11 virtualenv installed from poetry.lock passes
mypy --no-incremental api/utils api/core api/schemaswith no issues in 61 files, matching GitHub CI. The earlier 90-error result came from the borrowed local environment and must not be treated as a source-code blocker. No mypy settings or checks were changed. The outstanding review prerequisite remains: Codex review jobs are skipped, not approved.Use Ollama Cloud for AI pull request review
tomqwu commentedon Sep 10, 2026
Provider conversion pushed in draft PR #272, commit 785c894: AI review now calls Ollama Cloud at https://ollama.com/api/chat with glm-5.3-flash by default and OLLAMA_API_KEY from Actions secrets. OPENAI_API_KEY/openai-codex-action are no longer required by the workflow.
The check is named codex-pr-review-gate and fails closed on missing credentials, provider errors, malformed/incomplete output, missing/truncated/oversized diffs, stale head/base SHAs, or blocking findings. It publishes validated feedback, does not execute PR code, and does not merge. Agent instructions and docs/ai-pr-review.md describe configuration and limits.
Validation: 25 mocked workflow cases pass; full local suite passed (376 unit, 442 API, 16 CLI, 325 integration; web/contract 221). Black, Ruff, actionlint, and strict mypy passed. GitHub CI/live provider review is pending on this head; no local inference was possible without the key.
Keep this issue open. The workflow is not on the default branch yet, the key/live provider path still needs verification, and GitHub branch protection/rulesets have NOT been changed. Require the check only after rollout and observation on a PR; trusted review of workflow changes remains necessary.
tomqwu commentedon Sep 10, 2026
Ollama conversion validation completed for draft PR #272, commit 785c894.
No merge or protection/settings changes were made. The workflow is still in a draft PR, not the default branch. Keep rollout/enforcement work open and require the check only after the documented workflow-first rollout.
tomqwu commentedon Sep 10, 2026
Ollama authentication diagnosis on PR #272, commit 4530724:
No AI approval, merge, protection/ruleset change, or secret value disclosure. Keep this issue open; enforcement rollout remains pending.
tomqwu commentedon Sep 10, 2026
Final CI result for diagnostic commit 4530724 on PR #272: https://github.com/tomqwu/SignUpFlow/actions/runs/34491204461 passed, including the main test job, Playwright, and PostgreSQL migration check. The separate Ollama review remains failed with HTTP 401: https://github.com/tomqwu/SignUpFlow/actions/runs/34491204361. Replace the rejected OLLAMA_API_KEY Actions secret with a valid Ollama Cloud API key and rerun review. PR remains draft/unmerged; no enforcement settings were changed.
tomqwu commentedon Sep 10, 2026
Live Ollama access and review publication are now verified on PR #272, commit 3816f7c.
Authentication is no longer a blocker. Keep this issue open for trusted workflow review and the ordered enforcement rollout; no merge or GitHub protection/settings changes were performed.
tomqwu commentedon Sep 10, 2026
Final verification on PR #272 / 3816f7c: regular CI passed (https://github.com/tomqwu/SignUpFlow/actions/runs/34512130159), including Playwright, integration tests, and PostgreSQL migration. Ollama review also passed (https://github.com/tomqwu/SignUpFlow/actions/runs/34512130147), with HTTP 200 and a published SHA-bound report. The new key works; bounded streaming resolves the observed buffered-response timeout. Keep #259 open for trusted review, normal merge, and the ordered enforcement rollout. No merge or protection/settings changes were made.
Harden tenant authorization and repair password-reset delivery (#272)
[-]P1: Enforce CI and a fail-closed AI review gate on main[/-][+]P1: Make local validation and review evidence reproducible[/+]