docs: refresh post-pre-deploy-audit (M-1, M-2, N-1, N-2)#25
Conversation
Resolves the 4 documentation-drift findings deferred from PR #21: - M-1 (CLAUDE.md "Known issues" stale): rewritten — all 9 P1 + all 12 P2 closed; ~13 P3 remain; added pre-deploy-audit summary (4 MAJOR + 9 MINOR remaining after this PR closes M-1/M-2/N-1/N-2). - M-2 (WORKFLOW.md aspirational stages): status banner now reflects all 6 stages shipped 2026-04-27 (probe.py 12 tests, post_process.py 6 tests, validate.py 21 tests). - N-1 (test count drift): CLAUDE.md "52 tests, ~0.6s" -> "210 passed + 2 skipped, ~3s (live OpenAI gated)". - N-2 (cost claim unverified): PERSONALIZATION.md + HANDOFF.md annotated with 2026-04-27 live measurement (text-only ~$0.05/run across 2 runs; $0.32 forecast assumes 3 hero/feature images on top). Forecast number preserved since the breakdown is consistent. Drive-by: - README.md reading-order: WORKFLOW.md no longer 'Stages 1, 6 aspirational'; PERSONALIZATION.md no longer 'spec only'. - ROADMAP.md current-state: rewritten to reflect Phase 0-6 shipped + pre-deploy audit landed. P2-9 deferral clarified (doc reword closed 2026-05-10; A/B harness itself still deferred). - PERSONALIZATION.md status banner flipped SPEC ONLY -> SHIPPED with pointer to personalize/ + 89 tests + 2 gated live tests. - HANDOFF.md (historical 2026-04-27 snapshot) got a top-of-file update banner pointing readers to ROADMAP.md / TODO.md / PRE_DEPLOY_AUDIT for current state. - PRE_DEPLOY_AUDIT_2026-05-10.md findings table updated: M-1, M-2, N-1, N-2 marked RESOLVED with strike-through; executive-summary counts re-narrated.
📝 WalkthroughWalkthroughDocumentation updates reflect project milestone completion: all six phases shipped as of 2026-05-10, pre-deploy audit items resolved with PR ChangesPhase Completion and Audit Status Updates
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~3 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the project documentation across several files to reflect the current state of the codebase as of May 2026. Specifically, it marks all six development phases as shipped, updates test counts to 210, and reconciles OpenAI cost forecasts with actual measurements. The documentation now correctly shows that P1 and P2 audit findings are closed and provides an updated status for the pre-deploy audit. Feedback was provided to correct an inconsistency in the reported number of remaining major and minor issues in CLAUDE.md to ensure it aligns with the latest audit results.
| > CVE within M-3 also bumped there. Remaining: 4 MAJOR + 9 MINOR | ||
| > deferred (cryptography 41 bump, doc drift, in-memory rate-limit storage, | ||
| > Playwright 1.57 memory regression, Dockerfile hardening, etc.). |
There was a problem hiding this comment.
The counts for remaining findings and the list of deferred items are inconsistent with the updates in docs/PRE_DEPLOY_AUDIT_2026-05-10.md. Since this PR resolves M-1, M-2, N-1, and N-2 (and N-3 was already resolved via B-2), the remaining count should be 3 MAJOR (M-3 cryptography, M-4, M-5) and 6 MINOR (N-4 through N-9). Additionally, "doc drift" should be removed from the deferred list as it is the primary focus of this PR.
| > CVE within M-3 also bumped there. Remaining: 4 MAJOR + 9 MINOR | |
| > deferred (cryptography 41 bump, doc drift, in-memory rate-limit storage, | |
| > Playwright 1.57 memory regression, Dockerfile hardening, etc.). | |
| > CVE within M-3 also bumped there. Remaining: 3 MAJOR + 6 MINOR | |
| > deferred (cryptography 41 bump, in-memory rate-limit storage, | |
| > Playwright 1.57 memory regression, Dockerfile hardening, etc.). |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
README.md (1)
20-20:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winInconsistent status claim for PERSONALIZATION.md.
Line 20 describes
docs/PERSONALIZATION.mdas "spec only — not yet implemented", but line 176 in the reading order section states it was "shipped 2026-04-27". This contradicts the PR objective which indicates personalization was shipped in Phase 4.🔧 Proposed fix
-| **Architecture specs** | `docs/PROMPT_v2.md`, `docs/WORKFLOW.md`, `docs/PERSONALIZATION.md` | Optimized LLM prompt for design-system extraction, 6-stage workflow plan, and OpenAI Responses-API personalization architecture (spec only — not yet implemented). | +| **Architecture specs** | `docs/PROMPT_v2.md`, `docs/WORKFLOW.md`, `docs/PERSONALIZATION.md` | Optimized LLM prompt for design-system extraction, 6-stage workflow plan, and OpenAI Responses-API personalization architecture (shipped 2026-04-27). |🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@README.md` at line 20, The README has contradictory status text for docs/PERSONALIZATION.md: the Architecture specs table line that says "spec only — not yet implemented" should be reconciled with the Reading order entry that marks PERSONALIZATION as "shipped 2026-04-27"; update the Architecture specs table entry for docs/PERSONALIZATION.md to reflect the shipped status (or conversely update the Reading order entry if the correct state is "not yet implemented") so both sections match, keeping the chosen status and date consistent with the PR objective that personalization shipped in Phase 4.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/PRE_DEPLOY_AUDIT_2026-05-10.md`:
- Around line 14-18: The executive summary and CLAUDE.md disagree on remaining
issue counts; reconcile them to "3 MAJOR, 6 MINOR" by updating the incorrect
text. Use the detailed findings table to confirm that M-1 and M-2 are resolved,
M-3 partially deferred (cryptography), and M-4/M-5 deferred (total 3 MAJOR
remaining), and N-1..N-3 resolved with N-4..N-9 deferred (6 MINOR remaining),
then edit the CLAUDE.md/Executive Summary section so both state "3 MAJOR, 6
MINOR remaining after this PR" and ensure any nearby summary lines referencing
"4 MAJOR + 9 MINOR" are corrected.
In `@docs/WORKFLOW.md`:
- Around line 6-11: Update the status banner date in docs/WORKFLOW.md so it
reflects the actual completion date of the implementation: change the string
"Implementation status (2026-05-10)" to "Implementation status (2026-04-27)";
locate the banner line containing that exact phrase and replace the date to
match the Phase 5 landing date referenced in the surrounding lines (e.g., the
lines mentioning Phase 5 (2026-04-27) and scripts like `scripts/probe.py`).
---
Outside diff comments:
In `@README.md`:
- Line 20: The README has contradictory status text for docs/PERSONALIZATION.md:
the Architecture specs table line that says "spec only — not yet implemented"
should be reconciled with the Reading order entry that marks PERSONALIZATION as
"shipped 2026-04-27"; update the Architecture specs table entry for
docs/PERSONALIZATION.md to reflect the shipped status (or conversely update the
Reading order entry if the correct state is "not yet implemented") so both
sections match, keeping the chosen status and date consistent with the PR
objective that personalization shipped in Phase 4.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 4e3e618d-800a-469a-a850-0d3ffeffe962
📒 Files selected for processing (7)
CLAUDE.mdREADME.mdROADMAP.mddocs/HANDOFF.mddocs/PERSONALIZATION.mddocs/PRE_DEPLOY_AUDIT_2026-05-10.mddocs/WORKFLOW.md
| | Severity | Count | Status | | ||
| |---|---|---| | ||
| | 🔴 BLOCKER | 2 | **Both fixed.** | | ||
| | 🟡 MAJOR | 5 | 1 partially fixed (urllib3 within M-3); 4 deferred. | | ||
| | 🟢 MINOR | 9 | Documented, deferred. | | ||
| | 🔴 BLOCKER | 2 | **Both fixed** in PR #21. | | ||
| | 🟡 MAJOR | 5 | M-1, M-2 closed in follow-up docs PR. M-3 split: urllib3 fixed in PR #21; cryptography deferred. M-4, M-5 deferred. | | ||
| | 🟢 MINOR | 9 | N-1, N-2 closed in follow-up docs PR. N-3 resolved via B-2 fix. N-4..N-9 deferred. | |
There was a problem hiding this comment.
Inconsistency: remaining MAJOR/MINOR counts don't match CLAUDE.md claim.
The executive summary correctly states that after this PR:
- MAJOR: M-1, M-2 resolved; M-3 (cryptography part), M-4, M-5 deferred → 3 remaining
- MINOR: N-1, N-2, N-3 resolved; N-4..N-9 deferred → 6 remaining
However, CLAUDE.md lines 129-130 claims "4 MAJOR + 9 MINOR remaining after this PR".
This appears to be a counting error in CLAUDE.md. Either that file needs correction, or this executive summary needs adjustment to reconcile the discrepancy.
🔢 Verify and align the counts
Check which document has the correct count by reviewing the detailed findings table (lines 75-88). Based on that table:
MAJOR (5 total):
M-1: RESOLVED ✅M-2: RESOLVED ✅- M-3: split (urllib3 fixed, cryptography DEFERRED) → 1 remaining
- M-4: DEFERRED → 1 remaining
- M-5: DEFERRED → 1 remaining
Remaining: 3 MAJOR
MINOR (9 total):
N-1: RESOLVED ✅N-2: RESOLVED ✅- N-3: RESOLVED via B-2 ✅
- N-4..N-9: DEFERRED → 6 remaining
Remaining: 6 MINOR
If this count (3 MAJOR + 6 MINOR) is correct, update CLAUDE.md:129-130. If CLAUDE.md is correct, adjust this executive summary.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/PRE_DEPLOY_AUDIT_2026-05-10.md` around lines 14 - 18, The executive
summary and CLAUDE.md disagree on remaining issue counts; reconcile them to "3
MAJOR, 6 MINOR" by updating the incorrect text. Use the detailed findings table
to confirm that M-1 and M-2 are resolved, M-3 partially deferred (cryptography),
and M-4/M-5 deferred (total 3 MAJOR remaining), and N-1..N-3 resolved with
N-4..N-9 deferred (6 MINOR remaining), then edit the CLAUDE.md/Executive Summary
section so both state "3 MAJOR, 6 MINOR remaining after this PR" and ensure any
nearby summary lines referencing "4 MAJOR + 9 MINOR" are corrected.
| > **Implementation status (2026-05-10).** All 6 stages shipped. | ||
| > Stages 1 (`scripts/probe.py`, 12 tests), 3 (`scripts/post_process.py`, 6 tests), | ||
| > 6 (`scripts/validate.py`, 21 tests) all landed in Phase 5 (2026-04-27). Stage 4 | ||
| > Track B (LLM prompt v2) is the optional polished deliverable; Track A (Python | ||
| > generator) is the deterministic baseline. See `docs/AUDIT.md` for findings status | ||
| > and `ROADMAP.md` for the full phase log. |
There was a problem hiding this comment.
Date inconsistency in status banner.
Line 6 states "Implementation status (2026-05-10)" but line 8 says the stages "landed in Phase 5 (2026-04-27)". The status banner date should reflect when the implementation was completed (2026-04-27), not the date of this documentation update.
📅 Proposed fix
-> **Implementation status (2026-05-10).** All 6 stages shipped.
+> **Implementation status (2026-04-27).** All 6 stages shipped.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| > **Implementation status (2026-05-10).** All 6 stages shipped. | |
| > Stages 1 (`scripts/probe.py`, 12 tests), 3 (`scripts/post_process.py`, 6 tests), | |
| > 6 (`scripts/validate.py`, 21 tests) all landed in Phase 5 (2026-04-27). Stage 4 | |
| > Track B (LLM prompt v2) is the optional polished deliverable; Track A (Python | |
| > generator) is the deterministic baseline. See `docs/AUDIT.md` for findings status | |
| > and `ROADMAP.md` for the full phase log. | |
| > **Implementation status (2026-04-27).** All 6 stages shipped. | |
| > Stages 1 (`scripts/probe.py`, 12 tests), 3 (`scripts/post_process.py`, 6 tests), | |
| > 6 (`scripts/validate.py`, 21 tests) all landed in Phase 5 (2026-04-27). Stage 4 | |
| > Track B (LLM prompt v2) is the optional polished deliverable; Track A (Python | |
| > generator) is the deterministic baseline. See `docs/AUDIT.md` for findings status | |
| > and `ROADMAP.md` for the full phase log. |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/WORKFLOW.md` around lines 6 - 11, Update the status banner date in
docs/WORKFLOW.md so it reflects the actual completion date of the
implementation: change the string "Implementation status (2026-05-10)" to
"Implementation status (2026-04-27)"; locate the banner line containing that
exact phrase and replace the date to match the Phase 5 landing date referenced
in the surrounding lines (e.g., the lines mentioning Phase 5 (2026-04-27) and
scripts like `scripts/probe.py`).
2 participants
Resolves the 4 documentation-drift findings deferred from PR #21:
Drive-by:
Summary by CodeRabbit