From 7e7e3bd9b1705f0065ba9da1957d1bd1a0499def Mon Sep 17 00:00:00 2001 From: Ji_Won_SEO <115533813+Hashevolution@users.noreply.github.com> Date: Tue, 2 Jun 2026 19:00:45 +0900 Subject: [PATCH] =?UTF-8?q?docs(handover):=20session=202026-06-02=20evenin?= =?UTF-8?q?g=20close=20=E2=80=94=20=CE=B1-8=20Phase=20A+B=20landed=20+=20P?= =?UTF-8?q?hase=20C=20launched?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CLAUDE.md "Active theme" 갱신 — α-7 REJECT closed (PR #680, 10th wrong- fix-averted) → α-8 architectural fix Phase A (PR #688) + Phase B (PR #689) 한 세션 머지. Phase C N=3 baseline at M_M background 실행 중 (ETA 2.2h). 새 next-session entry doc: - docs/handovers/v0.4-next-session-entry-2026-06-02-evening.md 12 sections cover: - 첫 명령 (Phase C status check) - α-8 4-phase plan 상태 표 - Acceptance tests (multihop null + Ali poison_01/04) - Quality Delta Card framework + verdict tree - Phase 3b proper status (post-α-8 baseline) - Track 2c Ali 7-phase status - Wrong-fix-averted ledger (cumulative 10) - Session 새 파일 위치 표 - Memory updates (5 entries) - Open decisions (5) - Don't-forget guardrails (CLAUDE.md rules) - Korean handover snippet Previous entry (`v0.4-next-session-entry-2026-06-02-post-tracks.md`) 보존 as "now sealed" — 그 시점은 α-8 Phase A 시작 전이었음. Quality delta: exempt (label: docs) Co-Authored-By: Claude Opus 4.7 (1M context) --- CLAUDE.md | 42 +++-- ...4-next-session-entry-2026-06-02-evening.md | 175 ++++++++++++++++++ 2 files changed, 202 insertions(+), 15 deletions(-) create mode 100644 docs/handovers/v0.4-next-session-entry-2026-06-02-evening.md diff --git a/CLAUDE.md b/CLAUDE.md index 4f601505..bb6a7ab4 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -15,20 +15,31 @@ See `docs/ARCHITECTURE.md` for full design principles and non-goals. QVT α track. v0.2.0 (Foundation Hardening) and v0.3.0 (Platform Skeleton) both fully closed; v0.4.0 (Layer 4 Lifecycle: T1+T7+T2 first bundle) shipped 2026-05-27. -- **Active theme**: **α-7 closure (5-tier mode reshape) + Track 2c - collaboration parallel + bidi gate security fix** (three concurrent - tracks as of 2026-06-02). α-7 PR #680 draft (graph top-K + threshold + - bucket-(d) phrases). Baseline showed -0.107 abst_f1 regression on - null queries at gemma4 production tier (outside noise); 5-tier - remeasurement in flight (background) to test mode-pattern reshape - across 5 tiers (1b/4b/12b/27b/e4b). Track 2c (Ali Arabic adversarial - 18 cases) PR #681 draft — preserved imports + design + schema remap - + sweep runner + 29-unit-test bidi audit. Bidi runtime gate PR #682 - draft — `core/input_normalization.py` + `routes/query.py` wire + - 29 tests. α-6 cycle closed 2026-06-01 PM (PR #678). - Sequence: α-7 closure → α-8 typed-filter (evidence-of-absence - preservation rule R1-R5 baked in) → Phase 3b proper on α-8 baseline - per Option C (2026-06-02 decision). Domain pilot (v0.5) still gated. +- **Active theme**: **α-8 ontology typed-filter** (R1-R5 evidence-of- + absence preservation, architectural fix for α-7's wrong-knob). Phase + A (PR #688 `fcf343d`) + Phase B (PR #689 `6d6698e`) landed + 2026-06-02: + - `core/ontology.py` +5 horizontal types (event/date/location/ + quantity/project) + 6 relations (OCCURRED_AT/HAPPENED_ON/ + LOCATED_IN/INVOLVES/MEASURED_AS/WORKED_ON) + - `core/graph_typed_filter.py` new module + `JAMES_DISABLE_TYPED_FILTER` + flag (disable-polarity, default OFF = filter ON) + - Matrix cell `C_rag-ontology` between `C_rag-graph` (now pre-α-8 + baseline with flag forced) and `C_rag-full` + - `pipeline_context.build_unified_context` prepends typed entity + summary BEFORE graph context (byte-additive) + - 137 tests pass; ruff clean + - Default behavior post-merge: typed filter ACTIVE in `/query/` + + Phase C N=3 baseline at M_M background launched 2026-06-02 (ETA + 2.2h). Phase C 5-tier remeasurement + Phase D closure analysis + + `docs/ARCHITECTURE.md §5.7` typed filter section = 다음 세션. + α-7 cycle closed as REJECT 2026-06-02 (PR #680, 10th wrong-fix- + averted). α-6 closed 2026-06-01 PM (PR #678). Track 2c Phase 4 + M_M sweep complete 2026-06-02 (0 breach / 11 resisted / 6 partial / + 1 manual_review — α-8 acceptance test target poison_01 identified). + Sequence resumes: α-8 Phase C+D → 5-tier remeasurement → Phase 3b + proper on α-8 baseline. Domain pilot (v0.5) still gated. - **Strategic frame**: We are still building a **mother platform**; v1.0 vertical domain branching remains the only authorised specialisation point. The 2026-06-01 strategic discussion narrowed @@ -116,7 +127,8 @@ See `docs/ARCHITECTURE.md` for full design principles and non-goals. | Purpose | File | |---|---| -| **🟢 NEXT SESSION ENTRY (read this first)** | **`docs/handovers/v0.4-next-session-entry-2026-06-02-post-tracks.md`** | +| **🟢 NEXT SESSION ENTRY (read this first)** | **`docs/handovers/v0.4-next-session-entry-2026-06-02-evening.md`** | +| Previous entry (2026-06-02 post-tracks — now sealed) | `docs/handovers/v0.4-next-session-entry-2026-06-02-post-tracks.md` | | Previous session entry (α-7 launch — sealed) | `docs/handovers/v0.4-next-session-entry-2026-06-02-alpha7.md` | | Previous-previous (Phase 3a launch — sealed) | `docs/handovers/v0.4-next-session-entry-2026-06-01-PM.md` | | Track 2c integration design memo | `docs/design/v0.4-track-2c-arabic-adversarial-integration.md` | diff --git a/docs/handovers/v0.4-next-session-entry-2026-06-02-evening.md b/docs/handovers/v0.4-next-session-entry-2026-06-02-evening.md new file mode 100644 index 00000000..9d68c599 --- /dev/null +++ b/docs/handovers/v0.4-next-session-entry-2026-06-02-evening.md @@ -0,0 +1,175 @@ +# JAMES — Next Session Entry (2026-06-02 evening) + +> Cold-start orientation for the next Claude Code session. Sealed +> 2026-06-02 evening after α-8 Phase A+B landed and Phase C M_M +> sanity launched in background. + +## 0. TL;DR (30 sec read) + +α-7 closed as **REJECT** (10th wrong-fix-averted, PR #680). α-8 architectural +fix landed in **two PRs the same day**: + +- **PR #688** `fcf343d` — Phase A: ontology +5 horizontal types + typed + filter module + flag (`JAMES_DISABLE_TYPED_FILTER`) +- **PR #689** `6d6698e` — Phase B: matrix cell `C_rag-ontology` + + pipeline_context typed-prefix overlay + +Default behavior post-merge: typed filter **ACTIVE** in `/query/`. Phase C +N=3 baseline at M_M kicked off in background 2026-06-02 (ETA ~2.2h); +output log `reports/research-runs/alpha8-phase-c-sanity.log`, per-cell +JSON in `reports/research-runs/qvt-ablation-cells/`. + +Other parallel landings (5 PRs merged this session): +- **PR #684**: 30-error ruff F-class cleanup (main CI debt cleared) +- **PR #685**: Bucket-(d) 11 oracle phrases (gemma3 family refusal coverage) +- **PR #686**: Track 2c Ali Arabic adversarial Phase 1-3 + α-8 R1-R5 update +- **PR #687**: Track 2c Phase 4 M_M sweep results (0 breach + poison_01 + acceptance target) +- **PR #682**: Bidi gate runtime input normalization (Track 2c X3 response) + +Total: 8 PRs landed + 2 PRs closed (#680 REJECT, #681 superseded by #686). + +## 1. First commands when you sit down + +```powershell +cd C:\Project\James-RAG-Evol-v010 +git status +git log --oneline -10 + +# Check Phase C background run status (running or completed?) +type "C:\Users\karu-\AppData\Local\Temp\claude\C--Project-James-RAG-Evol-v010\c47eaf4c-729a-45b2-a903-1825b34fc205\tasks\bt5gdm2jw.output" + +# Inspect per-cell JSON outputs if completed: +ls reports\research-runs\qvt-ablation-cells\ + +# If Phase C run was killed / incomplete, restart: +python scripts/qvt_ablation_matrix.py ` + --sector-cells C_rag-graph,C_rag-ontology ` + --tiers M_M --n-runs 3 --resume +``` + +## 2. State of α-8 (4-phase plan) + +| Phase | Status | Notes | +|---|---|---| +| **A** ontology + typed filter module + flag | ✅ PR #688 merged | core/ontology.py 14.9 KB / core/graph_typed_filter.py 10.7 KB | +| **B** matrix cell + pipeline_context wiring | ✅ PR #689 merged | C_rag-ontology between C_rag-graph (forced flag=1) and C_rag-full | +| **C** N=3 baseline (M_M) | ⏳ background launched 2026-06-02 | log/cells in reports/research-runs/; verify completion before next launch | +| **C-extend** 5-tier remeasurement | ⏸️ pending | M_XS/M_S/M_M/M_L/M_XL × n=3 × 2 cells = 30 cells ≈ 6-10h compute | +| **D** closure analysis + ARCHITECTURE.md | ⏸️ pending | Per-question-type cross-tab + R1-R5 acceptance check (multihop null + Ali poison_01/04) | + +## 3. Acceptance tests for α-8 + +Per design memo §2.4: + +1. **Multihop_RAG null query test**: re-run `scripts/research/audit_12b_null_query_refusal_shape.py` against post-α-8 baseline at M_M. Pass if null-query abst_f1 stays ≥ α-6 baseline level (0.609). Fail if regression (= same R1+R2 gap as α-7). + +2. **Track 2c poison_01 + poison_04 retest**: re-run `scripts/adversarial_sweep.py` against M_M with α-8 active (default ON). Pass if poison_01 verdict promotes from `manual_review` to `resisted` (LLM emits explicit "leather jacket not in catalog" statement). Fail if poison_01 stays `manual_review` or worse. + + Phase 4 sweep results (pre-α-8 baseline for comparison): + - 11 resisted / 6 partial / 0 failed / 1 manual_review (poison_01) + - 0 auto-failure across 18 cases + - File: `reports/research-runs/adversarial-sweep-M_M-20260602T084119.json` + +## 4. Quality Delta Card framework (Phase D) + +Baseline reference (per design memo §1.4): **C_rag-graph** (post-α-7 baseline = pre-α-8 path with `JAMES_DISABLE_TYPED_FILTER=1` forced in sector cell). + +Verdict tree: +- `graded Δ ≥ +0.030` → ⭐⭐ adopt + ARCHITECTURE.md + finding memory promoted +- `+0.010 ≤ graded Δ < +0.030` → tier-gated adopt (which tiers per cross-tab) +- `graded Δ ≤ +0.010 OR abst_f1 < α-6 baseline` → ⭐ operational only (reject typed filter as architectural fix; heuristic top-K declared sufficient on this benchmark, memory promoted as such) + +Per design memo §1.3 honest framing: +- Even if Δ exceeds noise, finding is ⭐⭐ partial (typed filter is decades-old technique; MS GraphRAG / LlamaIndex have type hints). JAMES contribution = numbers + routing rule + empirical position. +- ⭐⭐⭐ candidate elevation requires cross-fixture sanity (multihop + Ali Track 2c crossover) — Phase D incorporates this. + +## 5. Phase 3b proper status + +Per `feedback_graph_layer_top_k_debt`: publishable graded claim requires α-7 closure → α-8 measurement → cross-family sanity. As of session close: + +- α-7 closure: ✅ REJECT +- α-8 baseline measurement: ⏳ Phase C in flight +- Cross-family (5 families × n=3 ≈ 45h compute): ⏸️ post-α-8 Phase D +- Position per design memo §1.4 + 2026-06-02 Option C decision: Phase 3b proper runs **on α-8 baseline**, not pre-α-8. + +## 6. Track 2c collaboration status (Ali) + +| Phase | Status | File | +|---|---|---| +| 1: preserved imports + integration design | ✅ PR #686 | `eval/adversarial/ar_ecommerce-{README-ali,REPORT-provia,provia-results,v1.1-pending}.{md,json,yaml}` + `docs/design/v0.4-track-2c-arabic-adversarial-integration.md` | +| 2: YAML remap (byte-exact U+202E preservation 18/18) | ✅ PR #686 | `eval/adversarial/ar_ecommerce-v1.1-james.yaml` | +| 3: adversarial_sweep.py + DSL parser (18 tests) | ✅ PR #686 | `scripts/adversarial_sweep.py` + `tests/test_adversarial_criteria_parser.py` | +| 4: JAMES M_M sweep | ✅ PR #687 | analysis: `reports/research-runs/track-2c-phase-4-jam-m_m-analysis-20260602.md` | +| 5: cross-stack report (Ali vs JAMES diff) | ⏸️ post-α-8 acceptance | | +| 6: α-8 acceptance integration (poison_01/04 retest) | ⏸️ Phase D | | +| 7: Ali cross-stack DM | ⏸️ post-Phase 6 | | + +## 7. Wrong-fix-averted ledger + +| Cycle | Count | +|---|---:| +| α-5 | 7 | +| α-6 | 2 (rate-limit + tier override) | +| α-7 | 1 (top-K rejected at 5-tier scale) | +| **Cumulative** | **10** | + +Additional category catch this session: 1 "cumulative branch contamination" (PR #681 superseded by #686) — recognised pattern; logged in PR #686 description. + +## 8. Files added this session (Where to find what) + +| Topic | Path | +|---|---| +| α-8 design memo | `docs/design/v0.4-alpha-8-ontology-typed-filter.md` (§2.4 R1-R5; §3 integration plan; §4 verdict tree) | +| α-7 closure analysis | `reports/research-runs/alpha-7-closure-analysis-20260602.md` | +| Track 2c Phase 4 analysis | `reports/research-runs/track-2c-phase-4-jam-m_m-analysis-20260602.md` | +| Track 2c Phase 4 raw JSON | `reports/research-runs/adversarial-sweep-M_M-20260602T084119.json` | +| Bidi audit | `reports/research-runs/bidi-normalization-audit-20260602.md` | +| α-7 bucket-(d) sub-finding | `reports/research-runs/alpha-7-bucket-d-oracle-phrase-gap.md` | +| α-6 recovery curve | `reports/research-runs/alpha-6-phase-3a-recovery-curve-20260601.md` (§3.1 has 12b correction annotation) | + +## 9. Memory updates this session + +New entries (4): +- `project_alpha_7_closure_state` — α-7 REJECT context +- `feedback_alpha7_top_k_destroys_positive_contributions` — wrong-knob mechanism +- `feedback_s4_citation_survives_context_reshape` — ⭐⭐⭐ candidate strengthened +- `feedback_12b_pure_capability_misclassified_as_plateau` — 12b plateau correction +- `project_alpha_8_phase_a_b_landed` — α-8 Phase A+B state (this session) + +## 10. Open decisions left for next session + +1. **Phase C completion**: verify background run completed; if partial, resume with `--resume` flag. If full, proceed to analysis. +2. **5-tier remeasurement scope**: full M_XS+M_S+M_M+M_L+M_XL × n=3 vs subset (e.g., M_M + M_XL only). Decision driver: Phase C M_M result magnitude. +3. **Phase D closure timing**: write closure analysis after 5-tier OR after just M_M+M_XL. Trade-off: completeness vs cycle velocity. +4. **Ali Phase 5 DM share**: wait for Phase 6 acceptance test (poison_01 retest at α-8) OR share Phase 4-5 results immediately for cross-stack discussion. +5. **Phase 3b proper launch position**: per design memo §1.4 Option C confirmed (post-α-8 baseline). Compute timing depends on Phase C completion. + +## 11. Don't-forget guardrails + +- **No domain features until v1.0** (CLAUDE.md rule #1) — α-8 5 new types are all horizontal (event/date/location/quantity/project); regulation/transaction/recipe deferred. +- **20 KB module gate** — `core/ontology.py` now 14.9 KB; further additions need split-first. +- **Quality Delta Card required** for `core/retrieval|graph|reasoning` PRs (CLAUDE.md rule #2). Phase B PR #689 used `Quality delta: exempt (label: code — A/B oracle infra)` — Phase D PR will land the actual Δ numbers. +- **Honest framing tier** for every finding (memory `feedback_finding_size_honest_framing`). α-8 ceiling = ⭐⭐ partial. + +## 12. Korean handover snippet + +이번 세션 (2026-06-02): α-7 REJECT 확정 (PR #680, 10번째 wrong-fix-averted) ++ α-8 Phase A+B 두 PR 한 세션 머지 (#688/#689) + Track 2c Phase 4 sweep +(0 breach, poison_01 acceptance target 명시) + 5 PR cleanup cascade +(#683/#684/#685/#682/#686/#687). + +α-8 Phase A: `core/ontology.py` 에 horizontal 5 type + 6 relation 추가, +`core/graph_typed_filter.py` 신규 모듈 + flag 추가. R1-R5 evidence-of- +absence preservation rule 모두 코드로 박힘. 137 test pass. + +α-8 Phase B: matrix cell `C_rag-ontology` + `pipeline_context` typed- +prefix overlay. Default ACTIVE in production (flag 미설정 시 filter ON). + +Phase C N=3 baseline at M_M background 실행 중 — log 위치 +`reports/research-runs/alpha8-phase-c-sanity.log`. 다음 세션 cold-start +시 완료 여부 확인 → 결과 분석 또는 재실행. + +다음 cycle 순서: Phase C M_M 분석 → 5-tier 확장 측정 → Phase D closure +analysis + ARCHITECTURE.md §5.7 → cross-family Phase 3b proper (45h) +→ v0.5 도메인 (horizontal moat = 기업 내부지식).