Skip to content

As a knowledge creator, I want Phase E and Phase D to operate more reliably so that knowledge file quality converges across fix rounds #274

@kiyotis

Description

@kiyotis

Situation

Phase E (fix) and Phase D (content check) exhibit systematic behavioral issues confirmed across nabledge-1.2, nabledge-1.3, and nabledge-1.4 generation runs. Analysis of phase-d findings logs (r1/r2/r3) across all three versions identified the same root causes repeating.

In the nabledge-1.4 run (536 files, 2 rounds):

  • r1→r2: 20 files developed new critical findings that did not exist in r1 (fix-induced regressions)
  • r1→r2: 50 files developed new fabrication findings not present in r1
  • r1 critical omission → r2 critical fabrication pattern observed in multiple files
  • r2 clean → r3 new issues appeared in 34 files

Pain

Knowledge file quality does not reliably converge after multiple Phase E rounds. Each fix round introduces new problems while resolving others, making the overall outcome unpredictable. Critical issues remain after 2 rounds not because Phase E failed to execute, but because Phase E itself generates new critical issues.

Benefit

  • Knowledge creators can trust that Phase E reduces total issues monotonically per round
  • Fix rounds converge toward a clean state rather than cycling between different problem types
  • Phase D severity judgments are consistent across rounds, enabling reliable quality gates

Success Criteria

Phase E root causes (confirmed across v1.2, v1.3, v1.4):

  • E-1: Fix the issue where Phase E modifies sections outside the scope of the fix instruction
    • No new issues appear in files that were clean in the previous round
  • E-2: Fix the issue where Phase E outputs newly generated sections without cross-referencing the source
    • The omission(critical) → fabrication(critical) problem type conversion no longer occurs
  • E-3: Fix the issue where Phase E rewrites source RST special notation and typos based on its own judgment, generating fabrications
    • Intentional non-standard notation and typos in source documents are preserved as-is

Phase E root causes (new in v1.4):

  • E-4: Fix the issue where Phase E distorts the meaning or structure of adjacent text during editing
    • Word inversions, deletions, and mistranscriptions (e.g., "normal termination" → "abnormal termination") no longer occur
  • E-5: Fix the issue where Phase E fabricates explicit rules from implicit patterns in the source
    • Rules and constraints not stated in the source are not added to knowledge files

Phase D root causes (confirmed across v1.2, v1.3, v1.4):

  • D-1: Fix the issue where Phase D assigns different severity and rationale to the same location across rounds
    • Severity judgments (critical/minor) for the same content do not reverse between rounds

Investigation and Prevention

  • Root cause identified with reproducible test
  • Issue verified as resolved in test environment
  • Horizontal check completed with method, results, and status
  • Recurrence prevention measures implemented

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions