Skip to content

Harden fraud validity checks and reframe K-Claims#2

Draft
Incheonkirin wants to merge 2 commits into
mainfrom
agent/k-claims-v03-learnability
Draft

Harden fraud validity checks and reframe K-Claims#2
Incheonkirin wants to merge 2 commits into
mainfrom
agent/k-claims-v03-learnability

Conversation

@Incheonkirin

@Incheonkirin Incheonkirin commented Jul 11, 2026

Copy link
Copy Markdown
Owner

What changed

  • require an explicitly configured repeated entity for entity holdout and reject unique row IDs
  • exclude configured IDs from duplicate-feature and label-conflict hashing
  • make average precision invariant to tied-score row ordering
  • recognize Korean entity-aggregate feature names for manual prior-period review
  • regenerate all audit and frozen LightGBM reports
  • reclassify K-Claims-Synth from an insurance benchmark candidate to a harness test fixture
  • add a label-honest insurance FDS rebuild plan covering selective labels, delayed outcomes, investigation capacity, and worklist metrics

Why

The previous implementation overstated three checks. ULB and BAF used unique row IDs as entity holdouts, duplicate checks included IDs and therefore hid repeated observable rows, and average precision depended on arbitrary row order when scores tied. K-Claims also used authored labels, so its model scores could validate test wiring but not insurance realism.

These corrections make the project narrower and more defensible. The harness is a shortcut-linting and holdout stress-test utility. K-Claims is a pipeline fixture. The insurance-specific next step is label-honest evaluation for the planned internal FDS rebuild, not further tuning of synthetic ROC-AUC.

Material result changes

  • AI Hub card entity holdout now uses 카드KEY; T4.2 remains PASS
  • AI Hub EFT entity holdout now uses 출금계좌일련번호; T4.2 remains PASS
  • ULB and BAF T4.2 are now INFO because no repeated entity is available
  • AI Hub EFT has 2,319,072 duplicate non-ID feature rows, 52.30% of audited rows
  • 1,399 EFT rows belong to identical non-ID feature groups with mixed labels
  • the core EFT amount-template failures remain unchanged

Validation

  • 13 unit tests pass
  • Python compilation passes
  • all 18 committed JSON artifacts parse
  • audits regenerated for AI Hub card, AI Hub EFT, ULB, BAF, and K-Claims
  • frozen LightGBM sensitivity reports regenerated
  • repository scrub found no private local paths or assistant/tool names

@Incheonkirin Incheonkirin changed the title Add learnability-gated K-Claims v0.3 Harden fraud validity checks and reframe K-Claims Jul 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant