Canonical Filament extraction engine + malformed-document parse failure (#127)#11
Closed
kreneskyp wants to merge 3 commits into
Closed
Canonical Filament extraction engine + malformed-document parse failure (#127)#11kreneskyp wants to merge 3 commits into
kreneskyp wants to merge 3 commits into
Conversation
The canonical Filament extraction engine treated a document with a complete
but unparsable frontmatter fence block as a clean extraction: it returned
only a `no_frontmatter` informational diagnostic with an empty `errors` list,
so per-document parse failures never reached Filament's `index_errors` (the
MarkStale -> apply_stale path only records documents with a non-empty
`errors` list).
Make the parser the single authority for the classification and the Filament
boundary own the policy:
- `extract_frontmatter` (FR-006) now returns a typed `FrontmatterStatus`
{ Absent, Present, Malformed }. Parity-preserving: `frontmatter`/`body`
outputs are byte-identical, so the TS/Py sweep is unaffected. Empty /
whitespace / comment-only blocks (YAML null) classify as `Absent` (a clean
skip, e.g. a `---`\n\n`---` thematic-break pair), not `Malformed`.
- `extract_filament_core` reads that status on the no-frontmatter branch
(cold path only) and, for `Malformed`, emits a `parse_failed` error plus a
`frontmatter_unparsable` error-severity diagnostic so the failure reaches
index_errors.
Spec: split the FR-040 behavior bullet, add FR-040-AC-6 and FR-006-AC-7 with
a CR-010 note; matrix TC-657/TC-658 (coverage 339 -> 341). Gap-analysis
SpecReview added under reviews/.
Incidental: fix two pre-existing clippy -D warnings (extract_tier2 complex
return type -> Tier2Extraction alias; stable_id needless borrow) that a newer
stable clippy flags and that block the CI gate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Author
|
Superseded by the rebased-onto-main PR (canonical Filament extraction under FR-045/FR-046). This branch was built on a stale base (12 commits behind main, colliding FR-040/FR-041/CR-010/TC numbering) and can't merge as-is. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This branch adds the canonical one-document Filament extraction engine (FR-040/FR-041, US-015, NFR-020) and the shared graph fixture corpus, and fixes issue #127: the engine treated a malformed document as a clean extraction, so per-document parse failures never reached Filament's
index_errors.#127 fix — malformed frontmatter now surfaces a parse failure
The engine previously returned only a
no_frontmatterinformational diagnostic (emptyerrors) wheneverparse_documentyielded no frontmatter — including when a complete but unparsable---…---fence block was present. TheMarkStale → apply_stalepath only records documents with a non-emptyerrorslist, so those failures vanished.Fix — parser reports the fact, boundary owns the policy:
extract_frontmatter(FR-006) now returns a typedFrontmatterStatus { Absent, Present, Malformed }— the single authority for the classification. Parity-preserving:frontmatter/bodyoutputs are byte-identical, so the TS/Py sweep is unaffected. Empty / whitespace / comment-only blocks (YAML null) classify asAbsent(a clean skip — e.g. a---\n\n---thematic-break pair), notMalformed.extract_filament_corereads that status on the no-frontmatter branch (cold path only) and, forMalformed, emits aparse_failederror plus afrontmatter_unparsableerror-severity diagnostic so the failure reachesindex_errors.Spec & tests
reviews/(Verdict CONDITIONAL; FND-002 resolved, FND-001 accepted).Verification
make cilocally: 344 lib + all integration tests pass, clippy-D warningsclean, fmt-check / audit-unsafe / deny green, spec dogfood green, SpecReviewquire-validated.extract_tier2complex return type →Tier2Extractionalias;stable_idneedless borrow) that a newer stable clippy flags and that block the gate.Closes #127
🤖 Generated with Claude Code