Canonical Filament core-data extraction engine (FR-045/FR-046/NFR-020)#12
Merged
Conversation
Adds the standalone one-document Filament extraction boundary and its
Python/WASM bindings, rebased onto current main. The original engine was
authored on a stale base as FR-040/FR-041; main independently reused those
numbers (object-edge vocabulary, authorable inverse edges), so this lands the
capability — which main does not yet have — under fresh IDs:
FR-040 -> FR-045 (core extraction engine)
FR-041 -> FR-046 (Python/WASM bindings)
US-015 -> US-016 (consume canonical extraction)
NFR-020 (unchanged; free on main)
CR-010 -> CR-011 (FR-006 frontmatter-status extension)
TC-633..658 -> TC-681..706
`extract_filament_core` takes one markdown document plus caller-supplied
ObjectType snapshots and returns core-data-shaped node/edge/objectType records
(Tier 1 frontmatter + Tier 2 body-extraction DSL), harvesting frontmatter
relationship sugar and body `ix://` links into deduplicated graph edges with
stable SHA-256 ids. Pure boundary: no registry, I/O, or runtime services
(NFR-020). Exposed to Python as `extract_filament_core` / `extract_core_data`.
Includes the #127 fix: a document with a complete but unparsable frontmatter
fence block now yields a `parse_failed` extraction error + `frontmatter_unparsable`
diagnostic (reaching Filament index_errors), while absent or empty/comment-only
frontmatter stays a clean skip. The parser (FR-006) is the single authority via
a new typed `FrontmatterStatus { Absent, Present, Malformed }`; parity-preserving
(frontmatter/body byte-identical).
Spec: FR-045/FR-046/US-016/NFR-020 + FR-006-AC-7 (CR-011); matrix TC-681..706 +
IC-001..003, coverage 374 -> 388. `make ci` green (fmt/lint/test/deny/audit).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CI's stable toolchain bumped to rust 1.97.0, whose clippy promotes `useless_borrows_in_formatting` under `-D warnings`. `&a[..].replace(..)` borrows an owned String — remove the `&`. Pre-existing on main; surfaced by the toolchain bump. Verified clean with rustup stable 1.97 locally (fmt/clippy/test all green). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Code-review + gap-analysis of the final (rebased) feature found several matrix TCs marked complete without in-repo backing. Resolved: - TC-703/704: backed by the corpus harness but untagged — add tracking tags. - TC-686: add Python binding tests (tests/python/test_bindings.py) exercising extract_filament_core / extract_core_data (native JSON, #127 parse_failed, alias determinism) — run by CI python.yml. - TC-690 / NFR-020-AC-1: add tests/filament_boundary_audit.rs static-inspection test asserting the extraction module reaches no PGlite/Electron/HTTP/ CloudManager/watcher/embedding surface. - TC-687 (downstream wasm), TC-688 (inspection), TC-689 (wasm compile): honestly downgraded ✅→🚧 with verification-method notes, matching main's convention for Static/Compile TCs. Adds a gap-analysis SpecReview under reviews/. Verified under rust 1.97 (clippy -D warnings, 387 tests, fmt). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Lands the standalone one-document Filament core-data extraction engine and its Python/WASM bindings — a capability
maindoes not yet have. Originally authored on a stale base as FR-040/FR-041;mainindependently reused those numbers (object-edge vocabulary, authorable inverse edges), so this rebases the engine onto currentmainunder fresh IDs.Renumbering (vs the stale branch)
What it does
extract_filament_coretakes one markdown document + caller-supplied ObjectType snapshots and returns core-data-shaped node/edge/objectType records (Tier 1 frontmatter + Tier 2 body-extraction DSL), harvesting frontmatter relationship sugar and bodyix://links into deduplicated graph edges with stable SHA-256 ids. Pure boundary — no registry, I/O, or runtime services (NFR-020). Exposed to Python asextract_filament_core/extract_core_data.Includes issue #127
A document with a complete-but-unparsable frontmatter fence block now yields a
parse_failederror +frontmatter_unparsablediagnostic (reaching Filamentindex_errors); absent or empty/comment-only frontmatter stays a clean skip. The parser (FR-006) is the single authority via a new typedFrontmatterStatus { Absent, Present, Malformed }— parity-preserving (frontmatter/body byte-identical).Verification
make cigreen locally: fmt-check, clippy-D warnings, 387 tests, deny, audit-unsafe, all 7 audit-static checks.Closes #127. Supersedes #11 (which was built on a stale base).
🤖 Generated with Claude Code