Conversation
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configuration
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
SkyeAv
added a commit
that referenced
this pull request
Oct 5, 2026
Cut 19.6.0 and bump the package version in pyproject.toml, CITATION.cff, and the uv.lock self-package entry. Minor: validate-kgx can now repair a graph in place with the new --prune (-p) flag, which rewrites the edges NDJSON keeping only strictly-valid edges via the new public biolink.prune_kgx_edges; the rewrite is atomic, byte-identical for kept lines, a no-op on a clean file, and never touches nodes (#211). Changelog: - Versioned the Unreleased section as 19.6.0 and added the #211 PR link to the --prune entry. Docs: none needed here; #211 already updated docs/cli.md. Testing: - uv run pytest -q -> 1687 passed, 52 skipped - cargo test --manifest-path rust/Cargo.toml -> 156 passed, 0 failed - cargo clippy --all-targets -- -D warnings -> clean - uv run ruff check . / ruff format --check . / pyright / uv lock --check / cargo fmt --check -> all pass
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a
--prune/-pflag tovalidate-kgxthat rewrites the edges NDJSON in place, keeping only edges that validate strictly, so a finished build can be repaired into a graph that downstream KGX consumers can ingest cleanly instead of just scoring it.Prune Core
biolink.prune_kgx_edges(edges_path, limit): one streaming pass classifies every edge againstvalidate_kgx's own strict bar (validate_recorddispatching on the record's owncategory), so a prune can never disagree with what a plain validation run reports. Real defects (non-coercible values, extras the model will never declare, rejected predicates), deliberate pending carryovers (synonym/xref/relation/provided_by, ...), and lines that are not JSON at all are all removed (malformed lines counted separately)..{name}.tmp(opened withnewline=""so kept lines stay byte-identical) and are swapped in withos.replaceonly after every line was written; a clean file is not rewritten at all (rewrittenfalse, bytes untouched), a missing edges file is reported without ever being pruned, and an unwritable destination raises with the original file intact._validate_fileextraction:validate_kgx's per-file loop is factored into a private helper with an unchanged report shape, so the agent consumer atagent.py:2035and the docs examples see identical output.before(the original file's section),after(equal to a plainvalidate_kgxrun over the rewritten file), anddropped(the removed records' problem histogram and examples), pluspruned/kept/malformed/rewrittencounts and strictok/ok_excluding_pendingverdicts over the post-prune state.CLI Flag
validate_kgx_commandgainsprune: without the flag the output is byte-identical to before; with it the command prints the nodes section, the edges section,edges: pruned N/M non-compliant recordsplus the dropped records' problems/examples, and the post-prune edges line, then gates the exit on the post-prune strict state (green exactly when the nodes are also clean).-pbinds through the parser beside--prune(pinned by aparse_argstest).Design
--pruneremoves pending carryovers too. Pending is a scoring forgiveness, not compliance, and the requirement for the final graph is zero compliance failures; on a real graph the carryovers are most of the edges (the DAKP example loses roughly 800085 of 2000085), and the flag trades them for compliance. This is stated in the help text,docs/cli.md, and the CHANGELOG so it cannot surprise silently.validate-kgx(no--prune) still crashes with a rawJSONDecodeErrortraceback on a malformed line; only the prune path handles them gracefully. A friendly report there is a follow-up.Docs
docs/cli.md: new--prune/-pflag-table row and a "Repairing a graph with--prune" section with example output.CHANGELOG.md: new## Unreleasedentry (release chore versions it, matching the feat: add a parquet source and reader #204 pattern).Testing
All testing ran on the 80-core remote box
wenceslaus(the laptop OOMs on the suite), using the repo venv against the pinnedbiolink-model 4.4.5:PYTHONUTF8=1 PYTHONPATH=src .venv/bin/python -m pytest->1687 passed, 52 skipped(full suite incl. 13 new prune tests and the e2e proof).venv/bin/python -m ruff check .->All checks passed!;ruff format --check .->101 files already formattedpyright --pythonpath .venv/bin/python->0 errors, 0 warnings, 0 informationsmkdocs build --strict-> built in 1.61srs.build_fullmap_dbredb, real six-stagebuild_pipeline): injected 2 defects + 1 pending carryover + 1 malformed line into the emitted edges file, ran the command ->edges: pruned 4/5 non-compliant records,KGX output is Biolink-compliant., exit 0; plainvalidate_kgxover the pruned pair ->1/1 valid (0 failures),oktrue, and every surviving line byte-identical to the original build.Caveats:
LANG=en_US, soPYTHONUTF8=1is required or five pre-existing fixture-decode tests fail on UTF-8 em dashes; those failures are environmental and unrelated to this change.cargois absent on the test host, so the Rust CI jobs could not run there; no file underrust/changed (git diff --name-only origin/main...HEAD), so those jobs are unaffected.