Repository navigation
Rust Rewrite - #255
Draft
sandorstormen wants to merge 38 commits into
Draft
Rust Rewrite#255sandorstormen wants to merge 38 commits into
sandorstormen wants to merge 38 commits into
Conversation
Brings in the in-progress Rust rewrite of mal-toolbox (maltoolbox-fileutil, maltoolbox-language, maltoolbox-model, maltoolbox-attackgraph, maltoolbox-patternfinder, maltoolbox-cli crates), previously developed in a separate toolbox-rewrite repo and verified against the real Python implementation as a golden-output oracle. See PORTING_NOTES.md for scope, architecture differences, and known upstream findings. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
1.3.1 fixes the grammar gap that made unlabeled detector context (e.g. `[tpr: 0.1]` with no preceding label) fail to parse; the four tests that were #[ignore]d for it now pass unmodified. Updates PORTING_NOTES.md accordingly (113 passing tests, 0 ignored) and syncs PR #255's description to match. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The Python oracle's _glob_assoc_traversal recomputes both its seed and every loop iteration from the same unchanging instigating_assets instead of the growing result, so the `*` operator never actually closes transitively. Rather than port that bug, feed the growing result back into the pattern traversal each iteration until a fixed point is reached.
Python's Model.to_dict writes "MAL-Toolbox Version" (hyphenated) but _from_dict reads "MAL Toolbox Version" (spaced), so the field never round-trips there. Rather than replicate that, from_dict now reads the same hyphenated key to_dict writes. Also updates PORTING_NOTES.md: moves this and the glob_assoc_traversal transitive-closure fix into a new "confirmed upstream bugs fixed, not reproduced" section, and restores the already-resolved tree-sitter-mal 1.3.1 note that was staged but not yet committed.
Adds back "abstract asset never extended" and "duplicate CIA classification", previously left out as dead weight since they don't affect compilation success or serialized output. Printed via eprintln! since this project doesn't port Python's logging/maltoolbox.yml config subsystem. The duplicate-CIA check lives in compiler::mod's visit_cias (during the raw tree-sitter walk), not in the later semantic::analyze pass, since by the time a langspec reaches that pass the risk flags are already deduplicated booleans and the repeated letter is no longer visible. Updates PORTING_NOTES.md accordingly and syncs PR #255's description, which had drifted out of sync with two earlier fixes.
mal_compiler.py's own visited_files dedup intercepts a repeated include before mal_analyzer.py's _include_stack cycle check ever runs, and that check never catches a cycle back to the root file anyway (the root is never itself pushed as an include target), so a genuine two-file mutual include silently compiles in the real implementation. Rather than reproduce that, compile_inner now tracks the active include chain separately from the set of fully-compiled files, and raises a "Include sequence contains cycle: ..." error for any real cycle - including ones that loop back to the root - while still allowing non-cyclic repeated includes (a "diamond": both B and C include A) to short-circuit as before. Flips the existing mutual_include_does_not_error test to assert rejection instead, and adds a diamond-include regression test to confirm the non-cyclic case still compiles. Updates PORTING_NOTES.md and syncs PR #255's description.
Model::remove_asset now returns a RemovedAssetSnapshot { name, lg_asset }
captured at removal time, and AttackGraph::partially_regenerate_graph's
removed_assets parameter carries one per id instead of a bare
HashSet<i64>. Every lookup that used to need model.get_asset_by_id to
still succeed for a possibly-removed id (nodes_to_be_removed,
switch_fieldname for removed_associations) now reads the snapshot
instead, so Model::remove_asset and partially_regenerate_graph can be
called in either order with no ordering contract and no efficiency
regression (same O(1) lookups, no scanning).
Flips partial_regen_removal_golden.rs to the new natural order
(remove_asset first, using its returned snapshot) as a direct regression
test; other call sites keep their existing order via a test-local
snapshot() helper since the order was never load-bearing for them.
Updates PORTING_NOTES.md: moves this out of "ported faithfully" (there's
nothing left to port faithfully - it's fixed) into the architecture
section as a solved design-level consequence of the arena/id-based graph
model, and renumbers subsequent sections.
Closes the last open item in PORTING_NOTES.md §6: attack_graph_from_dict/ attack_graph_from_file were the only unported pieces of maltoolbox/attackgraph/attackgraph.py. AttackGraph::from_dict rebuilds a graph from to_dict's output (creating nodes via the existing add_node, restoring tags/extras/children/parents), and load_from_file dispatches on file extension via maltoolbox-fileutil, same as every other load_from_file in this workspace. Unlike Python, the loaded graph never stashes `model` on itself (this port never had that field) - callers keep passing Some(&model) explicitly, same as to_dict/regenerate_graph/etc. Detectors and the deprecated ModelAsset.attack_step_nodes back-reference aren't reconstructed, matching what the Python original already effectively discards on load. Added 3 tests to test_attackgraph.rs (porting test_load_attack_graph, test_attackgraph_save_load_no_model_given, test_attackgraph_save_and_load_json_yml_model_given from the Python suite), including a real oracle round-trip against tests/testdata/attackgraph.json /.yml. 117 tests passing, 0 ignored. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Introduces python/maltoolbox-language-py and python/maltoolbox-pyo3, a new PyO3 compatibility layer that will eventually back maltoolbox's existing Python API entirely with the Rust core, with no dual-backend runtime switch. Both crates are deliberately outside the root Cargo workspace (each has its own empty [workspace] table) so the existing cargo build/test --workspace and CI are completely unaffected. Phase 1 brings maltoolbox.language.languagegraph's LanguageGraph, LanguageGraphAsset, LanguageGraphAttackStep, and LanguageGraphAssociation/Field to full parity, including the separate MalCompilerError/malAnalyzerException exception hierarchies used by the MAL compiler. Verified via an oracle diff against the real pure-Python implementation's _to_dict() output (exact match) and the existing test suite (full pass, no regressions). Also rewrites tests/conftest.py's dummy_lang_graph fixture to compile a small real .mal spec (tests/testdata/dummy_lang.mal) instead of hand-building LanguageGraphAsset/LanguageGraphAttackStep dataclasses with no backing graph - incompatible with the bindings' owner+id handle design, since an AttackStepId only means something relative to one specific graph's arena. See PYTHON_BINDINGS_IMPLEMENTATION.md for the full plan, the architecture decisions (container/handle pattern, dict/set-keying via value-based hash/eq instead of a handle cache, why mal-simulator's main branch is the acceptance gate), and live status tracking. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Introduces python/maltoolbox-model-py, bringing maltoolbox.model's Model and ModelAsset to full parity, including the ModelException hierarchy (reusing maltoolbox-language-py's MalToolboxException via a direct crate dependency, mirroring the same relationship the core maltoolbox-model crate already has with maltoolbox-language). Model holds both the core Rc<RefCell<Model>> and the actual Py<PyLanguageGraph> object passed to its constructor, so model.lang_graph preserves Python object identity across repeated access. ModelAsset is an owner+id handle with no freestanding constructor, following the same convention as Phase 1's LanguageGraphAsset/LanguageGraphAttackStep. Renames RemovedAssetSnapshot to AssetSnapshot and extends it with a final_state: ModelAsset field, captured in Model::remove_asset right after its associated-assets cleanup loop runs (i.e. the post-cleanup, pre-deletion state). This lets PyModel keep a read-only tombstone per removed asset id, so a removed ModelAsset's handle stays readable afterward - matching Python, where the object survives removal as a plain, independent reference - without restructuring Model's internals or touching how maltoolbox-attackgraph's partial regeneration already consumes this snapshot. Also rewrites tests/test_model.py's test_model_remove_nonexisting_asset to construct the "foreign" asset via a second real Model instance rather than a detached ModelAsset(...) construction with no backing Model - the real constructor takes no model argument at all, the same incompatibility with the owner+id handle design that Phase 1 hit for LanguageGraphAsset/LanguageGraphAttackStep. Verified via an oracle diff against the real pure-Python implementation (exact _to_dict()/to_dict() match), which also surfaced a distinct, previously undocumented Python bug: to_dict() always writes the live package version into "MAL-Toolbox Version", ignoring self.maltoolbox_version entirely. See PYTHON_BINDINGS_IMPLEMENTATION.md for the full decision log and verification detail. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ctor compat layer
New python/maltoolbox-attackgraph-py crate wraps maltoolbox-attackgraph
(core) to mirror maltoolbox/attackgraph/{attackgraph,node,detector}.py:
- PyAttackGraph: container (Rc<RefCell<AttackGraph>>) plus identity-
preserving lang_graph_py/model_py back-references (the latter doesn't
exist in the core crate at all - compat-layer-only, matching
PORTING_NOTES.md's documented AttackGraph/Model design split).
- PyAttackGraphNode: owner+id handle, but keyed on Py<PyAttackGraph>
rather than a bare Rc<RefCell<_>> (Phase 3 decision 3) - this is what
makes pickling fall out almost for free: a node's __reduce__ delegates
to (owner_py, id), and since owner_py is a real identity-stable Python
object, pickle's own memoization correctly shares a single AttackGraph
across multiple pickled references (graph itself + individual pickled
nodes) within one combined pickle call, verified directly.
- Detectors (Phase 3 decision 1): confirmed via a fresh mal-simulator
clone that both AttackGraphNode.detectors (dict) and
AttackGraph.detectors (list) are mutated directly by real downstream
tests and must persist independently, unsynchronized with each other -
exactly matching Python's real (redundant) behavior. Implemented as
compat-layer-owned, lazily-seeded Py<PyDict>/Py<PyList> containers,
never re-derived from the Rust core after first access.
- AttackGraph.nodes (Phase 3 decision 2): a custom lazy Mapping-protocol
view (PyAttackGraphNodesView) instead of rebuilding a full dict every
access, since mal-simulator indexes it (`.nodes[id]`) in per-step hot
loops - confirmed via grep, not assumed.
- PyDetector: a real, freely-constructible value type (mal-simulator
imports and constructs Detector directly in its own tests).
- Pickling closed out for all three container layers (Phase 2 decision 3
deferred this here): LanguageGraph/Model gained __getstate__-equivalent
__reduce__ support via _to_dict/_from_dict, AttackGraph bundles both
nested layers' state so a pickled graph needs no files on disk to
restore.
- AttackGraphException hierarchy, reusing MalToolboxException the same
way ModelException does.
Two small opportunistic core changes, both additive/zero-behavior-change:
node_to_dict made pub (crates/maltoolbox-attackgraph), and
module = "maltoolbox._native" added to PyLanguageGraph/PyModel's
pyclass attributes (needed so their pickled state resolves correctly).
Verified: 117 workspace tests unaffected, 107 pytest unaffected (no .py
file touched), oracle-diffed against the untouched pure-Python
implementation (structurally, since fresh node-id numbering is a
pre-existing, non-reproducible-byte-for-byte quirk of the core's
HashMap-based generation order - logged, not a regression), live
detector mutation reproduced against the real detector_lang.mal fixture,
and pickling identity-sharing reproduced directly (pickling
(attack_graph, node) together and confirming the restored node's owner
`is` the restored graph).
Logged, non-blocking gaps carried into Phase 4: no post-removal
tombstone for AttackGraphNode (no reproducing test today),
additive_model_effects/subtractive_model_effects raise
NotImplementedError when non-empty, __deepcopy__ not implemented, and
ExpressionsChain still has no wrapper class.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…aph/AttackGraph) Replaces maltoolbox's pure-Python Model/LanguageGraph/AttackGraph classes with thin re-export shims over the Rust-backed maltoolbox._native extension, completing the PyO3 bindings effort (Phases 0-3 landed the native crates; this phase wires them into the public package and deletes the superseded pure-Python logic). Also retrofits a per-owner handle identity cache (node_a is node_b now holds within one AttackGraph, matching mal-simulator's reliance on real object identity, not just equality), closes several gaps found only once real tests ran against the native bindings (live-mutable AttackGraphNode.extras/detectors, LanguageGraphAttackStep.detectors, detached AttackGraphNode construction, AttackGraph(None), pickling for LanguageGraphAsset/AttackStep, real __deepcopy__), and fixes a handful of HashMap-iteration-order determinism bugs in the core crates exposed by the switch to real test coverage. Full verification bar green: 99 passed, 3 deselected, 0 failed; ruff/mypy clean. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ed wheel CI) Switches the build backend from setuptools to maturin, with python-source wiring that lets uv sync/uv run build the native extension directly into maltoolbox/ - no more manual .so copy after every Rust change. Rewrites the PyPI publish workflow to use PyO3/maturin-action for cross-compiled wheels (manylinux/musllinux x86_64+aarch64, Windows x64, macOS Intel+Apple Silicon) instead of cibuildwheel, following current PyO3/maturin ecosystem best practices: OIDC trusted publishing via gh-action-pypi-publish, TestPyPI gating the real PyPI publish, and a GitHub-native build-provenance attestation on release artifacts. Deletes the stale nested python/maltoolbox-pyo3/pyproject.toml, confirmed by hand to silently produce a wrong, misnamed wheel when maturin is invoked with --manifest-path pointing directly at that crate. Also fixes a leftover Phase 4 HashMap->IndexMap widening gap in test_partial_regeneration.rs that broke `cargo test`/`cargo clippy` (though not this repo's ruff/mypy/pytest-only CI), plus one unrelated pre-existing clippy warning. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Both getters previously rebuilt a brand-new PySet from scratch on every access, hashing each element through the FFI boundary every time. Malsim's hot loop reads .parents twice per traversability check, so a 5000-step run paid millions of these full reconstructions. Lazily build the (children, parents) pair once per node and cache it on PyAttackGraph (mirroring the existing node_detectors/node_extras pattern), invalidating the whole cache only on structural mutation (add_node/remove_node/regenerate_graph/partially_regenerate_graph) - never on unrelated per-node state changes. Measured ~2.8x faster per access (351ns -> 127ns) for ~480 bytes/node of additional resident memory, paid once per node rather than per call. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Ran mal-simulator's rust-backed branch test suite against this repo's Rust-backed wheel and fixed the real bugs it surfaced: a pickling crash on any graph with linked nodes, a missing language_graph_model_effect shim module, existence_status/.parents/.children not settable on attached nodes (plus a cache-eviction bug that silently discarded direct edge-set mutations), no weakref support on AttackGraph (which uncovered a GC reference-cycle leak), and dropped git-URL language loading. Two gaps are deferred and documented as open items rather than fixed: a ModelAsset-resolution limitation when a graph has no backing Model (test_defend_compromised_defender), and a DynaMAL dynamic-model-effects asset-id bug (test_dyna_mal_simulator.py). A third failure category (test_event_logger.py's fnr: detector key) is a pre-existing grammar gap unrelated to this port, confirmed against the released PyPI 2.11.0 package. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
new_associations resolved an association's opposite fieldname via a plain switch_fieldname(model, left_id, ...), which requires the asset to still exist in model - but a DynaMAL trigger model effect can add a new association referencing an asset while removing that same asset in the same batch. removed_associations already handled this correctly via switch_fieldname_possibly_removed; new_associations never got the same treatment (a Rust-port-only asymmetry, since Python's original operates on live object references and never needed two variants). Fixes mal-simulator's test_easy_ransomware_lang_attack_and_reset. Also revisits and corrects this repo's Phase 6 writeup: an earlier pass wrongly claimed mal-simulator's fnr: detector-rate failure was "confirmed identical against PyPI 2.11.0" - it isn't, PyPI silently drops the malformed field instead of failing. A real grammar/compiler fix for fnr: was built and verified working this round but deliberately not landed here, since the user resolved the underlying gap directly in mal-simulator's own test suite instead. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…MAL gate Revisiting Phase 6 to close out mal-simulator's test_different_attackers found the prior "12 failures, out of scope" diagnosis was wrong about where the bug actually lived: 1. generate.rs's create_nodes_for panicked on an asset added and removed in the same partially_regenerate_graph batch (same shape as the already-fixed new_associations bug, never applied to node creation). 2. The PyO3 binding's node tombstone mechanism covered scalar fields but not .children/.parents/.detectors/.full_name/__repr__, so a node held across its own removal (a real, supported pattern) couldn't resolve these. Both fixed and verified: cargo test --workspace 119 passed/0 failed, pytest tests -m "not integration" 100 passed/3 deselected, ruff/mypy clean. Fixing these unmasked a third, genuine race in mal-simulator's own DynaMalSimulator (same-step action-ordering vs. DynaMAL node removal) - left undisturbed, documented as open item C with a handoff summary for a future session, since fixing it requires editing mal-simulator itself. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Pylance and mypy had no visibility into the compiled PyO3 extension's API surface. Add a stub-only package tree covering every class, exception, and the create_attack_graph factory, including the language.compiler.exceptions/mal_analyzer submodules injected into sys.modules at runtime. Ship the stubs in built wheels via maturin's include list, and widen one type: ignore that the new, more precise typing for LanguageGraph.load_from_file now surfaces. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
mypy's "exclude" for maltoolbox/translators only suppresses error reporting when the excluded file is reached through the same editable source tree used elsewhere; a real (non-editable) pip install - as CI's build job does - creates a second copy of the package, so the exclusion doesn't carry over and translators/updater.py's Model._from_dict call gets fully checked. The stub was missing that staticmethod, so CI failed on type[Model] has no attribute "_from_dict" while local uv-based runs (editable install) never surfaced it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The PEP 561 stub directory was named maltoolbox/_native/, colliding with the real compiled extension's module name. With no __init__.py, Python could resolve it as a namespace package (origin: None) instead of the real .so whenever the checkout didn't also contain the compiled artifact in-tree - which is exactly what happened in CI, since pytest's pythonpath=['.'] resolves maltoolbox from the bare source checkout and `pip install .` never places the extension there. Rename the stub directory to the PEP 561 `-stubs` convention so it can never shadow the real module, and switch CI to an editable install so the compiled extension actually lands in the source tree where pythonpath=['.'] expects it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
maturin-action --find-interpreter builds a wheel per interpreter found in the manylinux image, so dist/*.whl expanded to wheels for every CPython tag; pip installed the first and then errored on the next mismatched tag. Let pip resolve the matching wheel via --find-links instead.
--no-index blocked pip from reaching PyPI for mal-toolbox's own dependencies (e.g. graphviz), since only the mal-toolbox wheel itself lives in the local dist/ dir. Keeping --find-links dist without --no-index still prefers the local wheel while letting transitive deps resolve normally. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
PyO3 Bindings to Rust Implementation
Contributor
Author
Upstream bugs that are fixed in this implementation:
|
Contributor
Author
|
Notes from LLM implementation. |
The PyO3 port left dense, narrative comments throughout (internal "Phase N decision M" planning references, porting/debugging history, multi-paragraph doc comments on simple items). Compress these to short, idiomatic rustdoc stating current behavior/invariants, matching the project's existing docstring-over-comment convention. The detailed rationale already lives in PORTING_NOTES.md and PYTHON_BINDINGS_IMPLEMENTATION.md; load-bearing invariants (handle caching, cache-invalidation rules, JSON key coercion, tombstone semantics, etc.) are kept, just compressed. Comments/docs only, no behavior change. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ates Mechanical formatting-only pass so the new rustfmt CI check starts green. Covers the root workspace plus the four standalone PyO3 workspaces under python/ (each has its own Cargo.toml and wasn't reached by a single root-level cargo fmt). No behavior change; verified cargo build/test still pass after reformatting. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Matrixed over the top-level Cargo workspace and the four standalone PyO3 crates under python/ (each is its own workspace, so a single root-level cargo fmt/build doesn't reach them). Runs on push to main and on pull requests. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Clarifies that the directory holds the PyO3 language-binding crates, not arbitrary Python code, and removes the now-stale phase-by-phase porting log. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
These two edits were made after git mv staged the rename but before the commit, and never got re-staged. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…d maltoolbox/ to python/maltoolbox/ - Move whole-stack detector tests (model/attack-graph construction) from maltoolbox-language into maltoolbox-attackgraph's test suite, removing maltoolbox-language's backwards dev-dependencies on maltoolbox-model and maltoolbox-attackgraph. - Consolidate the four independent py-bindings/*-py Cargo workspaces into one shared workspace (py-bindings/Cargo.toml), cutting four Cargo.lock files and five CI matrix entries down to one each. - Mark all core and py-bindings crates publish = false to make publishing intent explicit. - Rename maltoolbox-fileutil to maltoolbox-io. - Rename crates/ to core/, and maltoolbox/ to python/maltoolbox/, updating every path reference across Cargo manifests, pyproject.toml, CI workflows, .gitignore, README.md and CLAUDE.md. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Factor the node_edges_cache field's nested type into a NodeEdgesCache alias, drop a redundant tuple type annotation that clippy flagged as overly complex, and replace a redundant closure with the associated function it wrapped. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
sandorstormen
marked this pull request as draft
October 7, 2026 07:18
pyo3 pyclasses from a shared dependency crate get a separate, unrelated type object in every cdylib that statically links it, so consumers like mal-simulator's malsim-pyo3 can't downcast a maltoolbox._native.AttackGraph directly. A PyCapsule wrapping the shared Rc<RefCell<AttackGraph>> gives them a safe way to recover the underlying pointer across the module boundary. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
sandorstormen
added a commit
to mal-lang/mal-simulator
that referenced
this pull request
Oct 8, 2026
Adds a rust.yml workflow (fmt + build, matrixed over the root and py-bindings cargo workspaces) and switches the PyPI publish workflow to maturin-action's multi-platform wheel matrix with TestPyPI gating PyPI, matching mal-lang/mal-toolbox#255. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0116xUQqCURws4Hk4w4cBdKo
Mirrors PyAttackGraph::__inner_capsule__ (c854d1d): a pyo3 pyclass from this shared dependency crate gets a separate, unrelated type object in every cdylib that statically links it, so consumers like mal-simulator's malsim-pyo3 can't downcast a maltoolbox._native.Model directly. A PyCapsule wrapping the shared Rc<RefCell<Model>> gives them a safe way to recover the underlying pointer across the module boundary - needed for mal-simulator's Phase B (DynaMalSimulator port), which mutates the live Model at runtime via model effects and needs both the Python and Rust sides to observe the same mutations. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…them Type checkers only resolve a submodule's stubs from a same-named .pyi file or directory next to it; the PEP 561 '-stubs' suffix applies to top-level distributions only, so maltoolbox/_native-stubs/ was never read and every native type resolved to Unknown in pyright/Pylance. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
Improvement suggestions:
|
assoc_left_assets found the parents of a changed association by following reverse_expr_chain's output, which keeps a SubType filter but applies it to the far end of the reversed hop. For `io[File].xattrs` that filtered the FileDescriptors walked back to by "is a File", so the parent was never marked affected and its new child was never linked. Walk the forward chain backwards instead (follow_reverse_expr_chain), filtering the starting assets of a reversed SubType hop by its subtype. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Partial regeneration spends most of its time on hash-set inserts while reconciling node children after an association changes. Switch the hot collections to rustc-hash's FxHash (keys are trusted ids/names, so SipHash's DoS resistance buys nothing): - AttackGraphNode.children/parents, the follow_*_expr_chain sets and the partially_generate internals (incl. AssocAffectedDict) - Model.assets / name_to_asset_id - AttackGraph.id_to_node / full_name_to_node and LanguageGraphAsset.attack_steps, via IndexMap with FxBuildHasher so insertion order (and node-id assignment) stays deterministic correct_node_children_on_modified_assoc also reconciles children in place instead of cloning the whole child set to diff against. ~22% faster per partially_regenerate_graph call on a 300-host coreLang model (~41k nodes). Signatures downstream crates name explicitly (partially_regenerate_graph, associated_assets, detector context) are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop the RefCell wrapper and the Clone derive on LanguageGraph: the graph is never mutated after construction, so PyLanguageGraph and Model can share one Rc. LanguageGraph.regenerate_graph() is now a deprecated no-op that emits a DeprecationWarning. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rewrites mal-toolbox's three core layers (language graph, model, attack graph) in Rust (crates/), then adds a PyO3 compatibility layer (python/) so the existing maltoolbox Python package becomes a thin wrapper over the Rust core — the public Python API (classes, methods, exceptions, pickling, dict shapes) is preserved exactly, verified against mal-simulator's own test suite as the acceptance gate.
Architecture
Breaking changes