Conversation
Loading the same file into the same space twice re-asserts every rule in it. A shared file imported transitively from more than one place gets loaded once per import path, so the duplication compounds with import depth: mm2compile ends up matching against an exponentially larger, mostly-duplicate rule set for the same statement, to the point where even a single tiny compileadd call can take minutes or exhaust the stack. Track (space, canonical-path) pairs already imported and skip the reload on a repeat import of the same file into the same space.
Collaborator
Collaborator
|
Superseded by #203 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
'import!'/3 reloads a file into a space on every call, with no check for whether that (space, file) pair was already loaded. When more than one file in the import graph imports the same shared file, that file gets loaded once per path into the same space, and each load re-asserts every clause in it again.
The duplication compounds with import depth, so a project with a few levels of shared imports ends up with an exponentially larger, mostly-duplicate clause set for the same source. Anything that walks or matches over that set pays for it, and mm2compile's case dispatch can turn a compileadd call on a single tiny, lambda-free, one-atom statement into a multi-second hang or a stack overflow, purely from how many times the shared files got pulled in.
The fix tracks which (space, canonical-path) pairs have already been imported and skips the reload on a repeat import of the same file into the same space.
I hit this while running https://github.com/MesTTo/PeTTaChainer, a PLN reasoner built on top of this runtime, whose petta_chainer.metta import graph pulls several chainer/context modules through more than one path. I reproduced it against a clean checkout of this repo with no other changes, plus a freshly built mork_ffi/MORK/PathMap stack: constructing a MeTTa reasoner and calling compileadd/add_atom on
(: a (A) (STV 1.0 1.0))didn't return within 60 seconds. Applying only this patch on the same clean checkout, nothing else changed, brought that same call under 1ms. PeTTaChainer's own test suite (tests/test_pettachainer.py, tests/test_pln_validator.py, etc.) also passes against the patched checkout.Tested on Linux (Ubuntu 26.04 LTS, x86_64), SWI-Prolog 9.3.33 with janus-swi. Haven't checked macOS or Windows.