Repository navigation
perf: Namespace's name cache holds at most 10,000 names, so a namespace that lives for the whole process stops growing with every distinct name read - #707
Merged
Conversation
…tself on a miss past that, so a lookup of a repeated name stays one dictionary hit (spec 5, 4.5) A process-lived namespace such as PROV, XSD or a user's module-level one no longer grows with every distinct name read. A miss at the limit clears the cache and refills it, so every name a lookup returns still equals, and hashes as, the one returned before.
…tences and every lookup in its bound test is used (spec 5, 4.5) The comment above the cache limit opened with a verbless fragment and left the antecedent of 'it' unclear, so it now states in two sentences what the cache holds and what a miss does when the cache is full. The bound test discarded the value of four lookups, which Codacy's Pylint reports as pointless statements, so each lookup now asserts the local part of the name it returns.
…f a rejected local part raises, and its bound test is one exact trace of the cache's contents (spec 5, 4.5) A lookup now builds the qualified name before it clears a full cache, so a local part that raises ProvExceptionInvalidQualifiedName no longer empties the cache of names that are still valid. The bound test ran the same checks over three builds that all produce an empty cache, so it is now one test on a fresh namespace that asserts the cache's contents after each lookup, which pins both the limit and the point at which the cache clears, and that a rejected local part leaves a full cache unchanged.
… so the trace ends after the name taken before the flush is looked up again (spec 5, 4.5) The test's final fill and second flush pinned a flush the trace had already pinned, so they added four lines and no coverage; a mutant that changes the flush condition from >= to > still fails the test.
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 0 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
… for the bounded namespace cache, with the measured cost of the LRU it was weighed against (spec 5, 4.5) F16 left the choice between the LRU and the flush to the maintainer on the A/B, so the decisions table now says which shipped and why: the LRU's pop and reinsert per hit measured 3.6% slower on the PROV-XML read (4.2% with GC on) in every direct run on a quiet machine, while the flush stayed inside the noise floor against main.
|
coverage: 98.418% (+0.002%) from 98.416% — perf/3.3.0-namespace-cache into main |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bounds
Namespace._cacheat 10,000 entries on every namespace, so a namespace that lives for the whole process stops growing with every distinct name read through it. Closes #694.Namespace.__getitem__memoises oneQualifiedNameper local part. A document's namespaces die with the document, butPROV,XSDand a user's module-level namespace live as long as the process, and every distinct name read through them stayed cached. The issue's reproduction (twenty documents, each with 1,000 distinctprov:andxsd:names, deleted after the read) leaves 20,085 entries inPROV._cache, 20,025 inXSD._cacheand 21.4 MB allocated onmain. On this branch each cache stays at or below 10,000 entries.A lookup that misses when the cache is full clears it and then stores the new name. The name is built before the clear, so a lookup of a local part the constructor rejects raises and leaves the cache as it was. The limit is the class constant
Namespace._CACHE_LIMIT, the same for every namespace. The benchmark's largest namespace holds 3,808 names, so an ordinary document never reaches the bound. Equality and hashing are by URI and nothing in prov relies on a name's identity, so a name looked up again after a flush equals, and hashes as, the one before it. A hit is now one dictionarygetwheremaintested membership and then indexed.An LRU on
collections.OrderedDictwas built and measured beside the flush, as spec 5's F16 set. Neither evicts on the benchmark document, and on a quiet machine the LRU's per-hit reorder cost 3.6% on the PROV-XML read (4.2% with GC on) against the flush in every direct run, so the flush ships.benchmarks/ab.py main --python 3.12reports OK on this branch, with GC off and with--gc.