Skip to content

perf: the value key holds its datatype's URI, so the garbage collector stops tracking a plain value's key - #704

Merged
trungdong merged 4 commits into
mainfrom
perf/3.3.0-value-key
Oct 29, 2026
Merged

trungdong merged 4 commits into
mainfrom
perf/3.3.0-value-key

Conversation

@trungdong

@trungdong trungdong commented Oct 29, 2026 •

Copy link
Copy Markdown
Owner

Makes every attribute value key hold its datatype's URI string instead of the datatype's QualifiedName, so a record of plain values leaves the garbage collector as little to track as it did in 3.2.3. Closes #690.

The value key is what attribute equality, record hashing and the per-attribute value set compare. Its slot 0 held the datatype's QualifiedName, which kept three tuples and three dicts per record tracked by the garbage collector, 13 tracked objects per record against 3.2.3's 7. _value_key keeps its hand-written rows, and each plain row now returns the URI string, read once into four module constants; an int's row returns int_datatype(value).uri, where int_datatype holds the size rule (xsd:int, xsd:long or xsd:integer) that canonical_xsd_datatype also uses. Literal._key reads its datatype's URI. For a value of an exact plain type the key and the value set then hold only atomic objects and CPython stops tracking them, which brings the count back to 7.0 per record.

A name compares and hashes by its URI, so keys stay equal for equal values, and equality, hashing, the value set and unpickling are unchanged; a value set pickled by 3.2.3 rebuilds its keys on load as before. test_a_plain_value_keys_as_its_twin pins each plain type against its Literal twin, the non-twins of #235, #34 and #663, the subclass rows, and the identifier and fallback rows.

One edge changes. A Literal built outside a record with a datatype that is not a qualified name, such as a bare string, now raises AttributeError when hashed or compared, where 3.2.3 returned. Asserting such a literal on a record already raised in both, and the constructor's annotation admits only a QualifiedName or None. #705 proposes a clear error at construction.

#670 proposed deriving the plain rows from the writers' table as well. That form was built and measured against this one. The two ran the same, and the derivation added a lookup map, a walk of a subclass's bases and an int fix-up to _value_key, so spec 5's F14 is amended and #670 stays open.

ab.py main --python 3.12 --passes 3 on CPython 3.12.8, garbage collection off, on a quietened machine (load average 2.3; the head against itself moved no row by more than 3.4%):

benchmark                                                      base min   head min   change
benchmarks/test_bench.py::test_construct                         0.0454     0.0447   -1.4%
benchmarks/test_bench.py::test_deserialize[json]                 0.0767     0.0756   -1.5%
benchmarks/test_bench.py::test_deserialize[jsonld]               0.0802     0.0801   -0.2%
benchmarks/test_bench.py::test_deserialize[provn]                0.2204     0.2196   -0.4%
benchmarks/test_bench.py::test_deserialize[rdf]                  0.6501     0.6548   +0.7%
benchmarks/test_bench.py::test_deserialize[xml]                  0.1326     0.1309   -1.3%
benchmarks/test_bench.py::test_equality                          0.0379     0.0373   -1.6%
benchmarks/test_bench.py::test_prov_to_graph                     0.1724     0.1677   -2.7%
benchmarks/test_bench.py::test_serialize[json]                   0.0516     0.0517   +0.1%
benchmarks/test_bench.py::test_serialize[jsonld]                 0.0555     0.0553   -0.3%
benchmarks/test_bench.py::test_serialize[provn]                  0.0278     0.0280   +0.5%
benchmarks/test_bench.py::test_serialize[rdf]                    0.7910     0.7986   +1.0%
benchmarks/test_bench.py::test_serialize[xml]                    0.1034     0.1019   -1.5%
benchmarks/test_bench.py::test_unified                           0.0580     0.0579   -0.2%

With --gc, construction is −3.9%, the PROV-JSON read −2.7% and equality −3.6%. The benchmark document has one integer per block, so documents of one plain type were timed separately (3,000 entities of three attributes): integers −2.3%, strings −1.2%, floats −1.5%.

There is no HISTORY entry, because the key of every value a record can hold is unchanged (spec 5, section 7).

@codacy-production

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 4 complexity

Metric Results
Complexity 4

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@coveralls

coveralls commented Oct 29, 2026 •

Copy link
Copy Markdown

Coverage Status

coverage: 98.405% (+0.02%) from 98.381% — perf/3.3.0-value-key into main

…set of a plain value hold only atomic objects and the garbage collector stops tracking them (spec 5, 4.3)

Every key held the datatype's QualifiedName in slot 0, which kept three tuples and three dicts per record tracked (13 GC-tracked objects per record against 3.2.3's 7). The hand-written rows of _value_key now return the URI string, read once into four constants, an int's through int_datatype, which holds the size rule canonical_xsd_datatype also uses, and Literal._key reads its datatype's URI. A name compares and hashes by its URI, so keys stay equal for equal values; a parametrised table pins each plain type against its Literal twin, the non-twins and the identifier and fallback rows.
…ws, with #670 left open (spec 5, 4.3)

PR 28 built #690 with #670's derived table and then the hand-written rows with four URI constants; on a quiet machine the two forms measured the same, inside a 2.7-point noise floor, with 7.0 GC-tracked objects per record each, and the derived table cost more readability than it bought, so the maintainer chose #690 alone. Section 4.3 describes the chosen form, F14 records the measurement, and #670 moves to Out of scope.
…t the key holds and why the hand-written rows stayed (spec 5, 4.3)

The pre-PR review found the spec's GC claim overstated for subclasses of plain types, F14's amended cell missing the reason the derived table lost, a docstring still naming the datatype where the key holds its URI, and twin-table assertions that hold for every row confined to the twin rows.
… the spec's #670 entry points at F14, and the twin test reuses its record (spec 5, 4.3)

The pre-PR maintainability pass found the int32 and int64 bounds spelled in the bounds table and again as constants below it, section 10 restating F14's ruling, and the twin test building one record twice; each now has one home.
@trungdong
trungdong force-pushed the perf/3.3.0-value-key branch from e716818 to cf3419c Compare October 29, 2026 11:08
@trungdong trungdong changed the title perf: the value key takes its plain datatypes from the writers' table and holds the datatype's URI perf: the value key holds its datatype's URI, so the garbage collector stops tracking a plain value's key Oct 29, 2026
@trungdong
trungdong merged commit afebc5f into main Oct 29, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Key attribute values by the datatype's URI string instead of its QualifiedName

2 participants