Skip to content

perf: record construction, unified() and add_namespace get faster, with output unchanged (3.3.0 spec 4, PR 24b) - #699

Merged
trungdong merged 8 commits into
mainfrom
feat/3.3.0-model-speed
Oct 28, 2026
Merged

trungdong merged 8 commits into
mainfrom
feat/3.3.0-model-speed

Conversation

@trungdong

@trungdong trungdong commented Oct 28, 2026 •

Copy link
Copy Markdown
Owner

Three speed changes to the model from the 3.3.0 performance spec (spec 4, section 3.2), with every output, error and warning unchanged except one case spec 4 rules on (S15). Against main, unified is 22.7% faster, construct 12.5%, prov_to_graph 11.9%, deserialize[json] 7.2%, deserialize[jsonld] 6.9% and deserialize[xml] 5.5%, and every other benchmark case is within −2.7% and +0.6%.

  • Record construction. _value_key tests Identifier first, since identifiers are the commonest attribute value and the type subclasses none of the others. add_attributes scans a call's attributes for prov:collection only when a single-valued attribute repeats, at most once per call, where it used to scan every call up front. A resolved qualified name is returned without the datetime and None checks it can never fail. The repeated-value comparison moves into a small helper, _check_repeated_value.
  • unified(). The merge map is keyed by each record's id(), so finding a record's merge no longer hashes and compares the whole record. Every copy of an equal record still merges, including two equal copies sharing an identifier with a third record.
  • add_namespace. An equal registered namespace is found by one lookup under its prefix instead of a scan of every namespace, so declaring many namespaces takes linear time (8,000 namespaces: 1.75 s on main, 0.01 s here). For an equal namespace it now returns the registered namespace, as its docstring says, rather than the argument; the two are equal (S16).

S15: on main, add_attributes compared every attribute name with prov:collection before storing anything, so a name object whose __eq__ raises made the call raise up front. It now raises only when a single-valued attribute repeats. Names from the readers and the API are strings or qualified names, so nothing they accept changes.

Why: these are the model-side changes the performance study measured as behaviour-free, after the writer-side changes of #698.

How it was checked: the output-identity script reads every fixture under src/prov/tests and writes the benchmark document, the example documents and every fixture document in each format. It then reads each one back and compares unified(), flattened(), document equality and prov_to_dot against main under PYTHONHASHSEED 1 and 2, and finds no difference. A probe also confirmed that a document repeating one formal attribute thousands of times reads as fast as on main (8,000 repeated prov:activity elements in PROV-XML: 0.024 s on both). New cases pin the prov:collection rule with the collection after the repeat and with a one-shot iterator, a single scan per call, two equal copies plus a third record merging into one, and add_namespace returning the registered namespace for an equal one without registering anything. Spec 4 records S15, S16 and the wording this PR needed. No HISTORY entry, since output is unchanged and add_namespace now returns what its docstring promised.

…n test only when a single-valued attribute repeats, so an attribute name whose __eq__ raises no longer raises before anything is stored (spec 4, T1.2)
…ypes, scans a call's attributes for prov:collection only when a single-valued attribute repeats, and returns a resolved qualified name without the datetime and None checks it can never fail; output is unchanged (spec 4, T1.2)
…a record's merge no longer hashes and compares the record; every copy of an equal record still merges (spec 4, T1.6)
… instead of scanning every namespace, so declaring many namespaces takes linear time; the result is unchanged (spec 4, T1.7)
…ibutes for prov:collection at most once, after the first repeat of a single-valued attribute, so a document repeating one many times reads in linear time; spec 4 says so and states S15 in plain sentences, and two cosmetic fixes in add_namespace and its test (spec 4, T1.2, T1.7)
…on scan and a helper compares and raises, a qualified name's None check sits in its branch, a test pins one scan per call, and spec 4 states S15 in sections 1 and 4, names the function A3 changes and what 24b pins (spec 4, T1.2, T1.7)
…ult in each branch, a comment that no longer held goes, the membership test names its three cases with one parameter, and the namespace test reuses its renamed namespace (spec 4, T1.2, T1.7)
@codacy-production

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 5 complexity

Metric Results
Complexity 5

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@coveralls

coveralls commented Oct 28, 2026 •

Copy link
Copy Markdown

Coverage Status

coverage: 98.381% (-0.02%) from 98.401% — feat/3.3.0-model-speed into main

… as its docstring says, instead of the argument; spec 4 records the ruling as S16 (spec 4, T1.7)

The two namespaces are equal, so no output changes. Names minted from the returned namespace now take _reconcile_qualified_name's identity shortcut.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants