Repository navigation
Faster pickling and from_json; stop mutating inputs; bump to 1.2.0 - #20
Merged
Merged
Conversation
Coverage Report for CI Build 37522182131Coverage decreased (-1.0%) to 91.026%Details
Uncovered ChangesNo uncovered changes found. Coverage Regressions21 previously-covered lines in 3 files lost coverage.
Coverage Stats
💛 - Coveralls |
iskandr
force-pushed
the
fix-generic-docstrings
branch
from
October 6, 2026 16:56
ef8cf34 to
9ec4a04
Compare
iskandr
force-pushed
the
faster-pickle-and-decode
branch
2 times, most recently
from
October 6, 2026 19:48
5f0f0f0 to
57cc43f
Compare
iskandr
force-pushed
the
fix-generic-docstrings
branch
from
October 6, 2026 19:51
e1cf7f3 to
5eb7c8b
Compare
- Pickle through from_dict(to_dict()) instead of converting the whole object tree to its JSON-shaped representation first: pickle.dumps is ~3x and pickle.loads ~4.5x faster, and pickles are ~35% smaller. Values in to_dict() are now pickled natively. Pickles written by earlier versions still load via from_serializable_repr. - from_json reconstructs objects bottom-up in a json object_hook rather than walking the parsed tree a second time (~1.6x faster). - from_serializable_repr no longer pops keys out of its input, so decoding the same representation twice works (it used to raise IndexError), and Serializable.from_dict no longer renames aliased keys in the caller's dict. - Cache __init__ argument names per class; this runs on every default to_dict/__eq__/__hash__. Drops the __new__ fallback, which could never succeed on Python 3. - Document (README, docs/guides/objects.md) and test that keyword-only __init__ args are excluded from the default to_dict(); varcode's SpliceOutcomeSet depends on it. Add a pickling section to the objects guide. The JSON wire format is unchanged. Claude-Session: https://claude.ai/code/session_01JRDMg3F5kNKmnjv5jvQBgr
iskandr
force-pushed
the
faster-pickle-and-decode
branch
from
October 6, 2026 19:52
57cc43f to
c1d173b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #18, which is stacked on #19. Bases retarget automatically as each PR merges.
Changes
from_dict(to_dict()).__reduce__(on bothSerializableandDataclassSerializable) now returns(type(self).from_dict, (self.to_dict(),))instead of converting the whole object tree to its JSON-shaped representation first. Values into_dict()are pickled natively, so e.g. adatetimefield now pickles (it never could before).from_serializable_repr, which is unchanged (tested by re-creating the pre-1.2 format with areducer_overridepickler).Cls.from_dict.from_dict(pyensembl'sEnsemblRelease/Species) still unpickle to the same object.copy.copy()now gives a true shallow copy; it used to rebuild nested objects. No downstream code copies these objects.from_json. Objects are rebuilt bottom-up in ajson.loadsobject_hookinstead of a second walk over the parsed tree. A test checks the result matchesfrom_serializable_repr(json.loads(...)).from_serializable_reprdamaged its input. It popped keys out of the dicts it was given, so decoding the same representation twice raisedIndexError. It no longer modifies its input.Serializable.from_dictmodified the caller's dict. It renamed aliased keys in place; it now copies first.__init__argument names. This lookup runs on every defaultto_dict/__eq__/__hash__. Also drops the__new__fallback, which could never succeed on Python 3.__init__args are left out of the defaultto_dict(). varcode'sSpliceOutcomeSetdepends on this.docs/guides/objects.mdnow say the defaultto_dict()uses positional args only, and the guide has a new "Pickle objects" section with a checked example.scripts/check_docs_examples.pyruns all 6 examples, and./docs.sh(strict build plus link check) passes.The JSON wire format is unchanged: output is byte-identical against golden payloads (nested objects, sets, tuples, namedtuples, non-string keys, functions, classes, OrderedDict, dataclasses, unicode, extreme floats).
Benchmarks
5k varcode-style
Variants, best of 5, Python 3.12, compared with 1.1.1:pickle.dumpspickle.loadsfrom_jsonto_jsonhash/==Downstream
Checked pyensembl, varcode, vaxrank, mhctools and sercol for overrides of
__reduce__,from_dict,_reconstruct_nested_objectsand keyword aliases, and for keyword-only__init__args. Nothing conflicts. A test-suite run of those five repos against 1.1.1 vs this branch is in progress; I'll post the results here../lint.shand./test.shpass (57 tests).https://claude.ai/code/session_01JRDMg3F5kNKmnjv5jvQBgr