Repository navigation
perf: track open cores in a flat table keyed by discovery key bytes - #189
Draft
danielAsaboro wants to merge 2 commits into
Draft
danielAsaboro wants to merge 2 commits into
danielAsaboro wants to merge 2 commits into
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CoreTrackerkeeps open cores in aMapkeyed by the hex discovery key, so every lookup (get,resume,opened, and eachondiscoverykeyfrom a peer) first turns the 32-byte key into a 64-character string. This replaces the map with a flat table keyed by the key bytes:cores: a dense array of the open cores. Each core stores its own index (core.trackerIndex), so removal is a swap with the last entry.slots: aUint32Arraywith open addressing and linear probing. Each slot holds an 8-bit tag from the hash in the upper bits andindex + 1in the lower 24 bits, so most mismatches are rejected without loading the core. Deletion is a backward shift, so there are no tombstones.The public surface is the same:
size, iteration,get(now a hex string or a buffer),resume/opened(the internal callers pass the buffer) andwatch.cores.mapis gone. Its only reader was an assertion intest/groups.jsthat could never fail, because it looked up a key in a map keyed by discovery key. That assertion now checkscores.get(a.discoveryKey) === null.Capacity is 2^24 open cores.
validation
test/cores.jsruns 20k random set, replace and gc operations against a referenceMap, using ids that share their first 8 bytes so the probe chains are long. It fails if deletion, the swap-remove order or the replace path is broken. Node and Bare 55/55,npm run lintclean.numbers (main → this branch, 1000 open cores)
cores.get(dk)get({ key })on an open coreMethod: both checkouts are loaded into one process with 1000 open named cores and 64 replication streams each, and 11 timed runs alternate between them. "wins" counts the paired runs where this branch was faster. Separate-process runs were too noisy on this machine to use; unchanged paths show ±15–25% in this harness, so only the first two rows are clear wins.