Skip to content

perf: profile and speed up diff, patch, checkpoints and describe - #11

Merged
leofmarciano merged 1 commit into
mainfrom
devin/1793688017-profiling
Nov 3, 2026
Merged

leofmarciano merged 1 commit into
mainfrom
devin/1793688017-profiling

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

What and why

Profiled the hot paths on a 546M-parameter bf16 synthetic model (cProfile, py-spy, tracemalloc) and fixed what showed up. The new benchmarks/profile_hotpaths.py reproduces the numbers.

  • _chunks.py: block-wise passes on a small shared thread pool (NumPy drops the GIL), used by diff, patch, snapshots, restores and stats. Nothing makes a full float64 copy of a tensor anymore.
  • diff / patch / checkpoints compare weights bitwise, and the original norm for rel_delta is computed once per tensor instead of on every write.
  • Checkpoints store a sparse delta from the originals (dense only when most weights changed).
  • describe: exact quantiles from one histogram pass plus a partition of the few bins holding the wanted ranks (same values as np.quantile), one-pass moments, blocked row/col norms, Gram spectrum in the input precision. Its output is keyed by content, so a touched but unchanged tensor is not recomputed.
  • Fixes: apply_patch did nothing on non-contiguous arrays; untouched NaN weights were counted as edits.
before after
describe embedding 151936 x 1024 6,475 ms, 2,376 MB 1,241 ms, 132 MB 5.2x
describe MLP 3584 x 1024 341 ms 130 ms 2.6x
write_tensor (+ diff over 237M touched weights) 371 ms 80 ms 4.6x
ws.diff() 381 ms 81 ms 4.7x
ws.patch() 170 ms 83 ms 2.1x
ws.checkpoint() 245 ms, 1,807 MB 88 ms, 8 MB 2.8x
ws.restore() + diff 460 ms 147 ms 3.1x
first edit of the embedding 152 ms 29 ms 5.3x
agent loop, 30 steps, 60 tool calls 9,242 ms 373 ms 24.8x

Public API and describe output are unchanged (diffed the text against main on six tensor shapes: identical).

How it was tested

  • ruff check . && ruff format --check .
  • pytest (55 passed; new tests/test_perf_paths.py forces the threaded/histogram paths on small arrays and checks them against NumPy)
  • Added a CHANGELOG.md entry (if user-facing)
  • mypy src/weightspace, python -m build, twine check --strict dist/*

…ntiles; content-keyed describe cache

Co-Authored-By: Leonardo Marciano <leonardo.marciano@toolzz.me>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@leofmarciano
leofmarciano merged commit 2f4ffcf into main Nov 3, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant