Skip to content

Frame generation: CPU reference for splitting the weight blobs - #174

Merged
phantoslayer21 merged 1 commit into
mainfrom
fg/ref-blobs
Nov 14, 2026
Merged

phantoslayer21 merged 1 commit into
mainfrom
fg/ref-blobs

Conversation

@phantoslayer21

Copy link
Copy Markdown
Owner

Splitting the frame-generation weight blobs into their parts, as a CPU reference (lane h). It uses the blob layout from clean/fg/layout.py (#148).

clean/fg/reference/blobs.py

  • split_net1(layer, data, *, check=True): from one net1 blob, returns the weights as flat float16 in stored order, and the distinct biases (copy 0 of each group of 8). With check, it also verifies two things and raises BlobError otherwise: the zero area holds only +0 or -0, and all 16 copies of every bias group are bit-identical.
  • split_net2(layer, weight_data, bias_data): checks the sizes of a net2 layer's weight and bias blobs.
  • split_all(blobs): splits every layer. A missing or extra blob raises BlobError.
  • build_net1(layer, weights, bias): the inverse of split_net1, for synthetic tests.

The order of the weights inside a blob is still OPEN (FORMATS.md item 2), so they stay flat; reshaping them into the reference's canonical order waits for that.

Refs #96

Tested

  • python -m unittest clean.fg.tests.test_fg_blobs: 4 tests pass locally, on synthetic blobs of every layer (all 67 blobs at their real sizes).
    • Round trips, and bias positions checked byte by byte.
    • A -0 in the zero area is accepted; a non-zero value there is rejected.
    • A differing bias copy is rejected, but copy 0 is still taken with check=False.
    • A blob of the wrong size, and a missing or extra blob.
    • Total weights and biases equal the layout's parameter totals.
  • The whole clean/fg test set passes. The line-ending check and the public gates pass, with no hits in clean/fg.

Not tested

  • No real blobs: no file bytes are ever read into tests or the repository.

clean/fg/reference/blobs.py: blob bytes to flat float16 weights (stored order)
and distinct biases. net1: the zero area skipped and checked to be zero, copy 0
of each bias group taken and the other copies checked; net2: the weight and
bias blob sizes checked. The in-blob axis order is OPEN, so weights stay flat.

Refs #96

Signed-off-by: phantoslayer21 <156617979+phantoslayer21@users.noreply.github.com>
@phantoslayer21
phantoslayer21 marked this pull request as ready for review November 14, 2026 16:04
@phantoslayer21
phantoslayer21 merged commit 9b2b6b7 into main Nov 14, 2026
9 checks passed
@phantoslayer21
phantoslayer21 deleted the fg/ref-blobs branch November 14, 2026 16:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant