Skip to content

Record: 8L Paid Prefix + Sparse Hard Blocks (1.0365)#278

Closed
nicolasdickenmann wants to merge 1 commit intoopenai:mainfrom
nicolasdickenmann:record-8l-paid-prefix-sparse-hard-blocks
Closed

Record: 8L Paid Prefix + Sparse Hard Blocks (1.0365)#278
nicolasdickenmann wants to merge 1 commit intoopenai:mainfrom
nicolasdickenmann:record-8l-paid-prefix-sparse-hard-blocks

Conversation

@nicolasdickenmann
Copy link

Summary

Adds a new 10min/16MB record submission built directly on PR #262, replacing the contiguous paid prefix with an inline-built sparse hard-block cache.

Result

  • val_bpb: 1.03647005
  • total bytes: 16,525,754
  • stride: 64 sliding-window eval

Notes

  • sparse prefix format: sparse_blocks_v1
  • block size: 256
  • selected blocks: 20,681
  • covered tokens: 5,294,336 (8.54%)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant