Skip to content

data: pin auditable MATH quality inputs - #15

Closed
0z5a wants to merge 1 commit into
feature/native-resident-colocationfrom
data/math-quality-inputs
Closed

0z5a wants to merge 1 commit into
feature/native-resident-colocationfrom
data/math-quality-inputs

Conversation

@0z5a

@0z5a 0z5a commented Oct 5, 2026

Copy link
Copy Markdown
Owner

Prepare immutable MATH inputs for the pending native recurrent RL quality experiments. The corpus now has verified source hashes, explicit answer repairs and exclusions, disjoint development/held-out partitions, and an official Ouro-Thinking tokenizer audit. This is a reconstructed public-data contract, not an exact reproduction of RLTT’s private JSONL.

Incremental draft on #14. No training runtime or reward implementation changes.

Input / check Result
Prepared train / development / MATH-500 / reserve 7,498 / 256 / 500 / 4,244
Explicit Ouro thinking training profile, cap 1,024 7,480; 18 over-cap rows recorded
MATH-500 questions or labels removed 0
Actual Dataset and canonical/missing-answer reward checks All 12,498 rows pass each check
Official-tokenizer row/profile observations 37,494
Unit, reward and evaluation tests 62 passed
Speed comparison Parent #14 Candidate Speedup
Audited quality inputs No corpus/length roster Reproducible inputs and full receipts Not a timing measurement
Full online RL Pending Pending Not measured

Regeneration is byte-identical. Source integrity, the 4 solution-hash-bound label repairs, 2 excluded train rows, 2 retained unknown difficulty labels, and all prompt length exclusions are recorded. Initial preparation/test failures are retained. Known-answer reward routing is data plumbing, not model accuracy or convergence; no GPU/model execution is claimed.

Full report: docs/math_quality_inputs.md. Raw hashes, row rosters, token-ID hashes and test receipts: benchmarks/results/math-quality-inputs/.

Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant