Repository navigation
Preserve backbone precision and extend smoke coverage - #4
Conversation
cd4e17b to
3d48583
Compare
3d48583 to
77db932
Compare
AngeLouCN
changed the title
Route text, media and prefix forwards through the existing adapter contract. Keep sequence hidden states in their native dtype and cast only the pointer readout vectors. Validate cache behavior and exact BF16 output/gradient parity; the CPU regression suite passes 303 tests.
Handle existing delimiter token schemas, expose frozen-weight precision and direct-token smoke, and reject incompatible direct-mode flags. Use tiny GLM dimensions validated for BF16 SDPA training and checkpoint reload on A100.
AngeLouCN
changed the title
77db932 to
95139d2
Compare
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Waiting for review
Waiting for review; no approval is submitted. AI semantic review has not completed. Reviewed head |
|
@copilot resolve the merge conflicts in this pull request |
Co-authored-by: ZhiningLiu1998 <26108487+ZhiningLiu1998@users.noreply.github.com>
Co-authored-by: ZhiningLiu1998 <26108487+ZhiningLiu1998@users.noreply.github.com>
Resolved — I merged |
Reconcile the adapter-forward extension with current selected-vector readout. Retain public FP32 hidden-state helpers and route CUDA graph forwards through the adapter, with precision and smoke regressions.
ZhiningLiu1998
left a comment
ZhiningLiu1998
left a comment
There was a problem hiding this comment.
Reconciled the adapter forward hook with the current FP32 public hidden-state and selected-vector contracts, including CUDA graph capture. The 228 focused checks pass (28 CUDA-dependent cases skipped locally), and both full CPU CI jobs and the site checks pass on this revision.
Ordinary decision scoring retains unused generation caches and casts the entire sequence of hidden states to FP32, although the pointer head reads only selected vectors. Route text, media and prefix execution through
BackboneAdapter.forward, retain the backbone's native hidden-state dtype, and cast only the vectors needed by the pointer head. Ordinary scoring defaults touse_cache=False; explicit prefix reuse continues to request a cache.Extend the backbone smoke CLI with separate frozen-weight precision and decision-mode options. Handle tokenizers that reuse existing delimiter tokens without requiring
jevany_token_schemametadata, reject incompatible direct-token training flags, and use a tiny GLM fixture with dimensions previously verified for BF16 SDPA on A100.Validation on the restored two-commit branch:
python -m pytest tests -m 'not server' -q -ra --strict-config --strict-markers— 432 passed, 37 skipped, 7 deselected.python -m build --no-isolation— source distribution and wheel both built successfully.Base:
mainat625aed9.Candidate-token latency optimization and its accuracy/latency results are reviewed separately in #9.