Repository navigation
feat(window): add opt-in slice gather for relative bias - #43
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add keyword-only
gather="slices"to relative-position bias and expose it through the Swin constants builders. The default remains"one_hot"; default serialized graphs stay identical. Slice gathering avoids the large one-hot intermediate and is intended for larger windows; window 7 measured slower.Fix round 1 (
b2343d8) restores the existing statuses and Swin validation records verbatim, removes the converter report additions, and keeps the four live block-parity comparisons separate from block promotion. The converter and its report expectations match main exactly.Base revision:
2ab98b719e7c8c082681c14ad2dd380b96eab90b.Changes by layer: window attention and Swin constants; deterministic fixtures and focused tests; generated catalogue and Swin guide.
Validation on the final head:
relative_bias_slices: all seven fixtures pass, zero cloud calls. Maximum mirror-versus-PyTorch absolute error: bias W7/W12/W14 0; unshifted block 1.7881393432617188e-7; shifted block 1.1920928955078125e-7; both strict difference fixtures 0.b2343d8(run 41698079914).The live selection ran once on clean revision
d0f72e812797720db16acabbda0bb72f0b297c30, using exactly eight requests. Its record is committed unedited and alone. Two-head bias tables at W7/W12/W14 match PyTorch and the mirror bit for bit. Both small block paths match the mirror bit for bit and their difference is exactly zero.Only the two-head bias lookup settings at W7/W12/W14 gain live validation. All existing statuses are preserved verbatim, with that live scope appended only to relative-position bias. Swin block/stage validation records remain exactly as on main. The four block parity fixtures retain their recorded live evidence; block-level promotion waits for a fix to validation reports' handling of tied nearest records.
The lookup fixture stores its head count in shape metadata. The validation test adapter supplies that count because the new lookup record restricts slices to two heads; this checks the narrow scope without broadening the record or editing the evidence.
The Swin guide retains qualitative guidance for large windows and removes numerical memory/latency claims without a committed measurement record. The live agreement numbers remain. No whole-model performance or model-accuracy claim is added.