Skip to content

feat(window): add opt-in slice gather for relative bias - #43

Merged
noelEOS merged 5 commits into
mainfrom
window/relative-bias-slice-gather
Nov 3, 2026
Merged

noelEOS merged 5 commits into
mainfrom
window/relative-bias-slice-gather

Conversation

@noelEOS

@noelEOS noelEOS commented Nov 3, 2026 •

Copy link
Copy Markdown
Owner

Add keyword-only gather="slices" to relative-position bias and expose it through the Swin constants builders. The default remains "one_hot"; default serialized graphs stay identical. Slice gathering avoids the large one-hot intermediate and is intended for larger windows; window 7 measured slower.

Fix round 1 (b2343d8) restores the existing statuses and Swin validation records verbatim, removes the converter report additions, and keeps the four live block-parity comparisons separate from block promotion. The converter and its report expectations match main exactly.

Base revision: 2ab98b719e7c8c082681c14ad2dd380b96eab90b.

Changes by layer: window attention and Swin constants; deterministic fixtures and focused tests; generated catalogue and Swin guide.

Validation on the final head:

  • G1: 36 fixture graphs equal; 9 trained graphs not compared because checkpoints are unavailable. Baseline and final both pass.
  • G1-ext: 19/19 graphs equal, before and after.
  • Catalogue check: in sync.
  • Offline selection relative_bias_slices: all seven fixtures pass, zero cloud calls. Maximum mirror-versus-PyTorch absolute error: bias W7/W12/W14 0; unshifted block 1.7881393432617188e-7; shifted block 1.1920928955078125e-7; both strict difference fixtures 0.
  • Plan: 8 interactive requests, 0 assets, 0 tasks; unchanged from the live plan.
  • Focused changed test files: 740 passed. With timm and SMP blocked: 683 passed, 57 skipped. Full-suite CI passed all three jobs on b2343d8 (run 41698079914).
  • Conversion comparison: complete SegFormer, standard Swin and wider one-hot Swin reports are identical to main. Standard block graphs are unchanged; the wider case retains five nearest records and outside fields dim, heads and window. Converter code and its report expectations are unchanged from main.

The live selection ran once on clean revision d0f72e812797720db16acabbda0bb72f0b297c30, using exactly eight requests. Its record is committed unedited and alone. Two-head bias tables at W7/W12/W14 match PyTorch and the mirror bit for bit. Both small block paths match the mirror bit for bit and their difference is exactly zero.

Only the two-head bias lookup settings at W7/W12/W14 gain live validation. All existing statuses are preserved verbatim, with that live scope appended only to relative-position bias. Swin block/stage validation records remain exactly as on main. The four block parity fixtures retain their recorded live evidence; block-level promotion waits for a fix to validation reports' handling of tied nearest records.

The lookup fixture stores its head count in shape metadata. The validation test adapter supplies that count because the new lookup record restricts slices to two heads; this checks the narrow scope without broadening the record or editing the evidence.

The Swin guide retains qualitative guidance for large windows and removes numerical memory/latency claims without a committed measurement record. The live agreement numbers remain. No whole-model performance or model-accuracy claim is added.

@noelEOS
noelEOS marked this pull request as ready for review November 3, 2026 19:16
@noelEOS
noelEOS merged commit 127f66c into main Nov 3, 2026
3 checks passed
@noelEOS
noelEOS deleted the window/relative-bias-slice-gather branch November 3, 2026 19:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant