Hi — first, thank you so much for this repo. The RLE breakthrough and the decoded-image model are exactly right, and everything below builds on them. With your research i was able to build a tool that effectively 'fixes' broken song files (missing samples, etc) which is a huge gain for my workflow and I look forward to sharing it for free with the community once ive polished some of the UI stuff.
I ran your format model against a full backup of my own OP-XY: 1,039 device-saved .xy files accumulated across ~18 months of firmware versions and heavy live recording. The lab corpus parses cleanly, but real-world files broke the scanner in several ways that turned out to be undocumented format behavior, not corruption. We (me + Claude) built a resilient parser that now reads all 1,039 files; here's everything we learned, with evidence, in roughly your Settled/Believed framing.
Settled (validated across the 1,039-file corpus)
1. The track base offset is firmware-dependent. Header byte[5] (your "firmware version" bytes) maps to where track 1's struct starts:
| header[5] |
track base |
files in my corpus |
| 0x0e, 0x0f |
3933 |
~380 |
| 0x10, 0x11 |
3433 |
~430 |
| 0x13 |
3449 (your documented value) |
~230 |
The global header shrank across firmware releases. Track stride (17876) and all track-relative offsets are unchanged in every family. Older-firmware files are byte-verified: e.g. a 09 0e file walks 3933 + 16×stride cleanly to its footer.
2. Live recording grows pattern structs beyond 17876 + 12×notes. Unknown data (tick/value pairs — looks like your performance-automation lanes) is appended per pattern struct, including clone structs, observed +4 to +512 bytes each, sometimes on many tracks at once (my worst file is +126 KB over baseline). Your own corpus contains five specimens — unnamed 39/107/108/109/120 — the known automation/live-record files. A fixed-stride walk silently misaligns on these; the SIG_RE fallback isn't sufficient because…
3. …the leader signature is unreliable. 00 00 00 [00-10] ff 00 fc 00 at leader+3: (a) completely absent in 09 0e-era saves (zero matches in those files), (b) erased by ordinary Bar-menu edits on newer firmware, (c) also present on clone structs, so a raw match list mixes leaders and clones.
4. Note events: live recordings break two assumptions. Tick is a signed i32 — pickup notes played ahead of the bar store small negatives (e.g. −129). And the trailing flags bytes take arbitrary values in live takes (observed 251, 255, 118, 52…), so "flags[1] is always 0" only holds for programmed notes. Validating on flags[1]==0 rejects real leaders.
5. The preset path field at +0x453F is 48 bytes, not 64. Non-string track data begins at +0x453F+48 (first bytes there in a factory-default track: 12 00 00 00 f0 00 00 00 35 64 …). Reading 64 bytes works only because the string null-terminates early.
6. Patterns can carry per-pattern presets. Clone structs sometimes hold a different preset path (and voice table) than their track leader — pattern-level sound switching. Example from my corpus: track 2 patterns 1–3 = drum/in phase, pattern 4 = a different kit. Clone ≠ byte-copy of leader.
7. Song-table footer, fully decoded (09 11/09 13). Slots of [scene_count u8][scene_ids…][loop u16]; an empty song slot is [00][loop u16] = 3 zero bytes; the table runs to exactly EOF. Baseline = 14 slots. Older firmware footers differ (09 10: 2-byte slots, 28 B baseline; 09 0e: 5-byte slots, 50 B baseline). Parser validated on 1,205/1,213 straight-walking files across both corpora.
8. Sample-path fields truncate at 72 bytes. 242 refs in my corpus are exactly 72 chars with no terminator — the device wrote truncated (dead) paths for long filenames. Worth documenting as a device limit (71 chars + NUL).
9. Your canonical-greedy RLE claim holds at scale: all 1,039 device files round-trip byte-exact through decode→encode. (Only tool-generated files are non-canonical, consistent with your bleez note.)
How we made scanning robust (in case useful)
Per-leader validation (preset window printable, steps ≤ 64, engine ≤ 0x20, note-count sane, note events sane: note/vel ≤ 127, tick-hi ∈ {≤0x40, 0xFF}); when the straight walk breaks, uniform-cost search over candidate segmentations where cost = unexplained growth bytes (+ a penalty for garbage-looking preset fields), accepting only segmentations whose chain ends exactly at a parsing song table. Result: 1,039/1,039 files read, one unreadable track total, zero misparses against string-anchor ground truth on the hard cases. Happy to share the implementation or sample files for any of the claims above (all findings reproduce from device saves; I can dig out specific specimens per firmware family if useful).
Hi — first, thank you so much for this repo. The RLE breakthrough and the decoded-image model are exactly right, and everything below builds on them. With your research i was able to build a tool that effectively 'fixes' broken song files (missing samples, etc) which is a huge gain for my workflow and I look forward to sharing it for free with the community once ive polished some of the UI stuff.
I ran your format model against a full backup of my own OP-XY: 1,039 device-saved .xy files accumulated across ~18 months of firmware versions and heavy live recording. The lab corpus parses cleanly, but real-world files broke the scanner in several ways that turned out to be undocumented format behavior, not corruption. We (me + Claude) built a resilient parser that now reads all 1,039 files; here's everything we learned, with evidence, in roughly your Settled/Believed framing.
Settled (validated across the 1,039-file corpus)
1. The track base offset is firmware-dependent. Header byte[5] (your "firmware version" bytes) maps to where track 1's struct starts:
The global header shrank across firmware releases. Track stride (17876) and all track-relative offsets are unchanged in every family. Older-firmware files are byte-verified: e.g. a 09 0e file walks 3933 + 16×stride cleanly to its footer.
2. Live recording grows pattern structs beyond
17876 + 12×notes. Unknown data (tick/value pairs — looks like your performance-automation lanes) is appended per pattern struct, including clone structs, observed +4 to +512 bytes each, sometimes on many tracks at once (my worst file is +126 KB over baseline). Your own corpus contains five specimens —unnamed 39/107/108/109/120— the known automation/live-record files. A fixed-stride walk silently misaligns on these; theSIG_REfallback isn't sufficient because…3. …the leader signature is unreliable.
00 00 00 [00-10] ff 00 fc 00at leader+3: (a) completely absent in 09 0e-era saves (zero matches in those files), (b) erased by ordinary Bar-menu edits on newer firmware, (c) also present on clone structs, so a raw match list mixes leaders and clones.4. Note events: live recordings break two assumptions. Tick is a signed i32 — pickup notes played ahead of the bar store small negatives (e.g. −129). And the trailing flags bytes take arbitrary values in live takes (observed 251, 255, 118, 52…), so "flags[1] is always 0" only holds for programmed notes. Validating on flags[1]==0 rejects real leaders.
5. The preset path field at +0x453F is 48 bytes, not 64. Non-string track data begins at +0x453F+48 (first bytes there in a factory-default track:
12 00 00 00 f0 00 00 00 35 64 …). Reading 64 bytes works only because the string null-terminates early.6. Patterns can carry per-pattern presets. Clone structs sometimes hold a different preset path (and voice table) than their track leader — pattern-level sound switching. Example from my corpus: track 2 patterns 1–3 =
drum/in phase, pattern 4 = a different kit. Clone ≠ byte-copy of leader.7. Song-table footer, fully decoded (09 11/09 13). Slots of
[scene_count u8][scene_ids…][loop u16]; an empty song slot is[00][loop u16]= 3 zero bytes; the table runs to exactly EOF. Baseline = 14 slots. Older firmware footers differ (09 10: 2-byte slots, 28 B baseline; 09 0e: 5-byte slots, 50 B baseline). Parser validated on 1,205/1,213 straight-walking files across both corpora.8. Sample-path fields truncate at 72 bytes. 242 refs in my corpus are exactly 72 chars with no terminator — the device wrote truncated (dead) paths for long filenames. Worth documenting as a device limit (71 chars + NUL).
9. Your canonical-greedy RLE claim holds at scale: all 1,039 device files round-trip byte-exact through decode→encode. (Only tool-generated files are non-canonical, consistent with your bleez note.)
How we made scanning robust (in case useful)
Per-leader validation (preset window printable, steps ≤ 64, engine ≤ 0x20, note-count sane, note events sane: note/vel ≤ 127, tick-hi ∈ {≤0x40, 0xFF}); when the straight walk breaks, uniform-cost search over candidate segmentations where cost = unexplained growth bytes (+ a penalty for garbage-looking preset fields), accepting only segmentations whose chain ends exactly at a parsing song table. Result: 1,039/1,039 files read, one unreadable track total, zero misparses against string-anchor ground truth on the hard cases. Happy to share the implementation or sample files for any of the claims above (all findings reproduce from device saves; I can dig out specific specimens per firmware family if useful).