Firmware recon: extract format ground-truth from assert strings (grep-gated side bet)

[kmorrill]

Goal

Use the OP-XY firmware as a source-of-truth check on the file-format model — not a full reverse-engineering project. We are hunting maybe five short functions, not trying to understand an embedded synth.

The crash asserts we keep hitting are leaked firmware internals: they embed the source filename, line number, and literal predicate as ASCII. Counts of strings already seen across docs/:

44  num_patterns > 0
10  i < length
10  fixed_vector.h:59
 7  serialize_latest.cpp:90
 7  fixed_vector.h:77
 5  length < thesize
 2  serialize_latest.cpp:30

A file literally named serialize_latest.cpp with num_patterns > 0, plus fixed_vector.h, already ratifies the inferred paradigm (one serializer; patterns are a counted fixed-capacity vector validated on load). If we can read the neighborhood of each assert, we get the real answers to the questions where byte-inference has plateaued.

Specific questions to answer (localized, each ~one function)

• Exact RLE decoder state machine (state after an extension byte; runs > 257; scope boundaries like the unnamed-109 lane separator).
• Track-body struct field layout (engine params, sample tables, step components, p-locks).
• Event-type dispatch table for 0x1C-0x2D (crash Add read-only preset inspection and parse capability checklist #2).
• Meaning of type 0x05/0x07 + 08 00 padding (crash Firmware recon: extract format ground-truth from assert strings (grep-gated side bet) #1 root).
• The actual arithmetic behind num_patterns > 0 — turns the scene-record/tail-count crash ledger from empirical into derived.

Setup (the part that makes this high-yield vs. a tar pit)

The firmware blob is not in the repo — needs re-fetching from teenage.engineering. Prior attempt (docs/logs/2025-02-14_firmware_package_notes.md) died on a "zstd frame with very large window"; that's a packaging problem and the tooling has moved.

1. Get a symbol-rich image.
   • (a) Beat the container: zstd --long=31 --memory=2048MB, then binwalk -eM recursively (often walks past the large-window wall stock zstd hit).
   • (b) Better: skip the container — pull the running/decompressed image off the device (MTP/USB, recovery/DFU, or a vendor update log). Already-decompressed code has the symbols and dodges the problem.
2. GO / NO-GO grep gate (do this BEFORE any disassembler):

   strings -t x image | grep -E '\.cpp:|\.h:'
   

   • Clean list of source files + asserts => symbols present => worth a real session.
   • Scrambled/encrypted => STOP. Cost just jumped an order of magnitude; the RLE-codec + decoded-corpus path is strictly better.
3. Targeted disassembly only after the gate passes. Load into Ghidra/radare2, jump to each assert string's xrefs, read the handful of referencing functions. Cross-check against our decoded corpus (we already know expected behavior — reading to confirm is ~10x faster than reading to discover).

Sequencing

Complementary to, and after, the RLE codec work (see docs/state_of_understanding.md -> "Next decisive test"):

• If symbol-rich: we read the actual decoder state machine + crash arithmetic; codec becomes transcription instead of grid-search. But we'll only recognize/trust the right function because we tried to infer it first.
• If locked: we've spent only a timeboxed hour; the codec path carries the load regardless.

Hard timebox: ~1 hour, gated on the grep. Refuse to disassemble anything until strings | grep comes out clean.

Notes / caveats

• Reading firmware to understand a file format whose output you own is ordinary interoperability work; redistributing or deep-modifying TE firmware has license implications. We extract format knowledge, not ship their code.
• Background context: docs/firmware/package_notes.md, docs/logs/2025-02-14_firmware_package_notes.md, docs/debug/crashes.md.

🤖 Generated with Claude Code