Use the raw FairPlay key for legacy receivers (#17)

[objevovat]

Fixes the black-screen half of #17. @3rd3 has confirmed this exact change works against an AppleTV3,2 on AirTunes/220.68.

The bug

ekey wraps the raw fpAesKey. On a legacy receiver that is the only key material it ever sees, so it decrypts the stream with that key directly. A HAP-paired receiver additionally mixes in the pair-verify secret:

SHA-512(fairplay_decrypt(ekey) || ecdh_secret)[:16]

The condition chose between them on whether a shared secret existed:

if c.PairKeys != nil && len(c.PairKeys.SharedSecret) > 0 {

But rawPairVerify stores a shared secret too, immediately under its own comment saying it does not enable HAP encryption. So the mixing fired on the legacy path as well: the sender encrypted with SHA-512(fpAesKey ‖ shared), the receiver decrypted with fpAesKey.

That matches the reported symptom exactly — pairing, /fp-setup, SETUP and RECORD all succeed, frames flow, picture stays black.

The change

Select on c.encrypted, which PairVerify sets and rawPairVerify deliberately does not. HAP-paired receivers keep exactly the bytes they get today.

This is the conditional form of 3rd3's fork patch, which reached the same key by switching unconditionally — correct for AppleTV3, but it would have dropped the mixing on HAP receivers too.

Tests

The derivation is extracted into deriveStreamMasterKey so it can be tested without a handshake:

• the four legacy/HAP × secret-present/absent combinations, including the one AppleTV3,2: FairPlay SAP m3 generation is incomplete; mirroring fails with RTSP 466 #17 was about
• a control that the two branches genuinely produce different keys, so the table above can't pass by coincidence
• a guard that fails if rawPairVerify ever starts setting c.encrypted, since that would silently switch legacy receivers back to the mixed key

I checked the tests actually catch the bug: reverting the discriminator fails legacy pairing, secret present and the branches-differ control, and nothing else.

go test ./... is green.

Not included

@3rd3 also needed video/x-raw,format=I420 added in capture.go on their XFCE/X11 setup. That looks like a separate capture-side issue, possibly already addressed by e9a395a, and I have no way to test a GStreamer pipeline against an Apple TV — so I've left it alone rather than guess.

[objevovat]

No rush on this at all — but a question that would help me, and I'd rather ask it than keep guessing.

I opened three PRs here yesterday (#22, #23, #24) without checking first whether outside contributions are actually wanted. That was the wrong order, and I'd like to correct it: is this the kind of thing you want, and in what form? A "not right now" or "smaller PRs please" is a genuinely useful answer and I won't take it badly.

Asking partly because I have more prepared and would rather not add to a queue uninvited. Some performance work on fairplaySAPHash and fairplayMD5Compress measures at roughly 2.6× on the full exchange, split into independent patches with very different review costs — one is 28 lines of GF(2⁸) multiplier tables worth about a third of the runtime, another uses SWAR byte arithmetic and would take real reviewing. I'm not going to open those unless you say you'd like them, and if you'd rather have them as an issue with a diff attached than as PRs, that's easy.

For context on why I'm in this code at all: my own FairPlay implementation now uses your fpsapDescriptorForSAP and fairplayMD5Compress directly, under LGPL-3.0 — your 29-line function replaced 7.2 MB of generated code on my side, so I deleted the generated version outright. Everything above was found while working on your algorithm, and I don't want to present any of it as independent work.

Of the three open PRs, this one is the only one where someone else has a stake: @3rd3 confirmed the change fixes their AppleTV3,2 on real hardware, which I couldn't test myself.