Skip to content

Add DirectApi::inject_rtp to feed externally reconstructed RTP packets - #1026

Open
hronro wants to merge 2 commits into
algesten:mainfrom
hronro:main
Open

Add DirectApi::inject_rtp to feed externally reconstructed RTP packets#1026
hronro wants to merge 2 commits into
algesten:mainfrom
hronro:main

Conversation

@hronro

@hronro hronro commented Aug 11, 2026

Copy link
Copy Markdown

Problem

Repair schemes str0m doesn't implement — FlexFEC (flexfec-03, RFC 8627) and RED — reconstruct whole RTP packets the receiver never saw on the wire. There is currently no way to hand one back.

Rtc::handle_input is the only entry point, and it expects an SRTP-protected packet, so a plaintext packet built by the application can't go in. That leaves only one option: turn on rtp_mode and reimplement depacketization and reordering in the application, replacing packet::{h264, h265, av1} and DepacketizingBuffer. For H.264/H.265/AV1 that's several thousand lines of code str0m already has and already tests.

Change

handle_rtp becomes a thin wrapper over a new handle_rtp_inner, which takes the plaintext payload as an Option<&[u8]>:

  • None — the existing path, unchanged. Decrypts via SRTP and rejects packets arriving before an SrtpContext exists.
  • Some — skips decryption and needs no SRTP context at all.

Everything after that point is shared: sequence-number extension, replay rejection, TWCC and NACK bookkeeping, RTX unwrapping, and depacketization. So an injected packet is indistinguishable from a received one downstream, and in particular it's registered as received, which means it stops being NACKed. That's the property that makes this useful rather than just convenient — a repair mechanism outside str0m still cooperates with the one inside it.

The public entry point is:

rtc.direct_api().inject_rtp(now, &packet);

It's on DirectApi rather than Rtc because declaring the receive stream up front already requires DirectApi::expect_stream_rx, so any caller is there anyway. Unknown SSRCs and malformed input are dropped with a trace!, matching how the receive path already treats unexpected packets.

Compatibility

Additive. The None arm of handle_rtp_inner is the old code path verbatim, so nothing changes for existing users — no behaviour change, no API break, no new dependency, and no cost when the feature isn't used.

Repair schemes str0m does not implement — FlexFEC (flexfec-03/RFC 8627) and
RED — reconstruct whole RTP packets the receiver never saw on the wire. There
was no way to hand one back: `handle_input` requires an SRTP protected packet,
so the only route was to give up sample mode and depacketize in the
application.

Split the receive path into `handle_rtp_inner`, taking the plaintext payload as
an `Option`. `None` behaves exactly as before and decrypts via SRTP; `Some`
skips decryption and needs no SRTP context. Everything after that point —
sequence number extension, replay rejection, TWCC and NACK bookkeeping, RTX
unwrapping, depacketization — is shared, so an injected packet is
indistinguishable from a received one downstream, and in particular counts as
received rather than being NACKed again.

The public entry point is on `DirectApi` rather than `Rtc`, since declaring the
receive stream up front already requires it.
@algesten

Copy link
Copy Markdown
Owner

As I understand it, the fundamental request isn’t about RED or FlexFEC. It’s that str0m has no incoming plaintext-RTP endpoint corresponding to its normal post-SRTP receive path. Is that accurate?

If so, I think the PR should motivate and document that general API directly. RED/FEC seems a bit beside the point.

It's especially strange given that this probably will land any day #982 and that I imagine we might implement FEC as well.

@xnorpx

xnorpx commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Probably be nicer to just implement flex_fec in str0m or have it injectable in some way for reconstruction.

@hronro

hronro commented Aug 12, 2026

Copy link
Copy Markdown
Author

Yes I agree. I'll rewrite the PR to motivate that API directly.

On RED: I'll drop it as an example.

By reading #982 I also found something I did wrong. As written, inject_rtp will:

  • mark the packet in the TWCC receive register, if it carries a transport-cc extension — telling the sender a packet arrived that never did;
  • fold it into the sequence register, so reported loss becomes post-repair rather than the path's;
  • add its bytes to media_bytes_rx.

The first two are the ones that matter, and the error isn't marginal by construction: every packet a repair scheme reconstructs is one the network actually dropped, so the distortion is exactly the recovery rate. That's also the signal the sender uses to decide how much repair to send, so hiding it suppresses the thing that produced the repair packets in the first place.

I don't think the conclusion is "always bypass the accounting", though, because the two legitimate uses of a plaintext ingress want opposite answers:

  • A reconstructed packet never arrived. It should reach the depacketizer and stop a pending NACK, but must not touch TWCC, loss, or byte counters.
  • A packet that genuinely arrived by another route — replaying a capture into str0m, decrypting once in a relay and fanning out to several consumers, bridging RTP in over a non-SRTP transport — should count exactly like one off the wire, or the stats lie in the other direction.

So if you'd take it, I'd rather the API say which:

rtc.direct_api().inject_rtp(now, &packet, Injected::Recovered);
rtc.direct_api().inject_rtp(now, &packet, Injected::Received);

Recovered doing what #982 does internally today, Received what this patch currently does. That gives #982's recovery path a public equivalent rather than a private one, and it's the general motivation you asked for instead of a FEC-shaped hole.

On str0m implementing FEC itself: if it does, most of my own need goes away. Two things would still sit outside it. libwebrtc's FlexFEC is draft-03, which isn't wire-compatible with the published RFC 8627, so whichever variant str0m implements the other one still exists in the wild. And the replay, relay and bridge cases above have nothing to do with FEC at all. But if you'd rather str0m not carry a general plaintext ingress, say so and I'll close this and keep it in a fork — no hard feelings.

Practically: #982 rewrites the part of session.rs this touches, so I'll wait for it to land and rebase on top rather than hand you a conflict.

The first version of this registered every injected packet as received, and the
commit message sold that as the feature. It is wrong for the case that motivated the
API.

A reconstructed packet is, by definition, one the network dropped. Registering it
updates the TWCC receive register and the register that feeds receiver-report loss --
both reported to the sender, and both what it uses to size its bitrate and its repair
overhead. So registering a recovered packet hides exactly as much loss as the repair
scheme fixed, which suppresses the signal that produced the repair data in the first
place. The error is not marginal by construction: it equals the recovery rate.

`inject_rtp` now takes `Injected`, because the two honest reasons to hand str0m
plaintext RTP want opposite accounting:

* `Received` -- the packet genuinely arrived by a route str0m did not decrypt: a
  replayed capture, a relay that decrypted once and fans out, a bridge from another
  transport. Counted exactly like one off the wire, or the statistics understate what
  arrived.
* `Recovered` -- reconstructed, and so deliberately invisible to receive accounting:
  no TWCC, no register update, no byte or packet counters. It still reaches the
  depacketizer, which is the entire point.

`Recovered` also leaves `max_rx_seq_lookup` alone, since that exists to keep SRTP's
rollover counter continuous and a packet that was never decrypted has no bearing on
it. `StreamRx::media_time_for` supplies the extended timestamp `update_register`
would otherwise have produced.

The cost of not registering is that a retransmission of a recovered packet may still
be requested and arrive. That is harmless, and was checked rather than assumed:
`DepacketizingBuffer::push_entry` drops an exactly-matching sequence number, and drops
anything at or before the last emitted.
@hronro

hronro commented Aug 12, 2026

Copy link
Copy Markdown
Author

I've pushed this as a second commit: inject_rtp now takes Injected::{Received, Recovered}, with Recovered doing what #982 does internally — reaching the depacketizer while staying out of TWCC, the receive register and the counters. There's a test that runs the same packets through both modes and asserts the accounting differs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants