Repository navigation
Description
What happened
During direct play on Android TV, a reverse proxy reload closed the open connections. Instead of resuming the media load at its byte offset, the app raised a player error and rebuilt the player from scratch. The viewer sees a buffering spinner for about 2 seconds. There were still 4.7 s of media buffered when it happened.
The client log has no progressive direct-play retry line for this failure: it goes straight from media load error to player error, then player idle, new tracks, new decoder, player ready.
Seen twice on the same device behind the same proxy: on 16 Nov around 22:15 UTC (server and proxy logs only, no client report) and on 17 Nov at 12:14 UTC (client report, server and proxy logs, below). Playback itself works fine otherwise: the same session played 13 minutes without any rebuffer before the reload.
Steps to reproduce
- Put Silo behind nginx with HTTP/2 to clients,
proxy_buffering offandworker_shutdown_timeout 30s. (Here it is BunkerWeb 1.6.17, which reloads nginx about once an hour on its own.) - On Android TV, play a movie at original quality (direct play) through that proxy and let it run a few minutes.
- Reload nginx and wait about 30 s, until the old workers are terminated and close their connections. (Both occurrences came from BunkerWeb's own hourly reload; I have not tried a manual
nginx -s reload.) - About 10 s later the app logs
media load errorthenplayer error, and the player is rebuilt with a short spinner.
Expected behavior
The media load should be resumed at its byte offset inside the loader, without a player error and without rebuilding the player, as described in #80 and implemented in PR #99 ("HTTP/2 stream resets, connection shutdowns, and premature EOF become retryable for progressive ORIGINAL_HTTP loads that have made real byte progress"). This load had made progress (1,008,199,311 bytes sent by the server) and the buffer was not empty.
Device
Android TV / Google TV
Device model and OS version
Freebox Player POP (Amlogic), Android 10 (API 29), Wi-Fi, 1920x1080 at 50 Hz
App version and build
1.2.0 (31)
Installed from
Other (not checked)
Silo server version
247fec91 (self-built image with a small unrelated local patch)
Playback details
Direct play. H.264 1920x800 SDR, E-AC-3 audio, about 9.8 Mbps (bitrate_kbps: 9814), file about 5.5 GB. Decoder OMX.amlogic.avc.decoder.awesome2. No dropped frames, no audio underruns, rebuffer_count: 0 before the error.
Diagnostics report reference
The report was sent to my own server, not to Silo Diagnostics. Excerpt below (client timestamps in UTC, info level, debug logging off).
Relevant server logs
Client report, 17 Nov (UTC):
12:01:12.043 I playback play requested (user_request)
12:01:15.845 I playback player ready
12:14:34.672 I network request completed POST /api/v2/events/ws-ticket 200
12:14:34.826 I network request completed GET /api/v2/events/ws 101
12:14:35.857 I network request completed GET /api/v2/playback/sessions/{id}/control/ws 101
12:14:35.949 I network request completed GET /api/v2/watch/{id} 200
12:14:43.159 I network request completed POST /api/v2/playback/{id}/progress 200 duration_ms=4404
12:14:43.538 I playback player stats snapshot {"bitrate_kbps": 9814, "buffered_ms": 4739, "rebuffer_count": 0, "dropped_frames": 0}
12:14:43.540 W playback media load error
12:14:43.649 E playback player error
12:14:43.653 I playback player idle
12:14:43.762 I playback tracks changed
12:14:43.770 I playback player buffering
12:14:44.698 I playback video decoder initialized {"decoder": "OMX.amlogic.avc.decoder.awesome2"}
12:14:45.308 I playback player ready
Proxy (nginx), same day, UTC:
12:11:46 reload started
12:14:03 old workers: "gracefully shutting down" (64 workers)
12:14:33 old workers terminated (worker_shutdown_timeout 30s); the events and control WebSockets of this device close at this second
The media request that was open since 12:01 has no line in the proxy access log, which fits a worker that was terminated rather than a request that completed.
Silo server, UTC:
12:14:33.201 connectEventsSocket ended, open 209 s
12:14:33.213 connectPlaybackControlSocket ended, open 800 s
12:14:35.523 getWatchState 200
12:14:43.107 direct stream ended outcome=client_gone status=206 range_start=6313 bytes_sent=1008199311
12:14:43.586 direct stream ended outcome=client_gone status=200 range_requested=false bytes_sent=5071328
12:14:43.636 direct stream ended outcome=completed status=206 range_start=5505960282 bytes_sent=191157
12:14:44.155 direct stream ended outcome=client_gone status=206 range_start=6313 bytes_sent=1139063
After the error the app opened the file again from the start (no Range) and from the tail, like a first prepare, instead of asking for the next byte.
Technical notes
Observations only, I have not read the code:
- The exception behind
media load erroris not logged at info level (the line has no attributes), so I can't tell which class it was. Logging it would help. - On another Android TV (same app build, behind the same proxy with a 10 s
send_timeoutat the time), HTTP/2 stream resets did go through the in-loader retry (progressive direct-play retry error=A: stream was reset: PROTOCOL_ERROR ... attempt=1). So the retry path works forPROTOCOL_ERRORresets, but apparently not for a connection closed when the proxy terminates its worker. On the 16 Nov occurrence the server shows the same pattern (WebSockets closed,getWatchState, file reopened from the start). - When a reload closes every connection at once, the app also reconnects both WebSockets and calls
getWatchStatewithin 2 s, and a progress POST on the API client took 4.4 s. The player error comes 10 s later, which fits the media loader only noticing when it starts reading again after its buffer drained from full. - Related: Recover progressive direct play transparently after a proxy terminates an idle HTTP stream #80 and PR feat(player): resume progressive direct play transparently after mid-stream transport failures #99 (ranged resume), PR fix(playback): bound the load-idle window so proxies stop dropping stream connections #147 (load-idle window), [bug] A second server restart in one session leaves the player frozen #586 (retry window not reset between outages), Android TV (v0.3.11): playback reliability + controls — elapsed time, pause consistency, ~10-min black-screen #70 (periodic hiccups behind a proxy). A related proposal to avoid long open media responses altogether: [proposal] Android TV direct play: use bounded range requests so proxy reloads and idle timeouts stop interrupting playback #980
AI harness
AI coding assistant, used to analyse the logs and draft this report. All logs come from my own server, proxy and device, and I checked the numbers against them.
Activity
Jycreyn commentedon Nov 17, 2026
Second device, and it points at the retry budget not being reset after a successful resume.
TCL Smart TV Pro, Android 14, app 1.2.0 (31), direct play H.264 1080p at about 7.6 Mbps, behind the same proxy. Client times in UTC.
send_timeout 60sat that point) resets the HTTP/2 stream. Nothing is visible during the pause.progressive direct-play retry error=A: stream was reset: PROTOCOL_ERROR bytes_loaded=1820490653 resume_offset=1820497135 attempt=1 retry_delay_ms=1000. The load resumes at the byte offset and playback continues. This path works.progressive direct-play retry exhausted error=A: stream was reset: CANCEL bytes_loaded=1073749057 resume_offset=2894128938 attempt=2. Thenplayer error, the player is rebuilt and the file is reopened from byte 0 (noRange) plus the tail.buffered_mswas 5944 and a spinner is visible.The second load had transferred 1 GB successfully, yet it was declared exhausted at
attempt=2on its first failure. That suggests the attempt counter and/or the cumulative window from #147 spans the whole session instead of being reset once a load makes progress. I have not read the code, so this is a guess from the logs.Two suggestions:
Rhainland commentedon Nov 18, 2026
Update: #992 is closed in favour of #993, which fixes the same retry window.
Reproduced on
origin/mainat db87f83 with a real ExoPlayer against a mock server: the stream is cut, resumes, and is cut again eleven minutes later. The second cut ends in a player error instead of resuming.Cause: a direct-play file is one Media3 load for the whole movie, and the 90 s retry window was measured from that load's first failure. In your second report the reload came eleven minutes after a resumed cut, so it was already outside the window and gave up at
attempt=2.Fix in #992: each outage gets its own retry window, so a later drop resumes at the next byte without a player error. The same test now plays to the end. Your first report, with no retry line at all, fits a TLS read failure (
SSLException) that was never retried. #992 now resumes those too, but that part is a likely explanation, not a confirmed one. Every failed direct-play load is now logged with its exception class, so a new report would show which it was. It was not tested on a TV behind a real proxy reload.AI disclosure: reproduced and fixed by Claude Code (claude-opus-5-5) at the maintainer's direction.
fix(player): start a new direct-play outage only after a real recovery