Repository navigation
Conversation
… networks The CM hands out only the Perfect World partner CDNs (dl.steam.clngaa.com, st.dl.eccdnx.com, xz.pphimalayanrt.com) to clients on mainland-Chinese networks. Those reject tokenless manifest/chunk requests from international accounts with HTTP 403, so no game could be downloaded without a proxy. Root causes and fixes: 1. GetCDNAuthToken protobuf field numbers were wrong (app_id/depot_id/host_name were 1/2/3; the protocol is depot_id/host_name/app_id). The CM returned an empty token, so the partner CDNs always 403'd. Fix the field order and thread the per-host token into manifest and chunk requests. 2. The logon response's cell_id was never saved, so GetServersForSteamPipe was called with cell_id=0. Persist it (matching the desktop client/SteamKit) and use it when requesting content servers. 3. The fallback CDN pool used cache*.steamcontent.com, which is behind DNS pollution and TLS SNI blocking on mainland networks. Fall back to the Asia-Pacific edges (dist*-hkg1 / dist*-sgp1) that are reachable from China and still honour international accounts. Also keep steam_china_only servers as a deprioritised fallback instead of discarding them, sweep every server on manifest/chunk retry, and emit per-host diagnostic logs. Verified on-device: Kaiju Princess 2 (3.21 GB) and Aokana (9.77 GB) download at 43-92 Mbps with no proxy.
The download speed setting capped concurrency at 32 workers. Measurement on a mainland-China network showed the app using only ~1.2 cores of CPU while downloading, i.e. the bottleneck is per-connection CDN throughput rather than local processing, so exposing higher worker counts lets fast devices and connections pull more bandwidth in parallel. Adds two tiers above Performance: Turbo (48) and Maximum (64). The native downloader already clamps to 64.
WinNative round-robined chunk downloads across the whole usable server list. On a mainland-China network that list mixes the reachable Asia-Pacific edges with blackholed international hosts, so a large share of workers sat in connect timeouts and effective throughput collapsed to 46-72 Mbps even though the same device and CDN can do far more (measured: the device Wi-Fi sustains 34 MB/s; Valve's own client pulls 225-373 Mbps from a single China CDN host). Port the way Valve's client and other Steam downloaders behave instead: * New `host_scheduler` module keeps a per-host EWMA throughput estimate, expands an escalating cooldown (3s/15s/60s/300s) on consecutive failures, samples unmeasured hosts once, and periodically explores the least-recently-used host so a recovered node is picked up again. * The parallel depot writer creates one scheduler per batch and asks it which host each chunk should use; retries walk to the next best *measured* host instead of the next index. The scheduler summary is logged per depot. * New "Adaptive (GameHub-style)" download speed tier (sentinel 100, 32 workers) selects the scheduler from the existing settings row. Measured on-device (Redmi K80, mainland China, no proxy, 2.1 GB download): before, round-robin 32 workers: 46-72 Mbps after, adaptive: 144 Mbps (18.05 MB/s) The logged scheduler summary shows why: the four reachable hosts scored 18.4/11.3/10.2/5.9 Mbps with zero failures, while seven blackholed hosts were cooled down after 3-26 consecutive failures each.
Three follow-ups to the adaptive scheduler, each measured on-device
(Redmi K80, mainland China, no proxy, 2.06 GiB depot):
1. Pre-flight reachability probe. The first wave of workers used to burn its
connect timeout on hosts that are DNS-poisoned or TCP-blackholed, so the
opening ~30 s ran at roughly a third of the attainable rate. Probe every
usable host once (8 parallel threads, 3 s connect timeout) before the chunk
loop and park the unreachable ones for 180 s. The probe log now names each
parked host.
2. Allocate concurrency in proportion to measured throughput. pick() now
maximises score / (in_flight + 1) instead of raw score, so a host that is
twice as fast holds roughly twice as many workers. Adaptive mode also
defaults to 64 workers, up from 32.
3. Rotate CM endpoints across reconnect attempts. nativePickCmUrl always
returned the first entry of the directory response; when that entry is
blackholed the client retried it forever and logon never completed even
though a reachable CM sat in the same response. It now sorts by
weighted_load and advances a rotation index per attempt, logging which
endpoint it picked.
Measured, same depot and network:
round-robin 32 workers 46-72 Mbps
adaptive 32 workers 144 Mbps
adaptive + probe + proportional 830 Mbps peak (1038 MB in 10 s), 2.06 GiB
completed in ~27 s
Logged proof: four hosts carried 2262 chunks with zero failures while four
others were parked by the probe before a single chunk was requested.
Three fixes for Steam logon on a mainland-China connection, all found by comparing against the official client's own logs. 1. Pick the CM by latency. connection_log.txt of a real client shows it pinging every advertised endpoint and connecting to the closest one (157 ms Shanghai instead of 276 ms Seattle), while we sorted by load and rotated. Add cm_selector: parallel probes with a 1.5 s connect timeout over the advertised ports plus 443/27018/27019, lowest RTT wins, load only breaks ties within 25 ms, failures are remembered for 120 s and a re-pick within 45 s counts as a failed connection. 2. Stop falling back to the `ext*` edge hosts. When the directory request failed the hardcoded list offered ext1-sea1/ext1-fra1 and every logon ended in SteamServerConnectFailure_t; they front the network but do not hold a session for a client that is 300 ms away. Fall back to real pool members (cmp*-hkg1, cmp*-sha1) and log the directory outcome so a fallback is visible. 3. Recover from invalidated secure storage. Clearing the app's data deletes the encrypted preferences file but leaves the AndroidKeyStore key behind, which permanently invalidates it; PrefManager turned that into a RuntimeException from every Activity.onCreate, so the app could not start at all after a data wipe. Delete the stale key and file, retry once, and degrade to plain preferences if encryption is still unavailable. Also plant Timber in release builds. It was debug-only, so a shipped APK logged nothing about logon state, which is why the CM edge-host problem stayed invisible in the field. Keeps for androidx.security.crypto/Tink are added because release builds reach them reflectively.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes Steam game downloads for international Steam accounts on mainland-China networks. Previously, the CM (Content Server Directory) returned only the Perfect World partner CDNs (
dl.steam.clngaa.com,st.dl.eccdnx.com,xz.pphimalayanrt.com) to clients on Chinese networks, and those reject tokenless manifest/chunk requests from international accounts with HTTP 403 — so no game could be downloaded without a proxy.Root causes
GetCDNAuthTokenprotobuf field numbers were wrong. The request serializedapp_id/depot_id/host_nameas fields 1/2/3, but the actual protocol (per SteamKit'sCContentServerDirectory_GetCDNAuthToken_Request) isdepot_id(1) /host_name(2) /app_id(3). The CM returned an empty token, so the partner CDNs always returned 403.The logon response's
cell_idwas never saved.GetServersForSteamPipewas always called withcell_id=0. The desktop client / SteamKit persist the logoncell_idand use it when requesting content servers.The fallback CDN pool used
cache*.steamcontent.com, which is behind DNS pollution and TLS SNI blocking on mainland networks. The Asia-Pacific edges (dist*-hkg1/dist*-sgp1) are reachable from China and still honour international accounts.Changes
pb/cmsg_client_get_cdn_auth_token.rs(new): correctedGetCDNAuthToken#1request/response protobufs.cm_client.rs: persist logoncell_id; addbuild_get_cdn_auth_token.jni.rs: use the realcell_idforGetServersForSteamPipe; add a per-hostCdnAuthTokenRefresher.depot_downloader.rs: keepsteam_china_onlyservers as deprioritised fallback; append Asia-Pacific edge fallback pool; sweep every server on manifest retry; thread the CDN auth token into manifest/chunk requests.depot_writer.rs: sweep every server on chunk retry.cdn_client.rs: expand HTTP error chains and include the HTTP status code in error strings.Verification
Verified on-device (Redmi K80, mainland China, no proxy):
Key log evidence (after fix):
All 184 Rust unit tests pass.