Skip to content

Feat: Leave a shaped path out of the session - #103

Open
b-re-w wants to merge 3 commits into
developfrom
feat/link-drop-shaped-path
Open

b-re-w wants to merge 3 commits into
developfrom
feat/link-drop-shaped-path

Conversation

@b-re-w

@b-re-w b-re-w commented Nov 8, 2026 •

Copy link
Copy Markdown
Member
experiment: link-drop-shaped-path
claim: none in docs/INTENT.md yet; tests the hypothesis below
hypothesis: A punched path whose rate in either direction is under 5% of the best path of the same punch should be left out of the session. Doing so raises the session download rate and lowers small-message latency under bulk on a campus whose inbound UDP is shaped.
status: live A/B done, rule not triggered
verdict: inconclusive
metrics:
  transfer_mbps: 25.96 down and 13.04 up (arm B), 26.31 down and 19.79 up (arm A), 100 MiB each
  rtt_ms: 193.5 idle, 873 median under 100 MiB upload (arm B), 646 (arm A)
  setup_s: 28.2 (arm B), 28.5 (arm A)

Conclusion

Inconclusive. On kaggle_60 from this campus on 2026-11-08 the UDP path was not shaped: all 5 punched paths (4 TCP, 1 UDP at 199 ms) passed the check in every connect, and the rule with ratio 0.05 dropped nothing. Arm B therefore behaved like arm A, and the A/B cannot say whether dropping a shaped path helps. The rule is tested on loopback only. It needs a run on a day when the UDP path is at 2% to 10% of the best path, which PR #93 saw at about 0.11 MiB/s.

Premises checked in the code

  1. The path manager floor is never set by letify. ManagerConfig::floor defaults to None, and manager_config in letify-core/letify-link/src/sidecar.rs only reads start_timeout_ms. So the "Below the floor" rule of docs/SPEC.md (Paths and the path manager, item 6) is off for every letify session. True.
  2. The per-path policing check is Policed only under floor_bps / 20. POLICE in letify/transport/linkchannel.py has floor_bps = 1 MiB/s, so the limit is 52 KiB/s. A path at about 115 KiB/s passes as Usable, and a Usable path joins the session. True by the arithmetic of policing::check. The 115 KiB/s figure is from the PR A UDP path presenting as HTTP/3 escapes UDP shaping on a campus network #93 observation and was not remeasured here.

Change

  • policing::shaped_out(rates, ratio) returns which paths of one set are under ratio times the best path in up or in down. The path with the highest share of the best in its weaker direction is never dropped, so a punch cannot lose its only usable path.
  • The punch sidecar op takes police.shaped_ratio (optional, absent means off). The initiator collects the measured rates of all usable paths, applies the rule, closes the shaped ones and reports them in policed and as path_policed events with reason shaped (the existing drops now carry reason policed).
  • letify passes shaped_ratio 0.05 (POLICE in letify/transport/linkchannel.py). To toggle per connect for the A/B, set linkchannel.POLICE without the key (arm A) or with it (arm B).
  • Spec: docs/SPEC.md, NAT traversal item 4 (Shaped paths), Sidecar interface (police, path_policed).

Deviation from the request

The rule uses the one burst measurement of the policing check (500 ms up and 500 ms down over bounded frames), not the manager's confirmation rule of 3 measurements 10 s apart. The check runs before the session exists, on a stream that is not reusable after the check, so a second measurement would need a second check per path. If the live A/B shows a good path dropped by one noisy burst, the next step is a repeated check, as its own branch.

Tests (red first)

  • tests/nat.rs: three loopback paths limited to 2 MiB/s, 2 MiB/s and 40 KiB/s (about 2%) with the real check; only the slow one is shaped out. A path at 50% is kept. A path shaped in one direction only is dropped. The best path is kept at any ratio, a lone path and an all-zero set are kept.
  • tests/sidecar.rs: with shaped_ratio 2.0 a 3 path punch keeps exactly 1 path, reports 2 as shaped, and the kept path carries a stream. Without shaped_ratio all 3 are kept.
  • tests/test_link_channel.py: letify asks for 0.05.
  • Red was a compile failure (shaped_out absent). Green: cargo test -p letify-link all pass, cargo fmt --all --check and cargo clippy -p letify-link --all-targets clean, ruff check letify tests clean, 69 Python link tests pass.

Live A/B

One kaggle_60 CPU session, one letify-link session per connect over its bridge, arms interleaved A B A B A B with 25 s between connects. Arm A: POLICE without shaped_ratio. Arm B: shaped_ratio 0.05. Local sidecar built from this branch, remote sidecar the released 3.0.2. Each connect measured 100 MiB up and 100 MiB down on a bulk stream while a ping (the probe P op, one every 50 ms) ran on an interactive stream of the same session.

4 of the 6 connects were refused by the account's link floor (10 MiB/s) because the session probe up rate was 2.1, 2.9, 9.5 and 10.0 MiB/s, so they have no measurements. The 2 that connected:

Connect Arm Paths kept Paths dropped Session probe up / down MiB/s 100 MiB down MiB/s 100 MiB up MiB/s Idle ping ms Ping under down ms (median, p95, n) Ping under up ms (median, p95, n)
3 A 4 TCP + 1 UDP none 10.0 / 21.51 26.31 19.79 193.8 546, 1202, 5 646, 757, 8
6 B 4 TCP + 1 UDP none 10.84 / 20.32 25.96 13.04 193.7 558, 947, 6 873, 1225, 9

The difference between the arms (up 19.79 against 13.04 MiB/s, latency under upload 646 against 873 ms) is between two sessions that kept the same 5 paths and so is run to run noise, not an effect of the rule. One connect per arm is too few for any claim. The sidecar reported no path_policed event in any connect.

The session closed through provider.stop: letify sessions shows kaggle_60 no active sessions afterwards and stop printed no notebook deletion warning.

Next

Rerun when the UDP path is shaped (check letify sessions is empty first, and look at the UDP path's rate in a plain connect). Until then the rule is code-complete but unproven as an improvement.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant