demos: add Envoy Rust dynamic module ingress implementation - #1329
demos: add Envoy Rust dynamic module ingress implementation#1329botengyao wants to merge 3 commits into
Conversation
The ingress path calls ResumeActor on ate-apiserver for every request, including requests to an actor already running on a known worker, since singleflight only collapses concurrent callers. This adds a runnable comparison of that path against an Envoy Rust dynamic module. Arm A runs the real atenet router and its real xds.go over ADS, so the baseline is the actual system. Measured against it, a module that caches the actor to worker binding raises throughput 11.5x and cuts ResumeActor calls by 99.7%. Removing the ext_proc hop alone accounts for only ~12% of that: the win is not making the call, not where the call is made. Arm C is the shape worth landing. The module sits in front of ext_proc as a pure cache, answering hits and letting every miss fall through to the Go handler untouched, then learning the binding from the metadata ext_proc published. The dataplane never talks to ate-apiserver, and removing the filter restores today's behaviour exactly. See DESIGN.md; ANALYSIS.md surveys the rest of the tree, egress above all. Prototype only, not wired into any build or deploy.
authority(), parse_actor_ref() and route_to() are pure functions of the parsed filter config, but they lived on the filter that resolves misses with an HTTP callout. The co-existence filter had to construct a throwaway instance of that filter per request just to borrow them, which reads as though the cache mode calls ate-apiserver. It never does. Move them onto Config so neither mode names the other, and drop a duplicated route_to and a per-request config clone with them. Also add rust-module/DESIGN.md: the thread model and why the cache is process-global, the two filter modes and which one is meant for production, the rule that anything unrecognized falls through to ext_proc, and what is still missing before the cache can front traffic.
The findings worth keeping are already in README.md and DESIGN.md, each carrying its own source references. The long-form survey was generated material that would go stale against the tree without anyone noticing.
|
I prefer |
|
Why is this PR "open" if it's a prototype? Given the limitations on cI runners we asked to please hold off until it's ready. |
AFAIY the workflows are not running until someone hit the button, no? |
In this case yes, but it's also open which is confusing, that signals that it's ready for review |
|
You can convert this to a draft if it's a PoC. I think the Github CI still gets triggered though... |
|
CI needs approval, draft is fine, once we decide the final approach will make it a final PR. |
not aware of, good to know, thanks. |
botengyao
changed the title
Only because you're not a repo collaborator yet. You can share a link to your branch without opening a PR FWIW. If you intend for a review pass, that is the time for a draft or real PR. Unfortunately there isn't any way to full disable actions for drafts. You can make it skip most of the logic early but it still spawns a workflow to evaluate the conditional. |
| license = "Apache-2.0" | ||
|
|
||
| [dependencies] | ||
| # The SDK revision MUST match the Envoy binary it is loaded into. This rev is the |
There was a problem hiding this comment.
It is not a MUST. I think the official support for ABI is two releases. In practice it had never been broken.
Having said that, this is definitely preferable to be kept in sync. Can dependabot do this?
| if let Some(entry) = cache().get(&key) { | ||
| if entry.expires_at > Instant::now() { | ||
| let worker_ip = entry.worker_ip.clone(); | ||
| drop(entry); |
6 participants
The ingress path calls ResumeActor on ate-apiserver for every request, including requests to an actor already running on a known worker, since singleflight only collapses concurrent callers. This adds a runnable comparison of that path against an Envoy Rust dynamic module.
Arm A runs the real atenet router and its real xds.go over ADS, so the baseline is the actual system. Measured against it, a module that caches the actor to worker binding raises throughput 11.5x and cuts ResumeActor calls by 99.7%. Removing the ext_proc hop alone accounts for only ~12% of that: the win is not making the call, not where the call is made.
Arm C is the shape worth landing. The module sits in front of ext_proc as a pure cache, answering hits and letting every miss fall through to the Go handler untouched, then learning the binding from the metadata ext_proc published. The dataplane never talks to ate-apiserver, and removing the filter restores today's behaviour exactly. See DESIGN.md; ANALYSIS.md surveys the rest of the tree, egress above all.
#1330
What
Adds
demos/envoy-rust-dynamic-module/: a self-contained, runnable comparison of the ingress routing path against an Envoy Rust dynamic module. Nothing is wired into the build, the manifests, or.ko.yaml— no production code path changes.Why
ingress.HandlercallsResumeActoron ate-apiserver for every request, including requests to an actor that is alreadyRUNNINGon a known worker, sincesingleflightcollapses only concurrent callers and never sequential ones.ingress/resumer.go:182names the cost outright:For a warm working set of actors, that is a control-plane round trip per request that changes nothing. Envoy's ext_proc client has no way to avoid it: the filter has no response cache, and its only cache-shaped fields
(
disable_clear_route_cache,route_cache_action) concern the route cache. A dynamic module is the supported place to put one.Results
Envoy v1.39.1 (the pinned version), 50 hot actors, concurrency 8, 20s after warmup, load generated inside the container network. Reproduced twice, within 2%.
Arm A runs the real
atenet routerbinary configured by the realxds.goover ADS, so the baseline is the actual system rather than an approximation.The split matters more than the headline. Removing the gRPC hop alone (A→B1) is worth +12%. Not making the call (B1→B2) is worth 12×. The cache is the win; Rust is the delivery mechanism.
Arm C reaches 96% of the full replacement while keeping ext_proc in the chain, so the dataplane never talks to ate-apiserver. CPU per request drops ~6×, and
atenet-router's own CPU goes from ~47% to ~1%.These are laptop-VM numbers against a stub control plane; the shape transfers, the absolute figures do not.
The shape worth landing
Arm C, described in
demos/envoy-rust-dynamic-module/DESIGN.md: the module is a cache in front of ext_proc, not a replacement for it.original_dstdynamic metadata ext_proc would have, marks the request, and callsclear_route_cache()so route selection lands on a route carryingExtProcPerRoute.disabled. Verified: 16 requestsproduced 1
ext_proc.streams_started.So it never holds an ate-apiserver credential, needs no new API surface, and cannot route anywhere ext_proc has not already routed. Landing it would be three additive edits to
xds.gobehind one flag defaulting off, with the generated xDS byte-identical when disabled.Status and limitations
This is a prototype for discussion, not a proposal to enable anything.
421+X-Ate-Assignment-Staleand on upstream reset, and — the important part — eviction must re-run the slow path within the same request,because a cache hit skips the
ResumeActorthat wakes a suspended actor.X-Ate-Assignment-Staleis currently forgeable: atunnel'sReverseProxyhas noModifyResponsestripping it from actor responses. That needs fixing before anything trusts the header.fails only ext_proc streams.
Envoy image requires rebuilding the module.
Related: the same missing-cache shape exists on egress, where
egress.go:168callsGetActorper CONNECT with no cache and no singleflight — see the TODO ategress.go:167referencing #592.Testing
go build ./...andgo test ./cmd/atenet/...pass. The demo itself is verified by running it:./bench/build.sh && docker compose up -d && ./bench/run.sh.