fix(gun): scope named connection processes to the local node

[kamilkowalski]

Problem

GRPC.Client.Adapters.Gun.ConnectionProcess registers itself under {:global, {ConnectionProcess, {ref, host, port}}} (connection_process.ex:180). :global is cluster-wide, which has two distinct consequences on a connected Erlang cluster.

1. A node can adopt another node's connection, and the channel is then unusable. Two nodes calling GRPC.Stub.connect(addr, name: :foo) collide on the same global name. The second gets {:error, {:already_started, remote_pid}}, which connect/2 treats as successful reuse — so it returns {:ok, channel} with a conn_pid that lives on the other node.

Nothing is actually sent over that channel. GRPC.Stub.call/5 checks Process.alive?(conn_pid) on every RPC (stub.ex:298), and Process.alive?/1 raises on a remote pid, so every call fails before a request leaves the node:

** (ArgumentError) errors were found at the given arguments:

  * 1st argument: not a local pid

  :erlang.is_process_alive(#PID<25985.168.0>)
  (grpc 1.0.4) lib/grpc/stub.ex:298: GRPC.Stub.call/5

GRPC.Client.Connection.channel_alive?/1 (connection.ex:939) has the same problem and would raise on the next DNS re-resolve, but the hot-path check fires first. The failure mode is therefore: connect/2 succeeds, then every RPC on that channel raises.

2. Every connect pays a cluster-wide registration, even when no name is ever shared. Callers that don't pass :name still get one — connect/2 fills in make_ref/0 — so channel.ref is non-nil and the process registers globally under a name nothing else will ever look up. :global.register_name/3 takes a global lock and multi-calls every connected node, and :global then keeps that name in a table it must reconcile on every node join and leave.

Production measurement

Deployed on an Elixir service whose pods are clustered via libcluster. Because that service uses unique per-connect names, it only ever hit consequence 2, never 1.

The measurement comes from an endpoint that should be very fast but opens a gRPC connection on every call, so it pays the connect penalty often — meaning it over-represents the connect path relative to steady-state RPC traffic.

Request volume flat at ~40/interval throughout. As the patched version takes over traffic, p95 goes from a spiky 100–215 ms to a flat line under ~5 ms, with p50/p75/p99 dropping together, sustained for 30+ minutes.

Traffic that reuses an established channel should see little or nothing from this change, since it doesn't touch :global after the initial connect.

Fix

Register in the node-local GRPC.Client.Registry, which GRPC.Client.Connection already uses:

{:via, Registry, {GRPC.Client.Registry, {__MODULE__, owner_key(channel)}}}

Reuse becomes per node, which matches the intent of #519 (keeping named channels alive after the calling process exits) — that change needed the process to outlive its caller, not to be shared across the cluster.

Tests

• test/grpc/adapters/gun_test.exs — a named channel connected twice returns the same conn_pid and is registered in GRPC.Client.Registry; an unnamed channel gets a fresh process each time.
• test/grpc/client/connection_test.exs — the existing two-node "named channels do not conflict across connected nodes" test asserted only that both connects returned {:ok, _}, which held while the two nodes shared one process. Extended to assert each node's conn_pid is local to that node, and to issue a real RPC from each node so the unusable-channel case is covered rather than just the pid location.

Both fail on master and pass with the fix. Full suite green; mix format --check-formatted clean. CHANGELOG entry added under Unreleased.

[sleipnir]

Thank you @kamilkowalski

[kamilkowalski]

@sleipnir thanks! Do you know when this would be released? We don't want to upgrade all of our services to v1 without this fix because the bug caused an outage, but also versions <=1.0.0 have the Erlpack vulnerability...

[sleipnir]

@sleipnir thanks! Do you know when this would be released? We don't want to upgrade all of our services to v1 without this fix because the bug caused an outage, but also versions <=1.0.0 have the Erlpack vulnerability...

Hi @kamilkowalski
We released version 1.0.5 in Hex, although pointing your project's dependencies to GitHub is sufficient in most cases. Links to the version:

https://hex.pm/packages/grpc
https://hex.pm/packages/grpc_server