[improve][ws] WebSocket proxy producers have no backpressure: the client memory limit is always disabled

[lhotari]

Search before asking

• I searched in the issues and found nothing similar.

Motivation

Producers created by the WebSocket proxy have no backpressure at all out of the box, so a client that publishes faster than the broker acknowledges makes the shared proxy JVM buffer without limit.

Both of the mechanisms that would bound it are off:

• Byte-based. WebSocketService.createClientInstance passes webSocketPulsarClientMemoryLimitInMB straight to ClientBuilder.memoryLimit, and that config is an int defaulting to 0, which the client reads as "no memory limit". [improve][ws] Add memory limit configuration for Pulsar client used in Websocket proxy #22666 added the setting, but because 0 is both its default and the client's "disabled" value, the client's own 64M default can never take effect — the proxy always overrides it with 0. Before [improve][ws] Add memory limit configuration for Pulsar client used in Websocket proxy #22666 the call was a hard-coded .memoryLimit(0, SizeUnit.BYTES), so the behaviour is unchanged from then.
• Message-count. ProducerHandler.getProducerBuilder uses the no-argument PulsarClient.newProducer(), and maxPendingMessages is left at its default of 0.

ProducerImpl only creates its pending-message semaphore if (conf.getMaxPendingMessages() > 0), and MemoryLimitController never blocks when its limit is 0, so neither path bounds the queue. The proxy also sets blockIfQueueFull(false) specifically so its threads are not blocked when a lot of messages are pending — which is the right choice, but it only takes effect once something actually bounds the queue.

Solution

Give the proxy byte-based backpressure by default, and let an operator opt out explicitly:

• Make webSocketPulsarClientMemoryLimitInMB an Integer with no default in WebSocketProxyConfiguration and ServiceConfiguration, and call ClientBuilder.memoryLimit only when it is set. Unset then means "use the client default" (64M) rather than "disabled", while 0 keeps meaning "disabled" for anyone who wants that. This is the same unset-vs-explicit-0 distinction as in [fix][client] Apply no-memory-limit producer queue defaults at producer creation #26342.
• Once the memory limit applies, review the maxPendingMessages query parameter: it is remote-supplied, so a value of 0 would ask for an unbounded pending queue. Whether the proxy should accept that from a client is worth deciding on its own.

Note this is a behaviour change for existing deployments that never set the config: their WebSocket producers become bounded by 64M of client memory, and a client that outruns the broker will start getting failed ProducerAck responses instead of the proxy buffering indefinitely. That is the intent, but it is worth calling out in the release notes.

Context: this came up in #26342, which fixes the equivalent problem on the client side (the pending-message defaults applied when a client's memory limit is disabled). A guard for the proxy's query parameter was included there at first and reverted, so the proxy is handled on its own here.

Alternatives

Bounding the proxy's producers with maxPendingMessages instead. The memory limit is the better fit: the proxy multiplexes many producers over one client, and what matters is the total memory held in the JVM, not the queue depth of any single one.

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!