Problem
mqdb agent start binds two listeners — MQDB_HTTP_BIND (:8081, REST/auth) and MQDB_WS_BIND (:8080, MQTT-over-WebSocket at /mqtt). The WS listener treats every request as a WebSocket upgrade: a plain HTTP GET to :8080 fails the handshake (No "Connection: upgrade" header) and returns no clean HTTP response (an ALB in front reports 502).
Because ALB target-group health checks are HTTP-GET only, they cannot probe :8080 directly. In our staging infra the WS target group is therefore forced to health-check :8081 /auth/session as a proxy. If the WS listener ever fails independently of the HTTP listener, the target still reports healthy, the LB never drains it, and clients silently drop to "local only" while every dashboard stays green.
Request
Serve a lightweight plain-HTTP health route on the WS bind (:8080), e.g. GET /health -> 200 OK, handled before the WebSocket upgrade attempt so non-upgrade requests to that path get a normal HTTP response. /mqtt keeps upgrading as today.
Why
Lets load balancers health-check the actual WS listener port instead of a proxy on a different port, closing the blind spot above.
Workaround in place
laboverwire/infrastructure runs a synthetic canary Lambda that does a real wss://.../mqtt handshake every minute and alarms on failure. This issue tracks the upstream fix that would let the ALB check :8080 directly and retire the canary.
Problem
mqdb agent startbinds two listeners —MQDB_HTTP_BIND(:8081, REST/auth) andMQDB_WS_BIND(:8080, MQTT-over-WebSocket at/mqtt). The WS listener treats every request as a WebSocket upgrade: a plain HTTPGETto:8080fails the handshake (No "Connection: upgrade" header) and returns no clean HTTP response (an ALB in front reports 502).Because ALB target-group health checks are HTTP-GET only, they cannot probe
:8080directly. In our staging infra the WS target group is therefore forced to health-check:8081 /auth/sessionas a proxy. If the WS listener ever fails independently of the HTTP listener, the target still reports healthy, the LB never drains it, and clients silently drop to "local only" while every dashboard stays green.Request
Serve a lightweight plain-HTTP health route on the WS bind (
:8080), e.g.GET /health->200 OK, handled before the WebSocket upgrade attempt so non-upgrade requests to that path get a normal HTTP response./mqttkeeps upgrading as today.Why
Lets load balancers health-check the actual WS listener port instead of a proxy on a different port, closing the blind spot above.
Workaround in place
laboverwire/infrastructureruns a synthetic canary Lambda that does a realwss://.../mqtthandshake every minute and alarms on failure. This issue tracks the upstream fix that would let the ALB check:8080directly and retire the canary.