Hummingbot's hyperliquid connector closes its own WebSocket every ~10min (self-inflicted, not HL-side) — stalls grid_executor

[pfeiffer1954]

Bug

A grid_executor (LONG, SOL-USD) filled to ~70% of its total_amount_quote ($70.47/$100) shortly after creation, then stopped placing any further orders — open_liquidity_placed: 0.0 for ~3 hours straight, despite:

• Current price ($76.45) staying within the configured grid range (start_price=74.894, end_price=77.92)
• Position not yet fully built (room for ~$30 more)
• activation_bounds=0.001 (tight but should trigger repeatedly as price moves)
• max_open_orders=2 (a "hard cap on concurrent open orders" per docs — should allow up to 2 resting orders at all times, not just an initial batch)

Evidence

Executor ID: 2XPXNzoEDteWxdzz9YWqz1Ak5e4NqU5cSq5g744BFdjk (hyperliquid_perpetual, SOL-USD, side=BUY)
Created 2026-08-08 14:25. As of 17:10 (~3h later): still open_liquidity_placed: 0.0, position_size_quote: 70.4676 (unchanged since first fill).

manage_executors(get_logs) for this executor returns 0 entries at any log level — no errors, no activity, nothing to diagnose from.

Config

min_spread_between_orders: 0.0001
min_order_amount_quote: 6.0
max_open_orders: 2
max_orders_per_batch: 1
order_frequency: 5
activation_bounds: 0.001
triple_barrier_config.open_order_type: LIMIT_MAKER (post-only)

Suspected cause (unconfirmed)

Possibly LIMIT_MAKER (post-only) order placement silently failing/rejecting near current price given the very tight min_spread_between_orders (0.01%), with no error surfaced to the executor's own log. Or a replenishment-loop bug specific to low max_open_orders values. Needs server-side log access beyond what's available via the MCP tool surface to confirm.

Ask

Investigate why order replenishment stalls after the first fill(s) for this config shape, and why no error/warning is logged when it does.

[pfeiffer1954]

Follow-up with real diagnosis (had docker log access I initially missed — hummingbot-api container logs, not just the MCP executor log endpoint which returns nothing for this executor type).

Real root cause found in hummingbot-api container logs:

1. Repeated user-stream WebSocket drops on the Hyperliquid connector, ~every 10 minutes, 70 occurrences in the last 3h:

HyperliquidAPIUserStreamDataSource - WARNING - The websocket connection was closed (The WS connection was closed unexpectedly. Close code = 1000 msg data: 1000)

2. A real order submission failure burst:

14:46:02-04 - HyperliquidPerpetualDerivative - WARNING - Failed to submit SELL order to Hyperliquid_perpetual. Check API key and network connection. (x3)

3. Grid engine logs confirm the executor legitimately entered a "Waiting for close order to be filled" state after its first level filled (logged every ~6s from 14:33-14:39), then logging for that executor stops entirely — consistent with the WS instability causing the engine to lose track of order/fill state and never resume the wait-loop or place further orders.

Revised conclusion: not a grid_executor replenishment logic bug as originally suspected — it's connector-level WebSocket instability causing the executor to get stuck mid-cycle. The frequent WS drops are the primary thing to fix; the grid_executor may also benefit from more defensive handling of a dropped/reconnected WS mid-wait (currently seems to just silently stop progressing rather than recover or surface an error).