fix(ws): pass skip_utf8_validation=True to run_forever to survive non-UTF-8 text frames#149
Merged
Merged
Conversation
IBKR occasionally emits non-UTF-8 bytes (e.g. 0x99) in text-opcode frames, which causes websocket-client's frame decoder to raise UnicodeDecodeError inside its receive loop. The run_forever thread then dies before the close handshake completes, so on_close never fires and the reconnect path is never triggered — event delivery stops silently. Pass skip_utf8_validation=True so websocket-client lets raw bytes through; decoding is already handled above this layer. Closes Voyz#148 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Owner
|
LGTM, thanks for the contribution! 🙌 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi maintainers,
Closes #148.
This PR fixes a silent WebSocket failure where
WsClient._run_websocketcallswsa.run_forever(...)withoutskip_utf8_validation=True. When IBKR emits a text-opcode frame containing bytes outside the UTF-8 range (we've observed0x99in the wild on order/trade channels),websocket-client's frame decoder raisesUnicodeDecodeErrorinside its receive loop. Therun_foreverthread then dies before the close handshake completes,on_closenever fires, the reconnect path is never triggered, and event delivery stops silently — no error, no reconnect, just a quiet stall.Passing
skip_utf8_validation=Truelets raw bytes through to the application layer (whereWsClientalready wraps message handling in its own try/except), which is consistent with how the upstreamwebsocket-clientlibrary expects this case to be handled.Key changes:
ibind/base/ws_client.py: passskip_utf8_validation=Truetowsa.run_forever()and a short# whycomment naming the failure mode.test/integration/base/test_websocket_client_i.py:_verify_startednow also assertsskip_utf8_validation=Trueis passed. Since this helper is shared by every happy-path test in the file (test_start_success,test_start_success_on_second_attempt,test_open_and_close,test_send,test_check_ping, …), the kwarg is regression-pinned. Also extended the localrun_forever_exceptionmock signature intest_open_exceptionso the new kwarg doesn't TypeError-loop the reconnect path.test/integration/base/websocketapp_mock.py: extended the sharedrun_forevermock signature withskip_utf8_validation: bool = Falsefor the same reason.Verification:
pytest test/unit test/integration→ 78 passed locally.make lintis unchanged on the modified files.Notes:
Happy to adjust scope, comment, or test layout based on review.
Thanks!