Repository navigation
Fix LineBuffer payload byte accounting - #98
Open
Sagar07-star wants to merge 2 commits into
Open
Sagar07-star wants to merge 2 commits into
Sagar07-star wants to merge 2 commits into
Conversation
brharrington
left a comment
Contributor
There was a problem hiding this comment.
The accounting is correct: counting UTF-8 bytes plus one separator per line after the first matches what SocketWriter sends, so buffer_size now means payload bytes. Two notes:
- Interaction with #95:
appendnow encodes the line, so a lone surrogate like"\ud800"raisesUnicodeEncodeErrorinsideappendrather than at flush time. #95'stest_udp_close_closes_socket_on_encode_errorwrites"\ud800"and expectsclose()to raise, sowrite()would raise first. Whichever PR lands second needs that test reworked. The new behavior is arguably better (a bad line is rejected before it is stored instead of breaking the flush for neighboring lines), but the exception still propagates out ofwrite()since_write_bufferonly catchesIOError. - This adds an encode on every write.
len(line) if line.isascii() else len(line.encode("utf-8"))avoids the allocation for the common ASCII case.
Avoid encoding ASCII lines, verify invalid UTF-8 preserves buffered data, and adapt the close error regression test after PR Netflix#95.
Author
|
Addressed both review points and updated the tests for #95. Could a maintainer approve the workflow run? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Count the UTF-8 payload bytes, including newline separators, when deciding whether LineBuffer should flush.
Problem
LineBuffer currently counts only characters in each input line, while SocketWriter sends the newline-joined buffer encoded as UTF-8. For example, two three-character lines produce seven payload bytes but count as six, so a seven-byte threshold does not trigger a flush.
Changes
The threshold still triggers after appending a whole line; this does not introduce a strict datagram size cap.
Validation