Skip to content

Fix UTF-16 TextDecoder unpaired surrogate handling (#4612)#4650

Open
Rahul-2k4 wants to merge 5 commits intoboa-dev:mainfrom
Rahul-2k4:fix/textdecoder-utf16-wellformed-4612
Open

Fix UTF-16 TextDecoder unpaired surrogate handling (#4612)#4650
Rahul-2k4 wants to merge 5 commits intoboa-dev:mainfrom
Rahul-2k4:fix/textdecoder-utf16-wellformed-4612

Conversation

@Rahul-2k4
Copy link

Summary

  • make UTF-16 TextDecoder output well-formed strings by replacing unpaired surrogates with U+FFFD
  • apply the fix to both utf-16le and utf-16be decode paths
  • keep truncation behavior correct for odd-length input (no extra replacement when the last decoded code unit is a high surrogate)

Tests

  • add regression tests for unpaired surrogate replacement in utf-16le and utf-16be
  • add regression tests for truncation edge cases in both endian modes
  • verified with:
    • cargo test -p boa_runtime decoder_utf16
    • cargo test -p boa_runtime
    • cargo fmt --all --check
    • cargo clippy -p boa_runtime --all-targets -- -D warnings

Closes #4612.

@github-actions
Copy link

github-actions bot commented Feb 21, 2026

Test262 conformance changes

Test result main count PR count difference
Total 52,862 52,862 0
Passed 49,504 49,504 0
Ignored 2,262 2,262 0
Failed 1,096 1,096 0
Panics 0 0 0
Conformance 93.65% 93.65% 0.00%

@Rahul-2k4 Rahul-2k4 closed this Feb 23, 2026
@zhuzhu81998
Copy link
Contributor

@Rahul-2k4 are you still working on this?

@Rahul-2k4
Copy link
Author

Rahul-2k4 commented Feb 23, 2026

yep just finished in a while rn final commits are taking place

@Rahul-2k4 Rahul-2k4 reopened this Feb 23, 2026
@Rahul-2k4 Rahul-2k4 force-pushed the fix/textdecoder-utf16-wellformed-4612 branch from 8c98bbc to 0e8a6ac Compare February 23, 2026 10:39
@jedel1043 jedel1043 added bug Something isn't working runtime waiting-on-author Waiting on PR changes from the author labels Feb 25, 2026
@codecov
Copy link

codecov bot commented Feb 25, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 57.13%. Comparing base (6ddc2b4) to head (45fcd6e).
⚠️ Report is 679 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4650      +/-   ##
==========================================
+ Coverage   47.24%   57.13%   +9.89%     
==========================================
  Files         476      549      +73     
  Lines       46892    60304   +13412     
==========================================
+ Hits        22154    34456   +12302     
- Misses      24738    25848    +1110     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Rahul-2k4 Rahul-2k4 force-pushed the fix/textdecoder-utf16-wellformed-4612 branch from 0e8a6ac to 45fcd6e Compare February 25, 2026 05:44
@jedel1043 jedel1043 added waiting-on-review Waiting on reviews from the maintainers and removed waiting-on-author Waiting on PR changes from the author labels Feb 25, 2026
@jedel1043 jedel1043 modified the milestone: v1.0.0 Feb 25, 2026
@Rahul-2k4 Rahul-2k4 requested a review from a team as a code owner February 26, 2026 09:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working runtime waiting-on-review Waiting on reviews from the maintainers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

utf-16 TextDecoder is incorrect

3 participants