Why only show ASCII characters in preview, not all UTF-8?

[rustyhu]

When previewing text files, I noticed that non-ASCII Unicode characters (such as emoji or languages other than English) are not displayed, only ASCII is shown. Looking at the code, it seems the preview logic previously used a function to filter out non-ASCII characters.

walk/main.go

Line 910 in c78d211

previewContent := leaveOnlyAscii(content)

I have many files contain UTF-8 characters. I have tried to edit this code to show all UTF-8, then in my simple tests all preview of UTF-8 content look good. Maybe my tests are limited.

Is there a specific reason for this design? Thanks for your explanation!

[hiSandog]

Valid UTF-8 should be preserved, but removing the ASCII filter needs a separate policy for invalid byte sequences, control characters, and binary files so preview cannot corrupt terminal state. Decode with replacement or classify binary content, strip only unsafe controls while retaining tabs/newlines, and calculate truncation by display width rather than bytes. Tests with CJK, emoji/combining marks, malformed UTF-8, ANSI escapes, and mixed binary data would make the change safe beyond the simple examples.

[antonmedv]

Yes, we need only leave displayable chars