fix(ci): the invisible-character gate never matched anything - #64
Conversation
MEASURED 2026-08-27: this gate's pattern caught 0 OF 6 invisible-character test
cases. It has never detected an NBSP, zero-width space, BOM, soft hyphen, bidi
override or word joiner.
ROOT CAUSE: the pattern used UTF-8 BYTE sequences (\xc2\xa0) while grep -P
matches CHARACTERS. Bytes c2 a0 are ONE character U+00A0; \xc2\xa0 asks for TWO
characters, U+00C2 then U+00A0, which is never present.
grep -P '\xc2\xa0' -> miss
grep -P '\x{a0}' -> MATCH
Only \x00 worked, being single-byte in both readings.
FIXED: codepoint escapes; C0 control characters \x01-\x08,\x0B,\x0C,\x0E-\x1F
added (TAB/LF/CR excluded); and grep -a, without which grep skips any NUL-bearing
file as binary.
The C0 range matters: a stray BACKSPACE byte made a workflow unparseable in
developer-ecosystem, so it never ran, and this linter called it clean.
Canonical fix: hyperpolymath/empty-linter#70. 1 file(s) here.
VERIFIED: YAML re-parsed, and the corrected pattern was confirmed to catch a real
NBSP before the change was kept.
|
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review. 📜 Recent review details⏰ Context from checks skipped due to timeout. (2)
|
| Layer / File(s) | Summary |
|---|---|
Pattern and binary-file scan .github/workflows/dogfood-gate.yml |
The regex now detects additional control, formatting, and invisible characters. The grep command uses -a to scan binary files as text. |
Estimated code review effort: 2 (Simple) | ~10 minutes
Merge Risk: ⚪ Minimal · up to f0582
This localized workflow fix improves detection of invisible characters without introducing an actionable merge-blocking risk; it is merge-ready after normal checks and review.
Poem
A rabbit checks each hidden sign
Zero-width marks now fall in line
Binary files join the scan
Control codes cannot hide again
The gate hops on, precise and bright
🚥 Pre-merge checks | ✅ 4 | ❌ 1
❌ Failed checks (1 warning)
| Check name | Status | Explanation | Resolution |
|---|---|---|---|
| Linked Issues check | The change implements codepoint escapes, C0 control detection, and grep -a. The provided summary does not show the required leading-BOM check, compiled-linter alignment, or estate-wide copy updates fr… | Add the separate leading-BOM check and implement the remaining issue #70 requirements, or update the linked issue and PR scope to document that this change covers only the CI gate file. |
✅ Passed checks (4 passed)
| Check name | Status | Explanation |
|---|---|---|
| Title check | ✅ Passed | The title clearly identifies the main change: fixing the CI invisible-character gate. |
| Description check | ✅ Passed | The description directly explains the gate failure, root cause, implemented fixes, and verification. |
| Out of Scope Changes check | ✅ Passed | The two-line workflow change is directly related to fixing the invisible-character gate and does not show unrelated changes. |
| Docstring Coverage | ✅ Passed | No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0… |
Full details: Linked Issues check
Explanation
The change implements codepoint escapes, C0 control detection, and grep -a. The provided summary does not show the required leading-BOM check, compiled-linter alignment, or estate-wide copy updates from issue #70.
Full details: Docstring Coverage
Explanation
No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (1 skipped: 1 unsupported.)
- Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
- Create stacked PR
- Commit on current branch
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Comment @coderabbitai help to get the list of available commands.
Up to standards ✅🟢 Issues
|
![codacy-production[bot]](https://avatars.githubusercontent.com/in/56611?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull Request Overview
The PR correctly identifies and addresses a weakness in the invisible-character gate by migrating to PCRE codepoint escapes and ensuring NUL-bearing files are scanned. However, a significant logic flaw exists: using \x{...} sequences in grep -P requires the (*UTF) prefix to correctly handle characters outside the ASCII range.
Without this prefix, the command will likely fail or produce false positives for characters like the Word Joiner (U+2060). Furthermore, because stderr is redirected to /dev/null, these errors will be suppressed, leading to a false sense of security where the gate reports success despite a failed scan. There is also a lack of test fixtures to verify these CI changes.
About this PR
- The PR lacks automated tests or fixture files (e.g., sample files containing invisible characters) to verify the gate's effectiveness. Without these, it is difficult to confirm that the new regex patterns correctly identify all target characters or to prevent future regressions.
Test suggestions
- Missing: Verify detection of Non-Breaking Space (U+00A0) using codepoint escape
- Missing: Verify detection of C0 control characters (e.g., Backspace \x08)
- Missing: Verify grep scans files containing NUL bytes rather than skipping them
- Missing: Verify detection of Word Joiner (U+2060) and other Unicode invisible characters
Prompt proposal for missing tests
Consider implementing these tests if applicable:
1. Missing: Verify detection of Non-Breaking Space (U+00A0) using codepoint escape
2. Missing: Verify detection of C0 control characters (e.g., Backspace \x08)
3. Missing: Verify grep scans files containing NUL bytes rather than skipping them
4. Missing: Verify detection of Word Joiner (U+2060) and other Unicode invisible characters
TIP Improve review quality by adding custom instructions
TIP How was this review? Give us feedback
| # non-breaking spaces, null bytes, and other invisible Unicode in source files. | ||
| set +e | ||
| PATTERNS='\xc2\xa0|\xe2\x80\x8b|\xe2\x80\x8c|\xe2\x80\x8d|\xef\xbb\xbf|\xc2\xad|\xe2\x80\x8e|\xe2\x80\x8f|\xe2\x80\xaa|\xe2\x80\xab|\xe2\x80\xac|\xe2\x80\xad|\xe2\x80\xae|\x00' | ||
| PATTERNS='\x00|[\x01-\x08\x0B\x0C\x0E-\x1F]|\x{a0}|\x{ad}|\x{200b}|\x{200c}|\x{200d}|\x{200e}|\x{200f}|\x{202a}|\x{202b}|\x{202c}|\x{202d}|\x{202e}|\x{2060}|\x{feff}' |
There was a problem hiding this comment.
🔴 HIGH RISK
Add the (*UTF) prefix to the pattern string to enable proper Unicode code point matching in PCRE mode. Without it, grep will fail with an error for any code point above 255 (such as the Zero Width Space or BOM) or incorrectly match single bytes of a UTF-8 sequence. Since stderr is redirected to /dev/null, these failures are currently invisible.
| PATTERNS='\x00|[\x01-\x08\x0B\x0C\x0E-\x1F]|\x{a0}|\x{ad}|\x{200b}|\x{200c}|\x{200d}|\x{200e}|\x{200f}|\x{202a}|\x{202b}|\x{202c}|\x{202d}|\x{202e}|\x{2060}|\x{feff}' | |
| PATTERNS='(*UTF)\x00|[\x01-\x08\x0B\x0C\x0E-\x1F]|\x{a0}|\x{ad}|\x{200b}|\x{200c}|\x{200d}|\x{200e}|\x{200f}|\x{202a}|\x{202b}|\x{202c}|\x{202d}|\x{202e}|\x{2060}|\x{feff}' |
| -o -name '*.idr' -o -name '*.zig' -o -name '*.v' -o -name '*.jl' \ | ||
| -o -name '*.gleam' -o -name '*.hs' -o -name '*.ml' -o -name '*.sh' \) \ | ||
| -exec grep -Prl "$PATTERNS" {} \; > /tmp/empty-lint-results.txt 2>/dev/null | ||
| -exec grep -aPrl "$PATTERNS" {} \; > /tmp/empty-lint-results.txt 2>/dev/null |
There was a problem hiding this comment.
⚪ LOW RISK
Nitpick: Remove the redundant -r flag. The find command is already handling the recursion and passing individual file paths to grep via the -exec argument.
| -exec grep -aPrl "$PATTERNS" {} \; > /tmp/empty-lint-results.txt 2>/dev/null | |
| -exec grep -aPl "$PATTERNS" {} \; > /tmp/empty-lint-results.txt 2>/dev/null |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
1 participant
Measured 2026-08-27: this gate caught 0 of 6 invisible-character test cases. It has never detected an NBSP, zero-width space, BOM, soft hyphen, bidi override or word joiner.
Root cause
The pattern used UTF-8 byte sequences (
\xc2\xa0) whilegrep -Pmatches characters. Bytesc2 a0are one character U+00A0;\xc2\xa0asks for two, U+00C2 then U+00A0 — never present.Only
\x00worked, being single-byte in both readings. The gate ran, passed, and could not see what it exists to see.Fixed
\x01-\x08,\x0B,\x0C,\x0E-\x1Fadded (TAB/LF/CR excluded)grep -a— without it grep skips any NUL-bearing file as binaryThe C0 range matters: a stray backspace byte made a workflow unparseable in
developer-ecosystem, so it never ran — and this linter called it clean.Canonical fix: hyperpolymath/empty-linter#70. 1 file(s) here.
Verified: YAML re-parsed, and the corrected pattern was confirmed to catch a real NBSP before the change was kept.