Repository navigation
Research/content region coverage v1 - #81
Conversation
…ers, and frozen protocol amendments
…ces at target FPR
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0ec055be52
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| not row["introduced_forbidden_codepoints"] | ||
| and not row["introduced_non_ascii_codepoints"] | ||
| and not row["introduced_normalization_instability"] | ||
| and row["cf_strip_noop"] |
There was a problem hiding this comment.
When a source already contains a Unicode Cf character and the transformation preserves it, introduced_forbidden_codepoints is correctly empty, but cf_strip_noop is still false because it examines the entire transformed string. This makes gates_pass reject a candidate for a pre-existing source property, contrary to the introduced-codepoint semantics used for the other forbidden categories and causes valid rows from such corpora to fail the robustness report.
Useful? React with 👍 / 👎.
| protected_intersections = 0 | ||
| for start_window, end_window in uncovered_spans: | ||
| char_start = offsets[start_window][0] if start_window < len(offsets) else len(text) | ||
| char_end = offsets[min(end_window - 1, len(offsets) - 1)][1] if end_window - 1 < len(offsets) else len(text) |
There was a problem hiding this comment.
For an isolated uncovered observation window, this endpoint includes only the token at the window's start, although each detector observation spans NGRAM_LEN (five) tokens. Runs are similarly shortened by four tokens, so uncovered_lexical_categories, unprotected_word_samples, and protected_span_intersections omit most of every uncovered region and can support incorrect coverage-hole conclusions.
Useful? React with 👍 / 👎.
1 participant
No description provided.