Release v1.8.0#57
Merged
Merged
Conversation
Stage-2 carve-out in _suppress_compound_split_valid_words: keep (do not suppress) compound-split tokens whose ed=1 SymSpell top-1 is a single in-syllable-diacritic INSERTION, emitting the corrected syllable at the narrow gold span. +0.0007 composite / +21 TP within the clean-FP cap, behind default-off flag compound_split_ortho_insertion_rescue. Also: split the candidate_not_generated FN telemetry into an honest candidate_suppressed bucket (generatable but not surfaced), and make the benchmark output filename collision-proof (microsecond resolution). Workstream: ortho-suppressor-rescue
Back-merge release/v1.7.1 → develop
Move the models/ ignore rule from .git/info/exclude into the tracked .gitignore so model artifacts (symlinked to external storage) are never committed regardless of checkout. Part of v1.8.0 WS-1 baseline-lock. Workstream: v18-hygiene-baseline-lock
v1.8.0 seed (WS-0): ortho-insertion rescue (default-off flag) + honest FN telemetry (candidate_suppressed split)
…bels
Full clean-set sweep (Opus 4.8) over all 93 clean-FP rows recalibrated the true clean-FP to 91 (+1 cap headroom). Fixes: 7 corrupted gold answers + 17 corrupted clean-row inputs ({ed-pa-kha}+flat-aa residue from the 2026-04-15 sweep, normalized to tall-aa, kept clean) + 2 cap mislabels reclassified clean->error (BM-424 broken stacking, BM-EXT-E026 missing loan-final asat) + 32 confusable_semantic->synonym_substitution relabels. ~3800 other flat-aa occurrences are correct and untouched (narrow whitelist only).
Workstream: v18-taxonomy-cleanup
Benchmark: myspellchecker_benchmark.yaml@1.5.0-v18b-clean-audit
WS-0b: benchmark clean-set audit (flat-aa residue + cap mislabels + relabels)
WS-1: hygiene/baseline-lock (models/ gitignore)
Bumps [actions/checkout](https://github.com/actions/checkout) from 6.0.2 to 6.0.3. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@de0fac2...df4cb1c) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: 6.0.3 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 8.1.0 to 8.2.0. - [Release notes](https://github.com/astral-sh/setup-uv/releases) - [Commits](astral-sh/setup-uv@0880764...fac544c) --- updated-dependencies: - dependency-name: astral-sh/setup-uv dependency-version: 8.2.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>
Pre-norm detector surfaces aw-vowel typos the normalizer silently repairs; deterministic canonical suggestion, default-off flag. Benchmark v18c annotates 67 generator-planted aw typos it missed. Workstream: v18-normalizer-unmask-aw-vowel Benchmark: myspellchecker_benchmark.yaml@1.5.0-v18c-aw-unmask-annotations Metrics: composite 0.6520 -> 0.6870 (+0.0350)
Detect aw-vowel spelling errors that normalization previously hid
p95 658 -> 401ms (gate PASS): instance-level memo of dictionary-valid nasal variants in SymSpell (was 15M is_valid_word calls on tail) plus LRU memo on the frozen probe's score_sentence (69% duplicate forwards). Byte-identical detections across all 2084 benchmark sentences. Workstream: v18-latency-reduction Benchmark: myspellchecker_benchmark.yaml@1.5.0-v18c-aw-unmask-annotations Metrics: composite 0.6870 -> 0.6870 (0.0000); p95 658 -> 401ms
Cut p95 check latency from 658ms to 401ms with two memoizations
The lat-02 score-cache (OrderedDict LRU) is built once and shared across all probe strategies, and check_batch_async hits it concurrently via asyncio.to_thread workers. An unlocked get/move_to_end racing another worker's popitem(last=False) near the 256 cap could raise KeyError and crash that check(). Guard the LRU dict ops with a threading.Lock (released around the encoder pass, so a duplicate miss merely recomputes) and re-check on insert. Workstream: v18-preship-fixes
…ents Remove the unused _is_orthographic_insertion_typo bool wrapper (no committed callers; the wired gate is _ortho_insertion_detail) and fold its full spec into that method's docstring. Correct two misleading comments that risk steering a future edit wrong: the symspell nasal-variant cache is serialized under _index_lock (not a "benign data race"), and U+103D is MEDIAL WA, not ha-htoe. No behavior change. Workstream: v18-preship-fixes
…l coverage The compound-split suppression tests build a MagicMock config, so the default-off compound_split_ortho_insertion_rescue flag read as a truthy Mock and the rescue fired in-test — turning one freq-gate test red on develop. Pin the flag to its production default in _build_mixin (opt-in via a new param). Add coverage that was missing: _ortho_insertion_detail shape gate and _narrow_ortho_insertion_error narrowing, plus the aw-vowel guard direction/boundary and span-fallback / duplicate-position branches. Workstream: v18-preship-fixes
The ortho-insertion rescue narrows a kept compound-split error and attaches a 0.9 suggestion but left error.confidence untouched. Narrowing runs in post-processing, after context fusion may have rewritten error.confidence into the [0.5, 0.60) INFORM band — which silently drops the recovered correction from corrected_text while a 0.9 suggestion still shows in the list. Pin err.confidence = 0.9 so the recovery is self-contained. Default-off carve-out; enabled in the v18c benchmark. Workstream: v18-preship-fixes Benchmark: myspellchecker_benchmark.yaml@1.5.0-v18c-aw-unmask-annotations Metrics: composite 0.6870 → 0.6870 (+0.0000)
The aw-vowel un-mask detector deferred the WHOLE whitespace chunk to the vowel-reorder detector when any _VOWEL_REORDER_ERRORS key appeared as a substring, silently dropping a recoverable aw-typo glued to a reorder-key form (e.g. ခော်ကေါင်း lost the ခော်→ခေါ် fix). Compute the reorder-key char ranges and defer only candidate spans that overlap them; non-overlapping spans still fire. FN-only change; never emits a wrong correction. Default-off detector; enabled in the v18c benchmark. Workstream: v18-preship-fixes Benchmark: myspellchecker_benchmark.yaml@1.5.0-v18c-aw-unmask-annotations Metrics: composite 0.6870 → 0.6870 (+0.0000)
v1.8.0 pre-ship fixes: probe-cache lock, rescue robustness, test coverage
…/checkout-6.0.3 chore(deps): bump actions/checkout from 6.0.2 to 6.0.3
…sh/setup-uv-8.2.0 chore(deps): bump astral-sh/setup-uv from 8.1.0 to 8.2.0
Bump version 1.7.1 → 1.8.0. Add CHANGELOG [1.8.0] entry (opt-in aw-vowel un-mask detector, ~40% hot-path latency reduction, concurrent-batch fix). Update README feature list, what's-new callout, and test badge. Workstream: release
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release v1.8.0. Merging this to
mainand taggingv1.8.0cuts the release.Highlights
ခော်→ခေါ်) that the pre-lookup normalizer previously masked before validation. Default-off; opt in viadetect_aw_vowel_unmask/MSC_DETECT_AW_VOWEL_UNMASK. +0.0350 spelling composite when enabled (+98 true positives at zero added false positives).check_batch_asyncworkers no longer race the cache under load.Benchmark
With the aw-vowel detector enabled: composite 0.6870, +98 TP / +0 FP, top-1 92% on the un-masked corrections, clean false-positive sentences within cap (91/779), p95 382 ms. Default behavior (detector off) is unchanged.
Compatibility
Default behavior unchanged — the new detection features are opt-in. The latency and concurrency improvements apply to all deployments with identical results. No public API signature changes.
Contents: version bump 1.7.1 → 1.8.0, CHANGELOG
[1.8.0], README updates, on top of the v1.8.0 feature work already merged todevelop(PRs #47, #54, #55, #56).