Skip to content

feat: add opt-in V3 delta search refresh - #255

Merged
skyoo2003 merged 4 commits into
mainfrom
feature/v3-delta-search
Sep 13, 2026
Merged

skyoo2003 merged 4 commits into
mainfrom
feature/v3-delta-search

Conversation

@skyoo2003

Copy link
Copy Markdown
Owner

Pull Request

Description

Updating one keyword in a large V3 dictionary currently rebuilds the entire search automaton before WaitForVersion completes. This adds an opt-in MemoryEfficient path that downloads changed buckets, publishes a base automaton with a small additions automaton and deletion set, and compacts it after one idle second. Changes above 128 keywords and other presets retain the full rebuild path. The Redis schema and V1/V2 behavior are unchanged.

The option remains disabled by default because the release gate has not been measured: median commit-to-visibility latency for one add and one removal must each improve by at least 50% across Redis and Valkey and three million-keyword distributions, while search p95 and peak RSS each worsen by no more than 20%. The opt-in scale test now records commit-to-visibility timing and repeated single-keyword updates.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring (no functional changes)
  • Test update

Checklist

  • Tests pass (make test)
  • Vet/make vet
  • Linting passes (make lint)
  • Build succeeds (make build)
  • Documentation updated if needed
  • Changelog fragment added (changie new) if this change belongs in the changelog — one sentence (two at the very most), on a single line, 400 characters max; skip for internal-only changes (CI, tests, refactors); see RELEASE.md
  • Commit messages follow guidelines

Additional Notes

make all passed, including API audit, documentation verification, and license check. go test -race ./pkg/acor -run TestVersionedOverlayLifecycle -count=1 passed. Million-keyword Redis/Valkey performance validation remains outstanding; the new path is opt-in until that gate passes.


By submitting this PR, I agree that my contributions will be licensed under the Apache License 2.0.

@github-actions github-actions Bot added documentation enhancement This issue is about improvement of project. go Pull requests that update Go code labels Sep 13, 2026
@skyoo2003
skyoo2003 marked this pull request as ready for review September 13, 2026 13:27
@skyoo2003
skyoo2003 merged commit 2d2e7f4 into main Sep 13, 2026
10 checks passed
@skyoo2003
skyoo2003 deleted the feature/v3-delta-search branch September 13, 2026 13:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation enhancement This issue is about improvement of project. go Pull requests that update Go code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant