Skip to content

Pattern-only fallback (embedding engine absent) has no precision floor — amplifies over-firing #195

Description

@aaronsb

Summary

When the embedding engine is absent, way matching falls back to pattern-only (SessionStart reports: "⚠ Embedding engine not installed — semantic way matching is unavailable. Only explicit pattern:/commands:/files: triggers will fire."). In that mode, disclosure precision drops sharply, because the coarse pattern: regexes become the sole gate — exactly when the more precise semantic layer that would normally disambiguate them is missing.

I ran an entire session in this degraded mode and saw the consequence: ways misfired on generic tokens (schema→migrations, profile→performance, version bump→release; details in companion issue #194). The patterns are the root cause, but the embeddings-off fallback amplifies it — there's no second signal to veto a coarse pattern hit.

The gap

The fallback is "fire on any pattern/files/commands trigger." That's the right availability choice (better some guidance than none), but it has no precision floor. A bare-token pattern that semantic scoring would have outvoted now fires unchecked.

Options

  1. Conservative gating in degraded mode. When embeddings are unavailable, raise the bar: fire a pattern:-only match only if it co-occurs with a files:/commands: signal, or only for patterns flagged high-specificity. Keep always-on macro: prepend ways (like core) unconditional.
  2. A semantic_only: true frontmatter flag for ways whose pattern: is inherently loose — they simply don't fire in pattern-only mode, accepting silence over a likely false positive.
  3. Specificity tiers. Mark each trigger's confidence; in degraded mode only fire high-confidence triggers. (Heavier; probably overkill.)
  4. Make the warning louder / nudge install. The SessionStart line is easy to skim past; if degraded mode is common, the misfire cost is ongoing. At minimum, surface which ways are running pattern-only.

Recommendation: (1) + (2) together — a precision floor in degraded mode, plus an opt-out flag for the ways most prone to collision. This composes with fixing the patterns themselves (#194): tighter patterns help in both modes; conservative fallback contains the blast radius when semantics are gone.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions