fix: invalidate discovery caches on config change and wire Clear Cache to native finder

[StellaHuang95]

Problem

Two verified defects in native Python discovery caching:

1. Stale soft-cache results. NativePythonFinderImpl's in-memory soft cache keyed only on the refresh scope (all / kind / URI list) and returned cached results before rebuilding or comparing the effective PET ConfigurationOptions. A change to workspace folders, extra search paths, or the conda/pipenv/poetry executable path or cache directory kept serving stale discovery results until an unrelated hard refresh happened.
2. Clear Cache left native caches intact. python-envs.clearCache cleared persistent, manager, and shell caches, but neither the finder's in-memory Map nor PET's on-disk cache directory. clearCacheDirectory had zero callers.

Root cause

The soft-cache key omitted the effective configuration, and the Clear Cache command had no hook into the native finder or PET's disk cache.

Fix

Cache invalidation (nativePythonFinder.ts)

• configurationEquals is extracted into an exported pure function (property-by-property; order-independent on the two directory arrays) and reused for both PET configure gating and soft-cache validation.
• DiscoveryResultCache stores { configuration, results, generation } per existing result-cache key. A soft hit is valid only when the key exists, is in the current generation, and configurationEquals(entry.configuration, currentConfig) holds — so alternating all / kind / URI calls keep their unrelated valid entries and only the changed key is invalidated.
• handleSoftRefresh builds the effective configuration from current settings (reads only, no discovery I/O; build errors propagate) and validates the hit; a miss falls through to a hard refresh.
• A hard refresh captures the generation before any await, builds one ConfigurationOptions, threads it through configure, the CLI fallback, and the result tag, and returns { results, configuration }. DiscoveryResultCache.set is a no-op when the generation has advanced, so a refresh that began before a clearCache() cannot repopulate the just-cleared map.

Clear Cache

• New required NativePythonFinder.clearCache(): Promise<void> clears the in-memory map (advancing the generation), sends PET's clear JSON-RPC request when a server process is live (bounded timeout, best-effort), and empties the on-disk cache directory via fs.emptyDir. Disk-clear failure propagates when it is the primary mechanism (no live clear) and is only logged when redundant after a successful live clear.
• extension.ts wires python-envs.clearCache to the shared finder through a small typed sharedNativeFinder handle, falling back to clearCacheDirectory(context) when Clear Cache runs before the finder exists. Ordering is preserved (persistent → manager → native → shell), with the native step in try/finally so shell-profile cleanup still runs.

PET's JSON-RPC clear method is present in the consumed binary (it predates the info method this repo already uses); its params are ignored and success replies null.

Tests

New unit tests under src/test/managers/common/:

• nativePythonFinder.discoveryCache.unit.test.ts — configurationEquals for every field (order-independence, defined-vs-undefined, fsPath equality) and DiscoveryResultCache (per-key same-config hit; config change invalidating only the affected key; alternating all/kind/URI not thrashing; hard result tagged with its captured config; explicit clear emptying entries and advancing the generation; a store from a refresh that began before clear not repopulating).
• nativePythonFinder.clearCache.unit.test.ts — live clear RPC plus on-disk sweep; no-live-server disk clear; disk sweep still runs after an RPC failure; fail-closed disk-failure propagation as the primary clear and ignored when redundant after a live clear; no-server/no-dir warning; the configure clear-race guard (stale lastConfiguration not resurrected); and clearCacheDirectory clearing the correct dir before a finder exists (with fsPath comparisons and scoped settings).

Pre-commit checks: npm run lint ✓, npm run compile-tests ✓, npm run unittest ✓ (1743 passing, 5 pending).

[StellaHuang95]

🔒 Automated review in progress — @StellaHuang95 is auto-reviewing this PR.

[StellaHuang95]

GitHub cannot anchor PR review comments to unchanged lines in the diff. Falling back to a general PR comment for src/managers/common/nativePythonFinder.ts:L699.

Warning · Non-blocking recommendation

clearCache() advances the in-memory generation but does not cancel or drain already-queued discovery work. That work can run after the PET RPC and disk sweep, recreating PET-owned disk caches even though its result is prevented from repopulating the map. Please add a worker barrier/cancellation strategy and cover queued pre-clear work resuming after clear.

[StellaHuang95]

Warning · Non-blocking recommendation

The CLI fallback uses the captured ConfigurationOptions but rereads venvFolders when the queued task executes. A settings change between scheduling and execution can therefore make its search paths differ from the configuration used for coalescing and cache tagging. Capture these paths with the refresh task as well.