feat: confirm inline-script env setup with a short-lived CodeLens (#1790)

## Problem

Setting up a PEP 723 inline-script environment signals success only by
the setup CodeLens **disappearing** — which is indistinguishable from
the lens never having been offered in the first place. It also leaves
the chosen base interpreter invisible, which matters when
`requires-python` matches several installed Pythons, or when one was
installed on demand via `uv python install`.

## Change

For five seconds after setup succeeds, the (now hidden) setup lens is
replaced by a passive confirmation anchored at the `# /// script` block:

> `Script environment ready (Python 3.12.4)`

then it expires on its own. The version comes from
`shortenVersionString(environment.version)` — the same helper
`getPythonInfo` uses for `script env (3.12.4)` — so the lens and the
environment's own name cannot disagree.

An empty command id renders the title as plain, non-clickable text: this
is a statement, not an action.

### No extra reflow

The line is already occupied by the setup lens at that moment, so the
confirmation does not add a shift — it delays the single existing one by
five seconds. This was the main objection to an earlier draft, and it
turned out not to apply.

## Scope

The entire behavioural change is one new row in `provideCodeLenses`:

| `isDirty` | metadata | `shouldRoute` | confirmation live | Before |
After |
|---|---|---|---|---|---|
| yes | — | — | — | `[]` | `[]` |
| no | none | — | — | `[]` | `[]` |
| no | yes | yes | **yes** | `[]` | **confirmation** |
| no | yes | yes | no | `[]` | `[]` |
| no | yes | no | — | setup lens | setup lens |

Everything else is byte-identical: the dirty guard, the metadata check,
`shouldRoute` semantics, and the setup lens title, command, arguments
and anchor. `routingRegistry.ts`, `envManager.ts`, `window.apis.ts`,
settings and `package.json` are untouched.

Both setup surfaces — the CodeLens and the unresolved-import quick fix
from #1788 — show the confirmation, since both route through
`python-envs.setupInlineScriptEnv`. The bulk command deliberately does
not: it already ends with its own `Set up {0} of {1} ...` summary and
can process scripts that are not open.

## Implementation notes

- `noteEnvironmentReady` cancels any existing timer for the script
first, so re-running setup restarts the window rather than inheriting a
nearly-expired one.
- Entries are keyed by `getInlineScriptRoutingKey`, so on Windows
`C:\App.py` and `c:\app.py` share one entry.
- `dispose()` clears every pending timer; a test asserts nothing fires
afterwards.
- Confirmations are in-memory and per-window, deliberately not persisted
— a confirmation is about an action you just took.

## Known limitation

If the file is edited **while** the environment builds,
`getSavedMetadataForPersistence` returns `{}` for the dirty document, so
`updateValidatedStateForSelection` leaves `shouldRoute` false and the
confirmation is unreachable until the next save — and lost entirely if
that takes longer than five seconds. Showing it anyway would mean
confirming an association that is not validated, so the miss is
preferable to the lie.

Expiry also depends on VS Code re-querying after
`onDidChangeCodeLenses`. That is normally immediate, but the lens can
outlast five seconds slightly under load. It always clears.

## Tests

Six new provider tests on `sinon.useFakeTimers()` (no wall-clock
dependency): shows with version, omits version when unresolved, expires
and fires exactly one refresh, routed-but-not-just-set-up shows nothing,
hidden while dirty, and no timer leak past `dispose()`. Three handler
tests cover the callback firing on success and staying silent when
creation returns nothing or throws.

The five pre-existing CodeLens tests are unmodified and still pass —
including *"hides the CodeLens once a validated association makes the
script routeable"*, which is the proof that the default routed path is
unchanged.

`npm run lint` OK, `npm run compile-tests` OK, `npm run unittest` OK
(2331 passing, 6 pending, 0 failing)

> `discovers a build that completes after the short retry window` is a
pre-existing flake unrelated to this change — verified by stashing these
changes and running it on a clean tree, where it failed 2 of 4 runs.

**Nothing here is user-visible by default**: the whole surface stays
behind the undeclared internal flag `python-envs.inlineScripts.enabled`,
which defaults to `false`.

---------

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Author: StellaHuang95

docs/managing-python-projects.md

@@ -99,6 +99,8 @@ When you create a script, the extension generates a single `.py` file with PEP 7
 
 An inline-script environment is built from the script's `# /// script` block and stored in the extension's cache, where it is shared by every script with the same dependencies and base interpreter. Because editing one would silently change the others, these environments are not user-managed: the Python Environments views do not offer install, uninstall, or version-change actions for them. Their package list remains visible.
 
+A CodeLens above the `# /// script` block offers **Set up environment for this script**, and the same action is available as a quick fix on an unresolved import. For a few seconds after setup succeeds it is replaced by a **Script environment ready (Python X.Y.Z)** confirmation naming the Python that was selected — useful when `requires-python` matches several installed versions, or when one was installed on demand. The confirmation is plain text rather than a clickable action, and it expires on its own; at every other time the setup CodeLens behaves exactly as before.
+
 Setup records which distributions it installed. If that record and the environment's contents later disagree — for example after installing a package into it from a terminal — every script sharing the environment needs setup again. Saving or reopening a script does not repair it; use the script's setup action to rebuild from its declared dependencies.
 
 Once a mismatch is confirmed during an environment lookup, the affected scripts' setup actions return without requiring a save.

src/common/localize.ts

@@ -44,6 +44,13 @@ export namespace InlineScriptStrings {
 
     export const diagnosticSource = l10n.t('Python Environments');
 
+    export function environmentReady(version: string | undefined): string {
+        const shown = version?.trim();
+        return shown
+            ? l10n.t('Script environment ready (Python {0})', shown)
+            : l10n.t('Script environment ready');
+    }
+
     export const unterminatedBlock = l10n.t(
         "This '# /// script' block is missing its closing '# ///' marker, so its inline script metadata is ignored.",
     );

src/features/inlineScript/codeLens.ts

@@ -11,8 +11,12 @@ import {
     languages,
     Range,
     TextDocument,
+    Uri,
 } from 'vscode';
-import { InlineScriptRoutingRegistry } from '../../common/inlineScript/routingRegistry';
+import { getInlineScriptRoutingKey, InlineScriptRoutingRegistry } from '../../common/inlineScript/routingRegistry';
+import { InlineScriptStrings } from '../../common/localize';
+
+export const READY_CONFIRMATION_TIMEOUT_MS = 5000;
 
 /**
  * Shows a single "Set up environment for this script" CodeLens above a `.py` file's PEP 723
@@ -29,10 +33,16 @@ export class InlineScriptCodeLensProvider implements CodeLensProvider, Disposabl
     private readonly _onDidChangeCodeLenses = new EventEmitter<void>();
     public readonly onDidChangeCodeLenses = this._onDidChangeCodeLenses.event;
     private readonly subscriptions: Disposable[] = [];
+    private readonly readyConfirmations = new Map<
+        string,
+        { readonly version: string | undefined; readonly timer: ReturnType<typeof setTimeout> }
+    >();
+    private disposed = false;
 
     constructor(
         private readonly routing: InlineScriptRoutingRegistry,
         private readonly setupCommand: string,
+        private readonly confirmationTimeoutMs: number = READY_CONFIRMATION_TIMEOUT_MS,
     ) {
         this.subscriptions.push(
             this.routing.onDidChangeRouteability(() => this._onDidChangeCodeLenses.fire()),
@@ -47,6 +57,22 @@ export class InlineScriptCodeLensProvider implements CodeLensProvider, Disposabl
         );
     }
 
+    public noteEnvironmentReady(uri: Uri, version: string | undefined): void {
+        const key = getInlineScriptRoutingKey(uri);
+        if (this.disposed || !key) {
+            return;
+        }
+        this.clearConfirmation(key);
+        this.readyConfirmations.set(key, {
+            version,
+            timer: setTimeout(() => {
+                this.readyConfirmations.delete(key);
+                this._onDidChangeCodeLenses.fire();
+            }, this.confirmationTimeoutMs),
+        });
+        this._onDidChangeCodeLenses.fire();
+    }
+
     public provideCodeLenses(document: TextDocument, _token: CancellationToken): CodeLens[] {
         if (document.isDirty) {
             // The association is validated against the saved file (the manager refuses to validate a
@@ -60,13 +86,24 @@ export class InlineScriptCodeLensProvider implements CodeLensProvider, Disposabl
             // No saved PEP 723 metadata (or it is currently being edited).
             return [];
         }
-        if (this.routing.shouldRoute(uri)) {
-            // A validated inline-script environment matching the current metadata already exists.
-            return [];
-        }
         const offset = metadata.sourceRange?.start ?? metadata.range.start;
         const position = document.positionAt(offset);
         const range = new Range(position, position);
+        if (this.routing.shouldRoute(uri)) {
+            // A validated inline-script environment matching the current metadata already exists.
+            const key = getInlineScriptRoutingKey(uri);
+            const confirmation = key ? this.readyConfirmations.get(key) : undefined;
+            if (!confirmation) {
+                return [];
+            }
+            // An empty command id renders the title as plain, non-clickable text.
+            return [
+                new CodeLens(range, {
+                    title: InlineScriptStrings.environmentReady(confirmation.version),
+                    command: '',
+                }),
+            ];
+        }
         return [
             new CodeLens(range, {
                 title: l10n.t('Set up environment for this script'),
@@ -77,21 +114,38 @@ export class InlineScriptCodeLensProvider implements CodeLensProvider, Disposabl
     }
 
     public dispose(): void {
+        this.disposed = true;
         this.subscriptions.forEach((s) => s.dispose());
         this.subscriptions.length = 0;
+        this.readyConfirmations.forEach((confirmation) => clearTimeout(confirmation.timer));
+        this.readyConfirmations.clear();
         this._onDidChangeCodeLenses.dispose();
     }
+
+    private clearConfirmation(key: string): void {
+        const existing = this.readyConfirmations.get(key);
+        if (existing) {
+            clearTimeout(existing.timer);
+            this.readyConfirmations.delete(key);
+        }
+    }
 }
 
 /**
  * Register the inline-script CodeLens provider for local `.py` files. Only called when the PEP 723
  * inline-script feature flag is enabled, so it is a no-op for everyone else.
  */
-export function registerInlineScriptCodeLens(routing: InlineScriptRoutingRegistry, setupCommand: string): Disposable {
+export function registerInlineScriptCodeLens(
+    routing: InlineScriptRoutingRegistry,
+    setupCommand: string,
+): { readonly disposable: Disposable; readonly provider: InlineScriptCodeLensProvider } {
     const provider = new InlineScriptCodeLensProvider(routing, setupCommand);
     const registration = languages.registerCodeLensProvider({ scheme: 'file', language: 'python' }, provider);
-    return new Disposable(() => {
-        registration.dispose();
-        provider.dispose();
-    });
+    return {
+        provider,
+        disposable: new Disposable(() => {
+            registration.dispose();
+            provider.dispose();
+        }),
+    };
 }

src/features/inlineScript/setupEnvironment.ts

@@ -17,6 +17,7 @@ import {
 } from '../../common/window.apis';
 import { asRelativePath, findFiles, getOpenTextDocuments } from '../../common/workspace.apis';
 import { EnvironmentManagers } from '../../internal.api';
+import { shortenVersionString } from '../../managers/common/utils';
 import { registerInlineScriptCodeLens } from './codeLens';
 import { promptUpdateExtensionsForInlineScripts } from './extensionVersionCheck';
 import { registerInlineScriptSetupCodeAction } from './setupCodeAction';
@@ -134,6 +135,7 @@ async function saveScriptBeforeSetup(scriptUri: Uri, routing: InlineScriptRoutin
 export function setupInlineScriptEnvironmentHandler(
     em: EnvironmentManagers,
     routing: InlineScriptRoutingRegistry,
+    onEnvironmentReady?: (scriptUri: Uri, version: string | undefined) => void,
 ): (scriptUri?: Uri) => Promise<void> {
     return async (scriptUri?: Uri): Promise<void> => {
         const uri = scriptUri ?? window.activeTextEditor?.document.uri;
@@ -164,6 +166,7 @@ export function setupInlineScriptEnvironmentHandler(
             notifyInlineScriptSetupOutcome(uri, routing);
             return;
         }
+        onEnvironmentReady?.(uri, shortenVersionString(environment.version));
         // Kept out of the try: the environment is already set up, so a failure in this follow-up
         // must not be reported to the user as a setup failure.
         await promptUpdateExtensionsForInlineScripts().catch((error) =>
@@ -362,10 +365,16 @@ async function filterInlineScriptFiles(files: readonly Uri[]): Promise<Uri[]> {
  * palette-gated behind the flag.
  */
 export function registerInlineScriptUx(em: EnvironmentManagers, routing: InlineScriptRoutingRegistry): Disposable[] {
+    const codeLens = registerInlineScriptCodeLens(routing, SETUP_INLINE_SCRIPT_ENV_COMMAND);
     return [
-        registerInlineScriptCodeLens(routing, SETUP_INLINE_SCRIPT_ENV_COMMAND),
+        codeLens.disposable,
         registerInlineScriptSetupCodeAction(routing, SETUP_INLINE_SCRIPT_ENV_COMMAND),
-        commands.registerCommand(SETUP_INLINE_SCRIPT_ENV_COMMAND, setupInlineScriptEnvironmentHandler(em, routing)),
+        commands.registerCommand(
+            SETUP_INLINE_SCRIPT_ENV_COMMAND,
+            setupInlineScriptEnvironmentHandler(em, routing, (uri, version) =>
+                codeLens.provider.noteEnvironmentReady(uri, version),
+            ),
+        ),
         commands.registerCommand(SETUP_INLINE_SCRIPT_ENVS_COMMAND, () =>
             setUpInlineScriptEnvironmentsInWorkspace(em, routing),
         ),

src/test/features/inlineScript/codeLens.unit.test.ts

@@ -2,10 +2,11 @@
 // Licensed under the MIT License.
 
 import assert from 'assert';
+import * as sinon from 'sinon';
 import { Position, TextDocument, Uri } from 'vscode';
 import { InlineScriptMetadata } from '../../../common/inlineScript/metadata';
 import { InlineScriptRoutingRegistry } from '../../../common/inlineScript/routingRegistry';
-import { InlineScriptCodeLensProvider } from '../../../features/inlineScript/codeLens';
+import { InlineScriptCodeLensProvider, READY_CONFIRMATION_TIMEOUT_MS } from '../../../features/inlineScript/codeLens';
 
 const SETUP_COMMAND = 'python-envs.setupInlineScriptEnv';
 
@@ -83,4 +84,65 @@ suite('Inline script CodeLens provider', () => {
         sub.dispose();
         assert.ok(fireCount >= 1, 'onDidChangeCodeLenses should fire when routing state changes');
     });
+
+    suite('post-setup confirmation', () => {
+        let clock: sinon.SinonFakeTimers;
+
+        setup(() => {
+            clock = sinon.useFakeTimers();
+            routing.setMetadata(scriptUri, makeMetadata());
+            routing.setValidatedAssociation(scriptUri, true);
+        });
+
+        teardown(() => clock.restore());
+
+        test('replaces the hidden setup lens with a non-clickable confirmation naming the version', () => {
+            provider.noteEnvironmentReady(scriptUri, '3.12.4');
+
+            const lenses = provider.provideCodeLenses(makeDocument(scriptUri), {} as never);
+
+            assert.strictEqual(lenses.length, 1);
+            assert.strictEqual(lenses[0].command?.title, 'Script environment ready (Python 3.12.4)');
+            assert.strictEqual(lenses[0].command?.command, '', 'the confirmation must not be clickable');
+        });
+
+        test('omits the version when none was resolved', () => {
+            provider.noteEnvironmentReady(scriptUri, undefined);
+
+            const lenses = provider.provideCodeLenses(makeDocument(scriptUri), {} as never);
+
+            assert.strictEqual(lenses.length, 1);
+            assert.strictEqual(lenses[0].command?.title, 'Script environment ready');
+        });
+
+        test('expires on its own and refreshes so the lens disappears', () => {
+            provider.noteEnvironmentReady(scriptUri, '3.12.4');
+            let fireCount = 0;
+            const sub = provider.onDidChangeCodeLenses(() => (fireCount += 1));
+
+            clock.tick(READY_CONFIRMATION_TIMEOUT_MS + 1);
+            sub.dispose();
+
+            assert.strictEqual(fireCount, 1, 'expiry must refresh the lenses');
+            assert.strictEqual(provider.provideCodeLenses(makeDocument(scriptUri), {} as never).length, 0);
+        });
+
+        test('shows nothing for a routed script that was not just set up', () => {
+            assert.strictEqual(provider.provideCodeLenses(makeDocument(scriptUri), {} as never).length, 0);
+        });
+
+        test('stays hidden while the document is dirty', () => {
+            provider.noteEnvironmentReady(scriptUri, '3.12.4');
+
+            assert.strictEqual(provider.provideCodeLenses(makeDocument(scriptUri, true), {} as never).length, 0);
+        });
+
+        test('does not leak timers past disposal', () => {
+            provider.noteEnvironmentReady(scriptUri, '3.12.4');
+
+            provider.dispose();
+
+            assert.doesNotThrow(() => clock.tick(READY_CONFIRMATION_TIMEOUT_MS + 1));
+        });
+    });
 });

src/test/features/inlineScript/setupEnvironment.unit.test.ts

@@ -327,6 +327,7 @@ suite('setupInlineScriptEnvironmentHandler', () => {
     let errorStub: sinon.SinonStub;
     let saveStub: sinon.SinonStub;
     let promptStub: sinon.SinonStub;
+    let readySpy: sinon.SinonStub;
 
     setup(() => {
         em = typemoq.Mock.ofType<EnvironmentManagers>();
@@ -338,6 +339,7 @@ suite('setupInlineScriptEnvironmentHandler', () => {
         errorStub = sinon.stub(winapi, 'showErrorMessage').resolves(undefined);
         sinon.stub(winapi, 'showInformationMessage').resolves(undefined);
         sinon.stub(winapi, 'showWarningMessage').resolves(undefined);
+        readySpy = sinon.stub();
         promptStub = sinon.stub(extensionVersionCheck, 'promptUpdateExtensionsForInlineScripts').resolves();
         saveStub = sinon.stub().resolves(true);
     });
@@ -358,8 +360,7 @@ suite('setupInlineScriptEnvironmentHandler', () => {
         return env;
     }
 
-    test('saves a dirty document before setup, because setup reads the block from disk', async () => {
-        openDirtyDocument();
+    test('saves a dirty document before setup, because setup reads the block from disk', async () => {        openDirtyDocument();
         expectEnvironmentCreated();
 
         await setupInlineScriptEnvironmentHandler(em.object, routing)(scriptUri);
@@ -417,12 +418,35 @@ suite('setupInlineScriptEnvironmentHandler', () => {
         sinon.assert.calledOnce(errorStub);
     });
 
-    test('does not report a failing companion-extension prompt as a setup failure', async () => {
-        expectEnvironmentCreated();
+    test('does not report a failing companion-extension prompt as a setup failure', async () => {        expectEnvironmentCreated();
         promptStub.rejects(new Error('boom'));
 
         await setupInlineScriptEnvironmentHandler(em.object, routing)(scriptUri);
 
         sinon.assert.notCalled(errorStub);
     });
+
+    test('reports the ready environment and its Python version once setup succeeds', async () => {
+        expectEnvironmentCreated();
+
+        await setupInlineScriptEnvironmentHandler(em.object, routing, readySpy)(scriptUri);
+
+        sinon.assert.calledOnceWithExactly(readySpy, scriptUri, '3.12.0');
+    });
+
+    test('reports no ready environment when setup produced none', async () => {
+        manager.setup((m) => m.create(scriptUri, undefined)).returns(() => Promise.resolve(undefined));
+
+        await setupInlineScriptEnvironmentHandler(em.object, routing, readySpy)(scriptUri);
+
+        sinon.assert.notCalled(readySpy);
+    });
+
+    test('reports no ready environment when setup throws', async () => {
+        manager.setup((m) => m.create(scriptUri, undefined)).returns(() => Promise.reject(new Error('boom')));
+
+        await setupInlineScriptEnvironmentHandler(em.object, routing, readySpy)(scriptUri);
+
+        sinon.assert.notCalled(readySpy);
+    });
 });