Skip to content

Commit 655b075

Browse files
Add inline script environment lifecycle telemetry (PEP 723 PR 15/16) (#1723)
> Part of #1602 (PEP 723 inline script env support). Design doc: #1601. > > Builds on the merged creation, uv-fallback, and persistence work in #1656, #1696, and #1697. This branch is rebased on current `main`. ### Roadmap context This is **PR 15 of 16** in the PEP 723 inline-script roadmap. It adds the remaining lifecycle telemetry for the internal manager without changing routing, creation, or cache policy. | Phase 5: Lifecycle and polish | PR | Status | |---|---|---| | | PR 13: clear inline-script cache | separate | | | PR 14: opportunistic TTL eviction | follow-up | | | **PR 15: lifecycle telemetry** | **this PR** | | | PR 16: status-bar decision | resolved; no code PR | ### Why this PR The manager can create, reuse, and persist inline-script environments, including consent-gated uv/Python fallback, but those lifecycle outcomes are not observable as a coherent feature funnel. This PR adds low-cardinality telemetry that answers: - whether setup built a new environment or reused a validated cache entry; - how long actual environment construction took; - how many normalized dependencies were involved; and - which stable failure category prevented setup. The instrumentation is deliberately attached to the underlying coalesced operation rather than every caller, and it excludes script/package/interpreter content. ### What this PR does **Adds three typed lifecycle events** - `inlineScript.envCreated` - `inlineScript.envReuseHit` - `inlineScript.envError` The telemetry constants and GDPR declarations use typed event/property mappings so call sites cannot send undeclared fields. **Reports verified creation only** - Emits `envCreated` only after environment creation succeeds, ownership/version validation passes, and `.meta.json` is persisted. - Measures the actual build/rebuild interval rather than lock waiting or cache inspection. - Reports normalized, deduplicated dependency count rather than raw dependency values. - Emits one creation event for the underlying coalesced operation, even when multiple callers await it. **Reports validated cache reuse only** - Emits `envReuseHit` only after sidecar, interpreter, ownership, and compatibility checks complete successfully. - Does not report a hit for uncertain, stale, malformed, or partially validated entries. - Preserves the existing cache reuse and `lastUsedAt` behavior. **Classifies lifecycle failures without leaking details** `envError` uses stable low-cardinality categories for outcomes such as: - environment discovery failure; - no compatible Python; - declined compatible-Python installation; - uv/Python installation failure; - general setup, cache-validation, or metadata-persistence failure; - package-install cancellation; - cache-lock timeout; and - retained, orphaned, or otherwise unavailable lock state. Errors remain logged through the existing paths; telemetry sends no exception text. **Preserves uv consent and compatibility behavior** - Adds detailed internal uv lookup/install result types so the manager can distinguish `available`, `declined`, `failed`, and `installed` outcomes. - Keeps existing compatibility wrappers for callers that only need boolean/path results. - Does not change prompt text, consent requirements, install selectors, restart-required behavior, or uv's existing telemetry. **Keeps event emission coalesced and deterministic** - Same-key concurrent creation/reuse callers receive one lifecycle result for the shared operation. - Cache inspection, lock acquisition, fallback selection, cancellation, and cleanup retain their existing control flow. - Non-applicable `create()` calls and pre-validation exits do not emit success-shaped lifecycle events. ### Event payloads and privacy | Event | Data | |---|---| | `inlineScript.envCreated` | build duration; normalized dependency count | | `inlineScript.envReuseHit` | normalized dependency count | | `inlineScript.envError` | stable failure category | The events send no: - script URI or filesystem path; - requirement or dependency value; - package name; - Python/interpreter version; - cache key; - prompt text; or - exception/error-message content. Script-controlled metadata therefore cannot create unbounded telemetry dimensions. ### Lifecycle examples ```text validated cache hit → emit one inlineScript.envReuseHit ``` ```text cache miss or stale entry → start build timer → build + install dependencies → validate ownership/version → persist sidecar → emit one inlineScript.envCreated ``` ```text no compatible Python → user dismisses consent prompt → emit one inlineScript.envError with the declined category ``` ### Tests Coverage includes: - creation emitted only after verified sidecar persistence; - validated reuse and rebuild behavior; - one event for concurrent/coalesced callers; - duration boundaries excluding lock wait/cache inspection; - normalized/deduplicated dependency counts; - discovery, compatibility, consent, installation, cancellation, setup, and lock categories; - no lifecycle telemetry for non-applicable calls; - direct detailed uv results for available, declined, failed, and installed outcomes; and - preservation of existing uv wrapper behavior. Validation on the rebased branch: - `npm run compile-tests` - `npm run compile` - `npm run lint` - focused lifecycle/detailed-uv suites: 39 passing The full Windows unit run reaches 1612 passing and 5 pending; the existing concurrent `writeMetaJson` rename test can still intermittently fail with `EPERM` on Windows. That writer is unchanged by this PR and the same failure is reproducible on `main`. ### Performance - No activation work, scan, timer, watcher, or new filesystem operation is introduced. - Instrumentation performs constant-size event construction around operations that already occur. - Coalesced setup emits once rather than once per waiter. - No user-controlled strings are normalized or transmitted beyond the dependency count already needed for the cache operation. ### User impact **No default-path user impact.** The inline manager remains behind the undeclared, default-off `python-envs.inlineScripts.enabled` flag. When the internal flag is manually enabled, prompts, environment creation/reuse, cancellation, error propagation, and cache behavior remain unchanged. This PR only records privacy-safe lifecycle outcomes. ### Scope and follow-up This PR intentionally does **not** implement: - script detection or automatic routing; - activation-time cache discovery; - project registration or user-facing setup UX; - cache clearing or TTL eviction; or - status-bar behavior. The telemetry is ready for those later entry points to consume once the feature is intentionally exposed. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3cb82ae9-7424-40a4-9156-8c54ac6e0895 Copilot-Session: 6b12d843-8011-4bfc-9ba9-f75761eadee2
1 parent 6104bbb commit 655b075

5 files changed

Lines changed: 704 additions & 95 deletions

File tree

‎src/common/telemetry/constants.ts‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -221,6 +221,29 @@ export enum EventNames {
221221
* - dependencyCount: number (number of entries in the `dependencies` list)
222222
*/
223223
INLINE_SCRIPT_DETECTED = 'inlineScript.detected',
224+
/**
225+
* Telemetry event fired when inline-script environment creation completes
226+
* successfully with a newly-built cache entry that passed verification and
227+
* metadata persistence.
228+
* Measures:
229+
* - duration: number (ms spent in the underlying create/rebuild operation)
230+
* - dependencyCount: number (normalized dependency count in the cache key)
231+
*/
232+
INLINE_SCRIPT_ENV_CREATED = 'inlineScript.envCreated',
233+
/**
234+
* Telemetry event fired when inline-script environment creation validates
235+
* and reuses an existing cache entry without rebuilding it.
236+
* Measures:
237+
* - dependencyCount: number (normalized dependency count in the cache key)
238+
*/
239+
INLINE_SCRIPT_ENV_REUSE_HIT = 'inlineScript.envReuseHit',
240+
/**
241+
* Telemetry event fired when inline-script environment creation cannot
242+
* complete.
243+
* Properties:
244+
* - category: stable low-cardinality failure category
245+
*/
246+
INLINE_SCRIPT_ENV_ERROR = 'inlineScript.envError',
224247
/**
225248
* Telemetry event fired once per session, per URI, the first time a `.py`
226249
* file that previously raised an `inlineScript.detected` event receives a
@@ -232,6 +255,16 @@ export enum EventNames {
232255
INLINE_SCRIPT_EDITED = 'inlineScript.edited',
233256
}
234257

258+
export type InlineScriptEnvErrorCategory =
259+
| 'compatible-python-declined'
260+
| 'discovery-failure'
261+
| 'no-compatible-python'
262+
| 'package-install-cancelled'
263+
| 'install-failure'
264+
| 'setup-failure'
265+
| 'lock-timeout'
266+
| 'lock-unavailable';
267+
235268
// Map all events to their properties
236269
export interface IEventNamePropertyMapping {
237270
/* __GDPR__
@@ -695,6 +728,36 @@ export interface IEventNamePropertyMapping {
695728
errorType?: string;
696729
};
697730

731+
/* __GDPR__
732+
"inlineScript.envCreated": {
733+
"dependencyCount": { "classification": "SystemMetaData", "purpose": "FeatureInsight", "isMeasurement": true, "owner": "StellaHuang95" },
734+
"<duration>": { "classification": "SystemMetaData", "purpose": "FeatureInsight", "isMeasurement": true, "owner": "StellaHuang95" }
735+
}
736+
*/
737+
[EventNames.INLINE_SCRIPT_ENV_CREATED]: {
738+
// Goes through the measures payload (numeric); listed here for GDPR only.
739+
dependencyCount?: number;
740+
};
741+
742+
/* __GDPR__
743+
"inlineScript.envReuseHit": {
744+
"dependencyCount": { "classification": "SystemMetaData", "purpose": "FeatureInsight", "isMeasurement": true, "owner": "StellaHuang95" }
745+
}
746+
*/
747+
[EventNames.INLINE_SCRIPT_ENV_REUSE_HIT]: {
748+
// Goes through the measures payload (numeric); listed here for GDPR only.
749+
dependencyCount?: number;
750+
};
751+
752+
/* __GDPR__
753+
"inlineScript.envError": {
754+
"category": { "classification": "SystemMetaData", "purpose": "FeatureInsight", "owner": "StellaHuang95" }
755+
}
756+
*/
757+
[EventNames.INLINE_SCRIPT_ENV_ERROR]: {
758+
category: InlineScriptEnvErrorCategory;
759+
};
760+
698761
/* __GDPR__
699762
"inlineScript.detected": {
700763
"trigger": { "classification": "SystemMetaData", "purpose": "FeatureInsight", "owner": "StellaHuang95" },

0 commit comments

Comments
 (0)