Skip to content

perf(pi-permission-system): load a prebuilt native runtime - #1114

Open
hyein-cbio wants to merge 2 commits into
gotgenes:mainfrom
hyein-cbio:perf/pi-permission-system-native-loading
Open

hyein-cbio wants to merge 2 commits into
gotgenes:mainfrom
hyein-cbio:perf/pi-permission-system-native-loading

Conversation

@hyein-cbio

Copy link
Copy Markdown

Motivation

Closes #1113.

A local investigation of loading time across the installed extensions identified pi-permission-system as one of the largest contributors.
This PR addresses that loading overhead without changing permission policies or skipping enforcement.
The package-specific benchmark below is separate from that local cross-extension observation.

Changes

  • Retain the TypeScript source and build one executable dist/runtime.cjs bundle during prepack, alongside the existing public declarations.
  • Point pi.extensions at a small extension.ts entry that native-loads the runtime instead of sending the full TS graph through Jiti.
  • Reuse Pi's host API/TUI modules, keep dependencies external, and preserve source-relative locations used for WASM resolution, legacy config paths, and infrastructure discovery.
  • Warn and fall back to TS for JS loading/evaluation errors or an invalid factory; propagate errors once factory initialization has begun to avoid duplicate hook registration.
  • Refresh only this runtime's native cache entry on reload and preserve node/session isolation.
  • Add loader and native-artifact regression tests, a reproducible loading benchmark, and raw measurement data.

All changed files are under packages/pi-permission-system/.
There are no changes to permission-gate logic, policy format, defaults, or the public service API, and no new dependencies or consumer installation scripts.

Loading measurements

Pi 1.5.0, Node v26.13.0 benchmark driver, macOS arm64.
Each run starts an actual Pi CLI process in offline RPC mode with only this extension, a disposable agent directory, YOLO on, and both logs off.
There are nine trials per condition after warm-up, alternating the order of source and compiled runs.

Median TS source Native JS runtime Reduction
Warm transform cache: import + factory 173ms 127ms 26.6%
Disabled transform cache: import + factory 624ms 177ms 71.6%
Warm transform cache: whole CLI process 351ms 299ms —
Disabled transform cache: whole CLI process 812ms 359ms —

These are isolated measurements, not a claim about the improvement of a full multi-extension TUI startup.
Disabling transform caching does not clear the OS filesystem cache.
The warm-up samples are retained, but these measurements do not establish first-install performance.

Raw trials, measurement metadata, and runtime hash.

Reproduce from the repository root:

pnpm --filter @gotgenes/pi-permission-system run build:runtime
PI_BIN="$(command -v pi)" pnpm --filter @gotgenes/pi-permission-system run measure:loading

Validation

  • pnpm run check — entire workspace.
  • pnpm run lint — repository-wide Biome, ESLint, Markdown and character checks.
  • pnpm run test — all workspace package suites and root script tests; pi-permission-system has 6,441 passing tests.
  • pnpm pack — runtime, declarations, entry and TS source included; development tests/scripts and raw benchmark data excluded.
  • Packaged manifest entry loads and completes its session lifecycle on Pi 1.0.0 and 1.5.0.
  • An intentionally corrupted JS runtime in a disposable copy warns and successfully falls back to TS on Pi 1.0.0 and 1.5.0.
  • Native-artifact tests cover actual YOLO ask approval, explicit denies, unknown-tool rejection, WASM-backed bash checks, fail-closed exceptions, service publication/teardown and concurrent session-approval isolation.
  • Deliberate mutations to YOLO handling, fail-closed blocking and session-state isolation each caused the relevant new regression test to fail; all mutations were restored.

These checks do not claim exhaustive verification of interactive TUI dialogs or real cross-process subagent approval round-trips through the new bundled entry.
The existing source-based suites still pass.

Development behavior

If dist/runtime.cjs exists, the entry prefers it.
After editing TS in a checkout, rebuild with build:runtime before reloading Pi; an unbuilt checkout continues to use TS directly.
The real global Pi installation and user settings were not modified.

Build one CommonJS runtime during packaging and load it through a small
Pi-aware entry while retaining the TypeScript source as a fallback.

Keep Pi host modules external, preserve source-relative asset resolution,
and reload rebuilt native code without sharing session approval state.
Fall back before initialization on JS load errors, but propagate factory
errors to avoid duplicate permission hooks.

Add native-artifact and loader regression tests plus a reproducible,
isolated CLI loading benchmark with raw measurements.

Validated with the package test suite, typecheck, lint, and packaged
extension loading on Pi 1.0.0 and 1.5.0.
Reproduce a packed checkout ignoring a newer source-only command because
extension.ts continues loading the old runtime snapshot.

Remove the working-tree runtime after packing and on public-type verifier
exit, including failed packing. Before native loading, check recursive
TS/TSX source mtimes and fall back to source when the bundle is stale or
freshness cannot be established. Accept equal packed-file timestamps.

Add freshness and failure-path regressions without changing permission
logic. Verify packed native loading and edited-source fallback on Pi
1.0.0 and 1.5.0.

Validation: 186 test files, 6444 passing tests, package typecheck and lint,
public-type verification, and runtime absence after verification.

Refs gotgenes#1113
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pi-permission-system: reduce extension loading overhead with a prebuilt runtime

1 participant