Skip to content

feat(island): a bytecode-backed lazy island seed - #797

Draft
espetro wants to merge 14 commits into
work/ytbenchfrom
work/lazyseed
Draft

espetro wants to merge 14 commits into
work/ytbenchfrom
work/lazyseed

Conversation

@espetro

@espetro espetro commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Part of the yt-dlp composite goal. Base is work/ytbench (#796).

The program-wide island seed carried every module's source as a string literal, and the runtime compiled it at startup. It now carries per-module marshalled, deflated bytecode, loaded on first import through a meta-path finder. The optimizer level is pinned to 0 at build time so the bytecode keeps the same semantics as the source path.

Composite on the frozen runner (min of 7 interleaved samples), geometric mean over 11 dimensions:

Candidate vs B1 before vs B1 after vs B2 before vs B2 after
V-bundle 1.374 0.924 1.441 0.981
V-minimal 1.379 0.916 1.445 0.972
V-host 0.755 0.433 0.793 0.459

Startup fell from 1.08 s to 0.40 s for --version and RSS from 163 MiB to 55 MiB. The pass rule is below 1.0 against both baselines with no dimension worse than 2x. V-minimal meets the first part. The remaining dimension over 2x is files_n against B2 (a 153 file tree against one file), and install_bytes and artifact_bytes against B2 sit at 1.9x. A onefile mode is the next lever.

Semantics: the -v --simulate -j extract output matches B1 byte for byte except the wall-clock epoch field. tests/test_island_program.py, tests/test_plane_split.py and tests/test_fallback_definitions.py pass unchanged.

Cost: the compiler now compiles every island module at build time. Boxed ledger on this tip reads churn 53.9 percent because the new commit and its tests are a tiny window; re-measure at the boundary.

Details in .agents/notes/ (lazy seed note).

🤖 Generated with Claude Code

Agent added 3 commits October 30, 2026 19:46
The program-wide island seed carried every module's source text as a
string literal in one dict, so the binary held about 10.6 MB of Python
source and the runtime compiled that blob at startup (0.22 s) and kept
it resident (about 28 MiB).

The seed is now three parts: a small finder bootstrap, a marshalled
index of {name: (origin, is_package, offset, length)}, and a body blob
of per-module deflated code objects. The runtime hands the last two to
the bootstrap through the __pyscriptc_seed__ module, and the finder
inflates and unmarshals one module body per import, so a run touches
only the pages of the modules it imports.

The optimizer level is pinned to 0 at build time so the bytecode does
not depend on how the compiler was invoked. The module origins stay
compile-time paths, as before, so a traceback names the same file and
line it did when the source was compiled at run time.
@espetro espetro added this to the v0.8.0 milestone Oct 30, 2026
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Deploying pyscriptc with  Cloudflare Pages  Cloudflare Pages

Latest commit: ebc8887
Status: ✅  Deploy successful!
Preview URL: https://250b095f.pyscriptc.pages.dev
Branch Preview URL: https://work-lazyseed.pyscriptc.pages.dev

View logs

Agent added 4 commits November 2, 2026 11:52
…he bytecode island seed

Both sides repinned the self-host census, so the merge takes the sum:
the runtime-linked manifest check and the bytecode island seed land at
3302 self-host fences.
… island seed

The runtime-linked stdlib manifest check adds 7 self-host fences in
stdlibzip.py, so the bytecode-seed branch's census pin had to be
recomputed against the merged tree. The merged tree measures 3302
fences and the same package sum; the pin moves with a comment that names
every delta.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

1 participant