feat(functions-compiler): plan, size and split an app's function shards

[yurynix]

Note

Description

Moves the rest of apper's fresh-build function pipeline into @base44/functions-compiler, so the CLI and apper's bundler service run one engine end to end. The package no longer just compiles a single module per call: it now assembles each function's reachable sources, plans the shard partition, compiles each shard, measures it against Cloudflare's ceilings, and halves any shard that is over one — all before anything is uploaded. It also stamps every compiled shard with a first-line manifest, so a script pulled back from Cloudflare can say what is inside it.

Related Issue

Follow-up to #621 (extraction of the production function compiler into this monorepo). Ports apper's function_bundle.py, shard_planning.py, cloudflare_wfp_runtime.py and apper PR #23460.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Refactoring (no functional changes)
• Other (please describe):

Changes Made

• Source assembly (src/assembly.ts): collectReachableFiles / cfwBundleInput walk a function's relative imports with esbuild's metafile and hand the compiler exactly the files the function reaches. Single-file functions keep the flat main.ts submission byte-for-byte; escapes into the frontend tree stay forbidden.
• Shard planning (src/shards/plan.ts): targetShardCount, assertWithinCapacity and planFreshShards for the fresh (no previous deploy) path. Capacity is judged at the global shard size so a ratcheted-down packing size never locks an app out; the single-shard path keeps caller order while a multi-shard plan sorts by name, as apper does, because order changes the emitted bytes.
• Size measurement (src/shards/size.ts): raw UTF-8 bytes and level-6 gzip judged against Cloudflare's 64 MiB uncompressed ceiling and the supplied compressed cap. Compression runs off the event loop.
• Split on overflow + whole-build validation (src/shards/build.ts): compileFunctionShards compiles shards with bounded concurrency, recursively halves any oversized shard, and refuses a partial result — every declared function must land in exactly one successful shard. Duplicate names are rejected before anything compiles.
• Self-describing bundles: prepareApp now emits a //!b44:1 {...} banner (sorted function names, wrapper flags, compiler version), prepended verbatim by esbuild so it survives minification. It deliberately carries nothing volatile, so unchanged code still hashes to the same version. The legacy single-function bundle() lane stays bannerless and byte-comparable with the engine apper still runs.
• src/version.ts: the compiler version is now a literal rather than a runtime package.json read, which silently degraded to "unknown" once the CLI bundled the module — making the same functions compile to different bytes in the CLI and in the service. version.test.ts fails the build if the literal drifts from package.json.
• Fixes found while porting: a zero or negative shardSize hung the chunking loop instead of refusing; the reachability walk rejected top-level await and the path.sep working dir that the compiler itself accepts; an unparseable source now falls back to the flat submission (so the compiler reports the real diagnostic) instead of throwing out of assembly, while a genuine walk-configuration fault still propagates.
• Public surface: the new assembly, plan, size and shard-build entry points are re-exported from src/index.ts and pinned by the existing published-surface test.
• README: rewritten from "what is missing" to what the package now does, with a provenance table, the banner format, the reproducibility contract, and the two deliberate divergences from apper's Python (unparseable-source handling, and Node's gzip reading ~0.7% heavier than Python's).

Testing

• I have tested these changes locally
• I have added/updated tests as needed
• All tests pass (npm test) — 295 of 296. The one failure is shared-dep-conflict.e2e.test.ts, which resolves date-fns/date-fns-tz live from the registry, is green in CI, and touches nothing on this branch

Six new test files (~1,000 lines): assembly.test.ts, shard-plan.test.ts, shard-size.test.ts, shard-build.e2e.test.ts, bundle-banner.e2e.test.ts, version.test.ts. They cover reachability against the Python fixtures, the capacity/ratchet and ordering rules, the one real 97 MB production rejection, multi-level split recursion, byte-identical recompiles, and banner survival past minification; the shard-build and banner suites are real compiles rather than mocks. CI runs the package's bun run test — the two boxes above are left unchecked because the suite was not executed in this session.

Checklist

• My code follows the project's style guidelines
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation (if applicable)
• My changes generate no new warnings
• I have updated docs/ (AGENTS.md) if I made architectural changes

Additional Notes

Auto PR Description regenerates this body on every push, so the four paragraphs below are re-applied by hand after each one. They are the parts that cannot be read off the diff.

Reviewed independently, and the findings are fixed. A Claude Fable review (full findings in the thread) raised one blocker and four fix-here defects, all now on this branch: the reachability walk compiled as iife and so rejected top-level await, which the real compiler accepts; it passed path.sep as a working directory, which is not absolute on Windows and failed every walk there; the fallback caught the compiler's own configuration faults and relabelled them as the user's unresolved import, which is what hid the first two; shardSize: 0 hung a chunking loop instead of being refused; and the banner's version, read from package.json at runtime, degraded to "unknown" in the artifact the CLI bundles this module into, so the two lanes emitted different bytes.

Two of the review's follow-ups are also in. The split recursion is now driven past one level — eight functions under a cap between a pair's size and a quad's, which only four shards of two can satisfy — along with the case where one function survives every halving and must fail the whole build rather than emit its siblings. And biome now passes over this PR's test files: CI lints packages/*/src and packages/*/tests while this package uses test/, so nothing was checking them and green CI was not evidence about them.

Three follow-ups are deliberately still out: a split's halves run outside the concurrency limiter, over-capacity throws where compile failures return ok: false, and an empty function list yields one empty shard.

Two accepted trade-offs, decided rather than overlooked. The compiler version stays in the emitted bytes, so a release of this package re-mints a version for every app even when the compiled body is unchanged — accepted in exchange for artifacts that name what produced them. And when apper adopts this package the banner arrives with it, so every deployed per-app shard re-mints once, fleet-wide; that is planned as part of adoption.

One thing for base44-dev/apper#25154 to absorb. That PR asserts the packaged compiler and apper's in-repo engine emit byte-identical modules, and its table includes a two-function app bundle. The banner breaks that row by design, and it is deliberately not mirrored into the in-repo engine — apper inherits the banner by adopting this package, and that engine is on its way out. While both exist, that row is a transitional check and belongs narrowed to the single-function shapes, which stay bannerless.

Only the fresh-build slice crosses over. Incremental shard reuse, the per-app shard_size_override ratchet, entitlement and settings reads, provider upload, binding resolution and secret delivery stay in apper — policy numbers arrive as inputs and the package reads no configuration of its own. Nothing in the CLI calls the new surface yet; this PR lands the engine.

---

_{🤖 Generated by Claude | 2026-09-17 13:56 UTC | 1596711}

[github-actions]

🚀 Package Preview Available!

---

Install this PR's preview build with npm:

npm i @base44-preview/cli@0.1.15-pr.623.1596711

Prefer not to change any import paths? Install using npm alias so your code still imports base44:

npm i "base44@npm:@base44-preview/cli@0.1.15-pr.623.1596711"

Or add it to your package.json dependencies:

{
  "dependencies": {
    "base44": "npm:@base44-preview/cli@0.1.15-pr.623.1596711"
  }
}

• 📦 Preview Package: @base44-preview/cli@0.1.15-pr.623.1596711
• 🔗 View this commit on GitHub

---

_{Preview published to npm registry — try new features instantly!}