feat(functions-compiler): extract the production function compiler into the CLI monorepo

[yurynix]

Note

Description

Adds @base44/functions-compiler as a new workspace package at packages/functions-compiler — the production engine that turns backend-function sources into a single Cloudflare Workers module (generated worker entry, Deno shim, runtime and private-data-source assets, npm/jsr resolution through @deno/loader + esbuild). It is extracted verbatim from apper's infra/base44-userapp-bundler so the CLI can compile functions locally instead of calling a bundler service over HTTP, and both consumers run one engine. Nothing in packages/cli consumes it yet; this PR lands the package, its tests, its packaging proof and its CI lane.

Related Issue

None

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Refactoring (no functional changes)
• Other (please describe):

Changes Made

• New package packages/functions-compiler (@base44/functions-compiler, published restricted): bundle (one function) and bundleApp (several functions into one module with per-function status), error attribution, and src/index.ts as the sole public surface.
• Compiler internals: deno-bundle.ts plus the src/esbuild/ resolver and virtual-module plugins, worker-entry.ts, actor-compat.ts, the src/shim/ sources, and the src/runtime/ + src/private-data-sources/ modules that are read as text and injected into the user bundle.
• Host-injectable diagnostics instead of vendored ones: setCompilerTracer (default no-op, replaces a direct dd-trace import) and setLogSink (default remains the Datadog-shaped JSON writer), so the package ships no tracer and no service credentials.
• Process isolation: installFetchGuard() / createGuardedFetch() block outbound fetch other than the compiler's own dependency resolution.
• Packaging: exports resolves only through lib/ (the bun → src/index.ts condition was dropped, since the tarball does not ship src/); build runs the shim build, tsc -p tsconfig.build.json and scripts/copy-assets.ts so the compile-time .ts assets sit next to the compiled JS; files is lib + README.md, and publishConfig.access is pinned to restricted rather than relying on the scoped-package default.
• Tarball guard: scripts/verify-package.ts packs the tarball, asserts it contains only lib/, README.md and package.json, installs it in a directory that can see neither repo, and compiles a real function (with an npm: dependency) on the host node. It runs in CI, so widening files fails the build instead of shipping.
• CI: new .github/workflows/functions-compiler.yml — typecheck, the vitest suite (compiles real npm packages and executes output in workerd via miniflare), bun run build, then the packaged-asset verification.
• Repo config: biome.json excludes the compile-time asset paths (reformatting them would change every compiled worker's bytes); knip.json gains a workspace entry with src/index.ts as an explicit entry point; .npmrc maps @jsr: to https://npm.jsr.io for @deno/loader; bun.lock updated for the new workspace.
• Docs: package README.md covering status and scope (internal, restricted, carries no credentials of its own), the asset/"do not reformat" rule, layout, commands, and an explicit table of what has not moved yet (source assembly, shard planning, size measurement, split-on-overflow, whole-build validation); docs/AGENTS.md gains the package in the architecture list.
• Tests: 23 files / 237 tests. Beyond the suite that moved with the engine, this adds coverage for seams the move left open — real repository paths (base44/functions/<name>/entry.ts fixtures with cross-directory and transitive imports, plus refusal of ../ escapes, absolute paths and file: URLs), partial bundleApp results (ok: true with a failed function), the new setLogSink / setCompilerTracer hooks, and six engine assertions previously reachable only through apper's HTTP specs (nested function names containing /, single-inlining of a dependency shared by two functions, syntax-error file/line/column attribution, refusal of a file: URL reached via a data: module, jsr: specifier resolution, and a declared-but-absent optional dependency staying external).

Testing

• I have tested these changes locally
• I have added/updated tests as needed
• All tests pass (npm test)

Checklist

• My code follows the project's style guidelines
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation (if applicable)
• My changes generate no new warnings
• I have updated docs/ (AGENTS.md) if I made architectural changes

Additional Notes

• The move is deliberately verbatim. Shim sources, compile-time assets and moved tests are byte-identical to apper at b27ce1f6; library modules differ only by the .js import extensions this repo requires, the injectable tracer/log sink, the new src/index.ts, and four internal types that lost export to satisfy knip. Evidence recorded on the feature commit: 207/207 moved tests green on Node 20.20.2 and 24.16.0, and across six output modes the extracted engine's modules hash the same as apper's once esbuild's node_modules path-depth comments are normalized.
• Do not run Biome or a formatter over src/shim/, src/runtime/, src/private-data-sources/, runtime-context.ts or static-egress*.ts — their text is injected into user bundles, so whitespace changes what every compiled function hashes to. biome.json and knip.json exclude them for that reason; test/ is likewise held byte-identical until apper's copy is deleted.
• One test is a characterization rather than an assertion of intent: prepareApp's comment claims each function is sealed into its own fn_<index>/ keyspace, and it is not — the resolver only checks map membership, so ../fn_1/main.ts resolves. Same app, same Worker, and the manifest-store gate that guards credentials is separate and holds, so the behaviour is pinned here rather than changed.
• The package is not wired into the CLI in this PR, and apper's engine copy is untouched — per the extraction plan it goes only once the service consumes a released version of this package.
• Test verification comes from this branch's commit evidence and the new CI lane (bun run test in packages/functions-compiler; the repo uses bun, not npm test). The suite needs registry access and a few minutes because it compiles real npm packages and runs them in workerd.

---

_{🤖 Generated by Claude | 2026-09-14 06:47 UTC | b7a8af7}

---

How much of this is apper's code, measured

Diffed against infra/base44-userapp-bundler at b27ce1f6, with whitespace removed so Biome's reflowing does not inflate the count.

Byte-identical to apper	52 of 76 files — 7,018 lines, not one character changed
Differ only by formatting plus a few characters	14 files. The largest engine file, worker-entry.ts, differs by 52 characters out of 14,574
Genuinely new	4 files, plus 6 new files of tests and build scripts (693 lines)

0.059% of the moved code changed — 178 characters out of 302,381. All of it accounted for: .js on relative imports (3 characters each), two ??= moved out of a return, one let result: annotation, and export dropped from four internal types (6 characters each).

The four files that are not moved code:

File	Lines	What it is
src/tracing.ts	32	Rewritten — apper's imports dd-trace; this takes an injectable tracer, default no-op
src/log.ts	52	apper's writer kept verbatim as the default sink, plus ~20 lines of setLogSink
src/index.ts	35	New content under a shared name; apper's index.ts is the server bootstrap, which did not come across
test/helpers.ts	40	A subset — the two compile helpers, without the HTTP ones that stayed with the service

Character counts show the text was copied, not that the behaviour survived; a single character in the right place would be enough to break it. That is what the byte-parity evidence is for: across six compile modes the emitted modules hash identically to apper's, and 237 tests pass on Node 20.20.2 and 24.16.0.

To check it independently: diff -r packages/functions-compiler/src <apper>/infra/base44-userapp-bundler/src at b27ce1f6. The whole delta fits on a screen.

What is missing: shards, and whole-app CFW bundles

This package compiles one module per call. It does not decide which functions belong in which module, how large the result may be, or what to do when it is too large — so it cannot yet produce the deployable Cloudflare Workers bundles for a whole app. That work is still Python in apper's backend/app/cloudflare_functions/ and is the next PR.

Missing piece	Where it lives today	What it has to do
Source assembly	function_bundle.py — cfw_bundle_input, collect_reachable_backend_files	Turn a function's directory plus the shared files it reaches into the entry + files this package takes, keeping the flat single-file case and the refusal to escape into the frontend tree
Fresh shard planning	shard_planning.py — full_repartition, target_shard_count	Group an app's functions into shards deterministically, and refuse a set over the supplied product capacity
Size measurement	cloudflare_wfp_runtime.py — measure_bundle_bytes, judge_bundle_size	Raw UTF-8 bytes and level-6 gzip, against Cloudflare's 64 MiB uncompressed limit and our compressed cap
Split on overflow	cloudflare_wfp_runtime.py — _build_shard_with_split	Halve an oversized multi-function shard deterministically and recompile; fail the build when one function alone is too big
Whole-build validation	apper #23460	Reject a partial result: every declared function in exactly one successful shard, or no build at all

Only the fresh-build slice crosses over. Everything that remembers a previous deploy stays in apper: incremental shard reuse (it needs the previous deployment map), the per-app shard_size_override ratchet, entitlement and settings reads, provider upload, binding resolution and secret delivery. Policy numbers — shard size, shard count, the gzip cap, whether the gate is enforced — arrive as inputs; the package never reads them itself.

Two details from the Python that must survive the port, both checked against the current code:

• The single-shard path builds in caller order while the multi-shard path sorts by name. Function order changes the emitted bytes, so both branches carry over as they are — normalising them is a byte change dressed as a cleanup.
• Capacity is judged at the global shard size while packing may use a smaller ratcheted one, so a legal plan can hold more shards than max_shards. A final "shard count ≤ max_shards" assertion would lock out apps that deploy fine today.

Also out of scope for the new lane: Deno deployment targets, existing-Worker reuse, incremental deploy state, and actors — though the engine keeps its actor support for the legacy service that still uses it.

The same section is in the package README, which is where it stays current.

[github-actions]

🚀 Package Preview Available!

---

Install this PR's preview build with npm:

npm i @base44-preview/cli@0.1.14-pr.621.b7a8af7

Prefer not to change any import paths? Install using npm alias so your code still imports base44:

npm i "base44@npm:@base44-preview/cli@0.1.14-pr.621.b7a8af7"

Or add it to your package.json dependencies:

{
  "dependencies": {
    "base44": "npm:@base44-preview/cli@0.1.14-pr.621.b7a8af7"
  }
}

• 📦 Preview Package: @base44-preview/cli@0.1.14-pr.621.b7a8af7
• 🔗 View this commit on GitHub

---

_{Preview published to npm registry — try new features instantly!}