diff --git a/packages/functions-compiler/README.md b/packages/functions-compiler/README.md index f33c0a2f..47957b03 100644 --- a/packages/functions-compiler/README.md +++ b/packages/functions-compiler/README.md @@ -17,52 +17,91 @@ Two consumers share this one engine: Internal to Base44 — published **public** so apper's bundler service can install it, but it is not a supported public API: the CLI bundles it at build time so end users never install it, and it carries no compatibility promise to anyone -outside this repo. It compiles functions and nothing else: shard planning, size -splitting, artifact writing, version creation and deploy all live above it. +outside this repo. Compilation is the whole of its job, and that now includes +source assembly, shard planning, size measurement and splitting. Artifact +writing, version creation, upload and deploy live above it. The package carries no credentials and reads no configuration of its own. It names the environment variables and headers the generated worker will use at runtime (`BASE44_*`, `X-Base44-*`) but holds none of their values, and the code it ships is the same code already compiled into every deployed user worker. -## What is missing: shards, and whole-app CFW bundles +## Shards and whole-app builds -Today this package compiles **one module per call**. It does not decide which -functions belong in which module, how large the result may be, or what to do -when it is too large. That work is still Python in apper's -`backend/app/cloudflare_functions/` and moves here next — it is what turns -"compile this set of sources" into "produce the deployable Cloudflare Workers -bundles for this app". +This package turns an app's whole backend function set into deployable Cloudflare +Workers: it assembles each function's sources, groups them into shards, compiles +each shard, measures it against Cloudflare's ceilings, and halves any shard whose +module is over one. `compileFunctionShards` is the entry point, and unlike +`bundleApp` it refuses a partial result — a module missing a handler is a broken +app, not a smaller success. -| Missing piece | Where it lives today | What it has to do | +Each piece came from apper, and the provenance is worth keeping: + +| Piece | Where it came from | Here | |---|---|---| -| Source assembly | `function_bundle.py` — `cfw_bundle_input`, `collect_reachable_backend_files` | Turn a function's directory plus the shared files it reaches into the `entry` + `files` this package takes, keeping the flat single-file case and the refusal to escape into the frontend tree | -| Fresh shard planning | `shard_planning.py` — `full_repartition`, `target_shard_count` | Group an app's functions into shards deterministically, and refuse a set that exceeds the supplied product capacity | -| Size measurement | `cloudflare_wfp_runtime.py` — `measure_bundle_bytes`, `judge_bundle_size` | Raw UTF-8 bytes and level-6 gzip, against Cloudflare's 64 MiB uncompressed limit and our compressed cap | -| Split on overflow | `cloudflare_wfp_runtime.py` — `_build_shard_with_split` | Halve an oversized multi-function shard deterministically and recompile; fail the build when one function alone is too big | -| Whole-build validation | apper PR #23460 | Reject a partial result: every declared function in exactly one successful shard, or no build at all | - -Only the **fresh-build** slice comes across. Everything that remembers a -previous deploy stays in apper: incremental shard reuse (it needs the previous -deployment map), the per-app `shard_size_override` ratchet, entitlement and -settings reads, provider upload, binding resolution and secret delivery. Policy -numbers — shard size, shard count, the gzip cap, whether the gate is enforced — -arrive as inputs; the package never reads them itself. - -Two details from the Python that must survive the port, both verified against -the current code: - -- The single-shard path builds in caller order while the multi-shard path sorts - by name. Function order changes the emitted bytes, so both branches carry - over as they are — normalising them is a byte change dressed as a cleanup. -- Capacity is judged at the *global* shard size while packing may use a smaller - ratcheted one, so a legal plan can hold more shards than `max_shards`. A - final "shard count ≤ max_shards" assertion would lock out apps that deploy - fine today. - -Also deliberately out of scope for the new lane: Deno deployment targets, -existing-Worker reuse, incremental deploy state, and actors — though the engine -keeps its actor support for the legacy service that still uses it. +| Source assembly | `function_bundle.py` — `cfw_bundle_input`, `collect_reachable_backend_files` | `src/assembly.ts` | +| Fresh shard planning | `shard_planning.py` — `full_repartition`, `target_shard_count` | `src/shards/plan.ts` | +| Size measurement | `cloudflare_wfp_runtime.py` — `measure_bundle_bytes`, `judge_bundle_size` | `src/shards/size.ts` | +| Split on overflow | `cloudflare_wfp_runtime.py` — `_build_shard_with_split` | `src/shards/build.ts` | +| Whole-build validation | apper PR #23460 | `src/shards/build.ts` | + +Policy numbers — shard size, shard count, the compressed cap — arrive as inputs. +The package reads no settings and no feature flags, and the two wrapper flags +(`postResponseTelemetry`, `runtimeSecrets`) are handed to it, because both change +the emitted bytes and only the platform knows their value for an app. + +Deliberately not here: Deno deployment targets, existing-Worker reuse, provider +upload, binding resolution, secret delivery, and incremental shard reuse — every +version is built from scratch, so nothing here remembers a previous deploy. The +engine keeps its actor support for the legacy service that still uses it. + +### What a bundle says about itself + +The first line of a compiled shard is its own manifest: + +```js +//!b44:1 {"functions":["cleanupForgottenDepartures","health","sendReminder"],"telemetry":false,"runtimeSecrets":false,"compiler":"0.1.0"} +``` + +`//!b44:` is a fixed sentinel, so `head -1` on a script pulled from +Cloudflare answers "what is in this?" without executing or parsing anything, and +the payload is JSON so a tool parses it in one call. Only the app path emits it; +the legacy single-function `bundle()` stays bannerless, which keeps that lane +byte-comparable with the engine apper still runs. + +`functions` is sorted whatever order the shard was built in. `telemetry` and +`runtimeSecrets` are there because they change the emitted bytes and a deploy has +to pair its secrets delivery with them. `compiler` is this package's version. + +Nothing volatile may be added: a timestamp or build id would re-mint a version +for unchanged code, the app id would make the same functions compile differently +per app, and the shard's position would make two identical shards differ. + +### Reproducibility is a contract, not a nicety + +A version's identity is the hash of the **compiled artifacts**, never of the +sources. Anything that shifts the emitted bytes therefore mints a new version of +code that did not change, and all of these do: + +- the version of this package, and the depth of the `node_modules` it was built + against — esbuild writes each vendored chunk's relative path into the minified + output, so building one directory shallower changes every user Worker's bytes; +- the order the caller hands functions over in, for a single shard: that path + builds in caller order, while a multi-shard plan sorts by name. Both branches + are as apper has them, and `shard-build.e2e.test.ts` pins the difference; +- either wrapper flag. + +Two behaviours differ from apper's Python on purpose, and both are tested: + +- **An unparseable source.** apper reads each file on its own, so a file it + cannot parse contributes no edges while the rest of the set still assembles. + esbuild's walk is one build over the whole graph, so anything unparseable in it + rejects — the fallback is the flat single-file submission, and the compiler then + reports the error itself. +- **Compressed size.** Node's gzip reads about 0.7% heavier than Python's on + identical input: 43,057 bytes against 42,765 on a real 122,324-byte module. The + direction is the safe one, since this lane refuses slightly earlier than the + service would. ## Using it diff --git a/packages/functions-compiler/src/assembly.ts b/packages/functions-compiler/src/assembly.ts new file mode 100644 index 00000000..9c67e8ba --- /dev/null +++ b/packages/functions-compiler/src/assembly.ts @@ -0,0 +1,201 @@ +/** + * Turn a backend function into the `{entry, files}` the compiler takes. + * + * A function may import a helper beside it or a module shared across functions + * (`../../shared/x.ts`). The compiler resolves relative imports by exact match + * inside `files`, so anything the function reaches has to be submitted with it — + * otherwise the import has no target and the build fails with "can't reach + * outside the function". + * + * Ported from apper's `backend/app/cloudflare_functions/function_bundle.py`. + * That walks the import graph with a tree-sitter parse, as a stand-in for what + * the bundler would see; here esbuild does the walk, so it is not a stand-in — + * the file set is what the compiler itself reaches. Verified against the Python + * fixtures in assembly.test.ts. + */ + +import path from "node:path"; +import { build } from "esbuild"; + +const NAMESPACE = "base44-reachability"; + +/** The flat shape a single-file function has always been submitted as. Keeping + * it means those functions compile to the same bytes they do today. */ +const FLAT_ENTRY = "main.ts"; + +export interface BundleInput { + entry: string; + files: Record; +} + +function loaderFor(filePath: string) { + if (/\.(ts|mts|cts)$/.test(filePath)) return "ts" as const; + if (filePath.endsWith(".tsx")) return "tsx" as const; + if (filePath.endsWith(".jsx")) return "jsx" as const; + if (filePath.endsWith(".json")) return "json" as const; + return "js" as const; +} + +function resolveRelative( + importer: string, + spec: string, + files: Record, +): string | null { + const dir = importer.includes("/") + ? importer.slice(0, importer.lastIndexOf("/")) + : ""; + const resolved = path.posix.normalize(path.posix.join(dir, spec)); + return resolved in files ? resolved : null; +} + +/** esbuild's own account of what the entry reaches: the metafile inputs, keyed + * by the paths the plugin resolved. Its own function so a failure can be + * handled by the caller. */ +async function walkInputs( + entryPath: string, + backendFiles: Record, +): Promise> { + const result = await build({ + entryPoints: [entryPath], + bundle: true, + write: false, + metafile: true, + logLevel: "silent", + // Only the metafile is read, never the output, so neither of these can + // change what the compiler emits. They decide what the walk ACCEPTS: the + // default iife format rejects top-level await, which a Deno function may + // legitimately use, and the walk would then reject a file the compiler + // itself compiles fine. + format: "esm", + platform: "neutral", + // Every path here is a project path served from memory and nothing resolves + // to the filesystem, so this only has to be a valid absolute path — and + // `path.sep` is not one on Windows, where esbuild refuses "\\". + absWorkingDir: process.cwd(), + // TypeScript drops an import whose bindings go unused, which would hide a + // file the function really does pull in. Keep every import as written. + tsconfigRaw: { compilerOptions: { verbatimModuleSyntax: true } }, + plugins: [ + { + name: "base44-reachability", + setup(pluginBuild) { + pluginBuild.onResolve({ filter: /.*/ }, (args) => { + if (args.kind === "entry-point") { + return { path: entryPath, namespace: NAMESPACE }; + } + if (args.path.startsWith("./") || args.path.startsWith("../")) { + const target = resolveRelative( + args.importer, + args.path, + backendFiles, + ); + if (target) return { path: target, namespace: NAMESPACE }; + } + return { path: args.path, external: true }; + }); + + pluginBuild.onLoad( + { filter: /.*/, namespace: NAMESPACE }, + (args) => ({ + contents: backendFiles[args.path], + loader: loaderFor(args.path), + }), + ); + }, + }, + ], + }); + return result.metafile.inputs; +} + +/** Did this build fail on the user's code, rather than on how the walk was + * set up? Every message esbuild raises about a source it loaded carries a + * location in this plugin's namespace; a configuration fault carries no + * location at all (verified against both shapes). */ +function isUserSourceFailure(error: unknown): boolean { + const errors = ( + error as { errors?: { location?: { file?: string } | null }[] } + ).errors; + return ( + Array.isArray(errors) && + errors.length > 0 && + errors.every((e) => e.location?.file?.startsWith(`${NAMESPACE}:`) === true) + ); +} + +/** + * Every backend file reachable from `entryPath` through `./` and `../` imports, + * including the entry. A specifier with no target in `backendFiles` — a typo, or + * a frontend file the caller deliberately excluded — is left out, so it stays + * forbidden by the compiler rather than being resolved here. `npm:`, `jsr:`, + * `node:` and bare specifiers are not edges into the project and are ignored. + * + * A source the walk cannot parse yields the entry alone, so the function + * compiles as the flat single-file submission and the compiler reports the + * error itself. + */ +export async function collectReachableFiles( + entryPath: string, + backendFiles: Record, +): Promise> { + if (!(entryPath in backendFiles)) { + throw new Error(`entry "${entryPath}" is not among the backend files`); + } + + let inputs: Record; + try { + inputs = await walkInputs(entryPath, backendFiles); + } catch (e) { + // apper's walk cannot fail this way: it reads each file on its own, so a + // file it cannot parse contributes no edges and the rest of the set still + // assembles. esbuild's walk is all-or-nothing — one unparseable source + // anywhere in the graph rejects here. Falling back to the entry alone keeps + // the flat submission a single-file function has always had, and the compile + // then fails with the compiler's own diagnostic, which the builder agent can + // act on. An exception out of assembly has nowhere to be reported at all. + // + // Only for the user's own sources, though. A fault in how this walk is + // configured also arrives here, and degrading it to a flat submission + // reports OUR bug as the user's unresolved import — which is how the format + // and working-directory defects above stayed invisible. + if (!isUserSourceFailure(e)) throw e; + return { [entryPath]: backendFiles[entryPath] }; + } + + const reached: Record = {}; + for (const input of Object.keys(inputs)) { + const filePath = input.startsWith(`${NAMESPACE}:`) + ? input.slice(NAMESPACE.length + 1) + : input; + if (filePath in backendFiles) reached[filePath] = backendFiles[filePath]; + } + return reached; +} + +/** + * The `{entry, files}` to compile for one function. + * + * A function that reaches nothing beyond its own entry keeps the flat + * `main.ts` submission, byte-identical to how it compiles today. One that + * reaches a helper or a shared module gets its real project path as the entry, + * so `../../shared/x.ts` resolves against the files beside it. + * + * `backendFiles` must hold the backend tree only. A frontend path left in it + * would resolve, and a function would silently bundle frontend code. + */ +export async function cfwBundleInput( + entryPath: string, + entryContent: string, + backendFiles: Record, +): Promise { + const files = await collectReachableFiles(entryPath, { + ...backendFiles, + [entryPath]: entryContent, + }); + const reachedOnlyTheEntry = + Object.keys(files).length === 1 && entryPath in files; + if (reachedOnlyTheEntry) { + return { entry: FLAT_ENTRY, files: { [FLAT_ENTRY]: entryContent } }; + } + return { entry: entryPath, files }; +} diff --git a/packages/functions-compiler/src/deno-bundle.ts b/packages/functions-compiler/src/deno-bundle.ts index 9d1b9f53..c6d81d45 100644 --- a/packages/functions-compiler/src/deno-bundle.ts +++ b/packages/functions-compiler/src/deno-bundle.ts @@ -73,6 +73,9 @@ export async function bundleToModule( // (npm/Emscripten glue), leaving loader-bound CJS locals intact. define: { __dirname: '"/"', __filename: '"/index.js"' }, minify: true, + // Prepended verbatim and not minified away, which is the point: it is + // the one place a compiled module says what is inside it. + ...(prepared.banner ? { banner: { js: prepared.banner } } : {}), sourcemap: false, // Errors are surfaced structurally (return value / BuildFailure); keep // esbuild from dumping diagnostics to the service's stderr. diff --git a/packages/functions-compiler/src/index.ts b/packages/functions-compiler/src/index.ts index 62a51536..6c450ba1 100644 --- a/packages/functions-compiler/src/index.ts +++ b/packages/functions-compiler/src/index.ts @@ -2,6 +2,8 @@ // src/ is either an internal engine module or a compile-time asset read as text // by the esbuild plugins. +export type { BundleInput } from "./assembly.js"; +export { cfwBundleInput, collectReachableFiles } from "./assembly.js"; export type { AppErrorClassification, AppFunctionStatus, @@ -30,6 +32,28 @@ export { DenoCompatError } from "./errors.js"; export { createGuardedFetch, installFetchGuard } from "./fetch-guard.js"; export type { Field, Level, LogSink } from "./log.js"; export { setLogSink } from "./log.js"; +export type { + CompiledShard, + ShardBuildFailure, + ShardBuildResult, +} from "./shards/build.js"; +export { compileFunctionShards } from "./shards/build.js"; +export type { ShardPolicy } from "./shards/plan.js"; +export { + assertWithinCapacity, + planFreshShards, + ShardCapacityError, + targetShardCount, +} from "./shards/plan.js"; +export type { BundleSize, SizeVerdict } from "./shards/size.js"; +export { + BUNDLE_GZIP_LEVEL, + judgeBundleSize, + measureBundleBytes, + WORKER_RAW_SIZE_CEILING_BYTES, + workerGzipCapBreach, + workerRawSizeBreach, +} from "./shards/size.js"; export { STATIC_EGRESS_ARTIFACT_MARKER } from "./static-egress-marker.js"; export type { CompilerTracer } from "./tracing.js"; export { setCompilerTracer } from "./tracing.js"; diff --git a/packages/functions-compiler/src/shards/build.ts b/packages/functions-compiler/src/shards/build.ts new file mode 100644 index 00000000..71160401 --- /dev/null +++ b/packages/functions-compiler/src/shards/build.ts @@ -0,0 +1,228 @@ +/** + * Compile an app's functions into the final Cloudflare Workers shards. + * + * The loop is: plan the partition, compile each shard, measure it, and halve any + * shard whose module is over a ceiling before anything is uploaded. Ported from + * `_build_shard_with_split` in apper's `cloudflare_wfp_runtime.py`. + * + * That function also splits on two exceptions, and neither is a trigger here. + * The bundler's 413 is its HTTP request-body cap — a property of the shard's + * packed SOURCE going over the wire, not of the module coming out, and there is + * no request to reject locally. Cloudflare's 10027 is a real Worker limit, but + * it is the same 64 MiB `workerRawSizeBreach` already computes from the bytes in + * hand, so reaching it from a rejected upload is a failsafe, not the design. + * Measuring is the design, and it happens here. + * + * Unlike the compiler's own `bundleApp`, this refuses a partial result. A module + * missing a handler is not a smaller success, it is a broken app. + */ + +import { bundleApp } from "../bundler.js"; +import type { AppFunctionInput } from "../contracts.js"; +import type { BundleErrorItem } from "../errors.js"; +import { planFreshShards, type ShardPolicy } from "./plan.js"; +import { judgeBundleSize, type SizeVerdict } from "./size.js"; + +/** Shards compile independently; bound how many esbuild runs are in flight so + * peak memory does not scale with the app's function count. */ +const MAX_PARALLEL_SHARDS = 4; + +export interface CompiledShard { + /** Position in the emitted list. Not a deployment identity — the caller owns + * placement, and a split makes more shards than the plan had. */ + index: number; + /** Function names in this shard, in the order they were compiled. */ + functions: string[]; + module: string; + mainModule: string; + rawBytes: number; + gzipBytes: number; +} + +export interface ShardBuildFailure { + /** The function this is about, when it belongs to one. */ + function?: string; + message: string; + errors?: BundleErrorItem[]; +} + +export type ShardBuildResult = + | { ok: true; shards: CompiledShard[] } + | { ok: false; failures: ShardBuildFailure[] }; + +/** + * Compile every function into shards, or fail the whole build. + * + * Succeeds only when each declared function appears exactly once across the + * shards and every one of them compiled. + */ +export async function compileFunctionShards( + functions: AppFunctionInput[], + policy: ShardPolicy, + options: { postResponseTelemetry?: boolean; runtimeSecrets?: boolean } = {}, +): Promise { + const duplicate = firstDuplicate(functions.map((fn) => fn.name)); + if (duplicate) { + return fail([{ message: `Duplicate function name "${duplicate}".` }]); + } + + const byName = new Map(functions.map((fn) => [fn.name, fn])); + const plan = planFreshShards( + functions.map((fn) => fn.name), + policy, + ); + + const groups = plan.map((names) => names.map((name) => byName.get(name)!)); + const outcomes = await mapWithConcurrency( + groups, + MAX_PARALLEL_SHARDS, + (group) => buildWithSplit(group, policy, options), + ); + + const failures = outcomes.flatMap((o) => (o.ok ? [] : o.failures)); + if (failures.length > 0) return fail(failures); + + const shards = outcomes + .flatMap((o) => (o.ok ? o.shards : [])) + .map((shard, index) => ({ ...shard, index })); + + const missing = assertEveryFunctionPlacedOnce(functions, shards); + if (missing.length > 0) return fail(missing); + + return { ok: true, shards }; +} + +type GroupOutcome = + | { ok: true; shards: Omit[] } + | { ok: false; failures: ShardBuildFailure[] }; + +/** Compile one planned group, halving it when its module is over a ceiling. The + * verdict comes from the bytes, so nothing is uploaded to discover it. */ +async function buildWithSplit( + group: AppFunctionInput[], + policy: ShardPolicy, + options: { postResponseTelemetry?: boolean; runtimeSecrets?: boolean }, +): Promise { + const response = await bundleApp({ functions: group, ...options }); + + const compileFailures = response.functions + .filter((fn) => !fn.ok) + .map((fn) => ({ + function: fn.name, + message: `Function "${fn.name}" failed to compile.`, + errors: fn.ok ? undefined : fn.errors, + })); + // A partial module is what the service ships and a whole-app build must not: + // the peers compiled, but the app is missing a handler. + if (compileFailures.length > 0) + return { ok: false, failures: compileFailures }; + if (!response.ok) { + return fail([{ message: "The combined build produced no module." }]); + } + + const verdict = await judgeBundleSize(response.module, policy.gzipCapBytes); + if (!verdict.breach) { + return { + ok: true, + shards: [shardOf(group, response.module, response.main_module, verdict)], + }; + } + + if (group.length === 1) { + // Splitting is exhausted: one function's module alone is over the ceiling. + return { + ok: false, + failures: [{ function: group[0].name, message: verdict.breach }], + }; + } + + const mid = Math.floor(group.length / 2); + const halves = await Promise.all([ + buildWithSplit(group.slice(0, mid), policy, options), + buildWithSplit(group.slice(mid), policy, options), + ]); + const halfFailures = halves.flatMap((h) => (h.ok ? [] : h.failures)); + if (halfFailures.length > 0) return { ok: false, failures: halfFailures }; + return { ok: true, shards: halves.flatMap((h) => (h.ok ? h.shards : [])) }; +} + +function shardOf( + group: AppFunctionInput[], + module: string, + mainModule: string, + verdict: SizeVerdict, +): Omit { + return { + functions: group.map((fn) => fn.name), + module, + mainModule, + rawBytes: verdict.rawBytes, + gzipBytes: verdict.gzipBytes, + }; +} + +/** Every declared function must land in exactly one emitted shard. A name that + * vanished, or turned up twice after a split, would deploy a broken app. */ +function assertEveryFunctionPlacedOnce( + functions: AppFunctionInput[], + shards: CompiledShard[], +): ShardBuildFailure[] { + const placed = new Map(); + for (const shard of shards) { + for (const name of shard.functions) { + placed.set(name, (placed.get(name) ?? 0) + 1); + } + } + const failures: ShardBuildFailure[] = []; + for (const fn of functions) { + const count = placed.get(fn.name) ?? 0; + if (count !== 1) { + failures.push({ + function: fn.name, + message: `Function "${fn.name}" appears in ${count} shards; expected exactly 1.`, + }); + } + } + for (const name of placed.keys()) { + if (!functions.some((fn) => fn.name === name)) { + failures.push({ + function: name, + message: `Shard holds unknown function "${name}".`, + }); + } + } + return failures; +} + +function firstDuplicate(names: string[]): string | null { + const seen = new Set(); + for (const name of names) { + if (seen.has(name)) return name; + seen.add(name); + } + return null; +} + +function fail(failures: ShardBuildFailure[]): ShardBuildResult { + return { ok: false, failures }; +} + +/** Bounded concurrency, input order preserved. */ +async function mapWithConcurrency( + items: T[], + limit: number, + fn: (item: T) => Promise, +): Promise { + const results = new Array(items.length); + let cursor = 0; + const worker = async (): Promise => { + while (cursor < items.length) { + const index = cursor++; + results[index] = await fn(items[index]); + } + }; + await Promise.all( + Array.from({ length: Math.min(limit, items.length) }, () => worker()), + ); + return results; +} diff --git a/packages/functions-compiler/src/shards/plan.ts b/packages/functions-compiler/src/shards/plan.ts new file mode 100644 index 00000000..93c4b761 --- /dev/null +++ b/packages/functions-compiler/src/shards/plan.ts @@ -0,0 +1,119 @@ +/** + * Which functions go in which shard, for a build with no previous deploy to + * reuse. Ported from apper's `shard_planning.py` (`target_shard_count`, + * `full_repartition`) and the capacity refusal in `deploy_app`. + * + * Only the fresh path crosses over. Incremental reuse needs the previous + * deployment map, and the per-app shard-size ratchet is state written across + * deploys; both stay with the service. + */ + +/** Everything the planner is allowed to know. It reads no settings and no + * entitlements — the caller resolves these and passes them in. */ +export interface ShardPolicy { + /** Functions packed into one shard. May be lower than `globalShardSize` when + * the caller carries a ratcheted override. */ + shardSize: number; + /** The size capacity is judged at. A lowered `shardSize` changes packing only + * and must never lock an app out of deploying. */ + globalShardSize: number; + maxShards: number; + /** Compressed ceiling for one shard's module, in bytes. */ + gzipCapBytes: number; +} + +export class ShardCapacityError extends Error { + constructor(message: string) { + super(message); + this.name = "ShardCapacityError"; + } +} + +/** The three counts have to be whole numbers of at least one. A zero or + * negative `shardSize` made the chunking loop below never advance — it hung + * rather than refusing, where the Python raises on the same input. */ +function assertUsablePolicy(policy: ShardPolicy): void { + for (const field of ["shardSize", "globalShardSize", "maxShards"] as const) { + const value = policy[field]; + if (!Number.isInteger(value) || value < 1) { + throw new Error( + `ShardPolicy.${field} must be an integer of at least 1, got ${value}`, + ); + } + } +} + +function ceilDiv(a: number, b: number): number { + return Math.ceil(a / b); +} + +/** + * How many shards the desired set needs. `maxShards === 1` — sharding off — is + * always one shard whatever the function count: the whole app bundles into a + * single Worker, which is the legacy per-app behaviour. Going through the + * division would trip the capacity check for a flag-off app. + */ +export function targetShardCount( + functionCount: number, + shardSize: number, + maxShards: number, +): number { + if (maxShards === 1) return 1; + return ceilDiv(functionCount, shardSize); +} + +/** + * Refuse a set the product cannot hold. Judged at the GLOBAL shard size: a + * ratcheted-down `shardSize` affects packing only, so judging at it would lock + * a recovered app out of deploys. The resulting plan may therefore hold more + * shards than `maxShards` — bounded by the function count, which is no more + * Workers than the legacy per-function topology used. Do not "tidy" this into a + * check on the final shard count. + */ +export function assertWithinCapacity( + functionCount: number, + policy: ShardPolicy, +): void { + const needed = targetShardCount( + functionCount, + policy.globalShardSize, + policy.maxShards, + ); + if (needed > policy.maxShards) { + throw new ShardCapacityError( + `App has ${functionCount} functions — over the per-app Worker capacity of ` + + `${policy.globalShardSize * policy.maxShards} ` + + `(${policy.maxShards} shards × ${policy.globalShardSize}).`, + ); + } +} + +/** + * The fresh partition, as a list of shards holding function names. + * + * A single planned shard keeps the caller's order; a multi-shard plan sorts by + * name before chunking. The two differ in apper and the difference is load + * bearing — function order inside a combined module changes the emitted bytes, + * so normalising them would be a byte change wearing a cleanup's clothes. + */ +export function planFreshShards( + names: string[], + policy: ShardPolicy, +): string[][] { + assertUsablePolicy(policy); + assertWithinCapacity(names.length, policy); + + const target = targetShardCount( + names.length, + policy.shardSize, + policy.maxShards, + ); + if (target <= 1) return [[...names]]; + + const sorted = [...names].sort(); + const shards: string[][] = []; + for (let i = 0; i < sorted.length; i += policy.shardSize) { + shards.push(sorted.slice(i, i + policy.shardSize)); + } + return shards; +} diff --git a/packages/functions-compiler/src/shards/size.ts b/packages/functions-compiler/src/shards/size.ts new file mode 100644 index 00000000..29c9f841 --- /dev/null +++ b/packages/functions-compiler/src/shards/size.ts @@ -0,0 +1,92 @@ +/** + * How large a compiled Worker module is, and whether Cloudflare will take it. + * + * Ported from apper's `cloudflare_wfp_runtime.py` (`measure_bundle_bytes`, + * `worker_raw_size_breach`, `worker_gzip_cap_breach`, `judge_bundle_size`). + * Deciding this before an upload is the whole point: the one real production + * rejection spent 30 seconds uploading 97 MB to be told the same thing this + * computes for free. + */ + +import { promisify } from "node:util"; +import { gzip } from "node:zlib"; + +const gzipAsync = promisify(gzip); + +/** zlib's default, and the level `wrangler --dry-run` reports — so the number + * here means the same thing as the one in Cloudflare's docs, in a support + * thread and on a developer's terminal. It is not a claim about Cloudflare's + * own server-side compressor, which is unpublished; the safety margin lives in + * the cap instead. */ +export const BUNDLE_GZIP_LEVEL = 6; + +/** Cloudflare's uncompressed per-Worker ceiling, in the units Cloudflare states + * it in. Taken from a real 10027 in production, which reads verbatim: "Your + * Worker exceeded the uncompressed size limit of 64 MiB." Decimal 64_000_000 + * would refuse 3 MiB early for no reason. */ +export const WORKER_RAW_SIZE_CEILING_BYTES = 64 * 1024 * 1024; + +export interface BundleSize { + rawBytes: number; + gzipBytes: number; +} + +export interface SizeVerdict extends BundleSize { + /** Why the module is over a ceiling, or `null` when it is within both. */ + breach: string | null; +} + +/** `(raw, gzip)` for a compiled module. Compression runs off the event loop: + * a multi-megabyte gzip is long enough to matter when several shards are in + * flight. */ +export async function measureBundleBytes(module: string): Promise { + const raw = Buffer.from(module, "utf8"); + const compressed = await gzipAsync(raw, { level: BUNDLE_GZIP_LEVEL }); + return { rawBytes: raw.byteLength, gzipBytes: compressed.byteLength }; +} + +/** Why Cloudflare will reject this module outright, or `null`. The only verdict + * taken on a number Cloudflare states itself — no estimate, no compression, no + * headroom to argue about. */ +export function workerRawSizeBreach(rawBytes: number): string | null { + if (rawBytes > WORKER_RAW_SIZE_CEILING_BYTES) { + return ( + `bundled module is ${rawBytes} bytes uncompressed, over Cloudflare's ` + + `${WORKER_RAW_SIZE_CEILING_BYTES}-byte (64 MiB) per-Worker ceiling` + ); + } + return null; +} + +/** Why the module is over the compressed ceiling, or `null`. Our side of the + * comparison is the estimate, not Cloudflare's number: it compresses + * server-side with an unpublished algorithm, so level-6 gzip is a proxy for + * the figure it measures. The cap carries the margin. */ +export function workerGzipCapBreach( + gzipBytes: number, + gzipCapBytes: number, +): string | null { + if (gzipBytes > gzipCapBytes) { + return ( + `bundled module is ${gzipBytes} bytes gzipped (level ${BUNDLE_GZIP_LEVEL}), ` + + `over the ${gzipCapBytes}-byte cap for a Cloudflare Worker script` + ); + } + return null; +} + +/** Measure a module and say whether it breaches either ceiling. Never throws: + * what a breach costs depends on whether the caller can re-partition, and only + * the caller knows that. */ +export async function judgeBundleSize( + module: string, + gzipCapBytes: number, +): Promise { + const { rawBytes, gzipBytes } = await measureBundleBytes(module); + // Uncompressed first: it is the exact one, so when both are over it is the + // verdict worth reporting. + const breach = + workerRawSizeBreach(rawBytes) ?? + workerGzipCapBreach(gzipBytes, gzipCapBytes); + return { rawBytes, gzipBytes, breach }; +} diff --git a/packages/functions-compiler/src/version.ts b/packages/functions-compiler/src/version.ts new file mode 100644 index 00000000..ccb22ddb --- /dev/null +++ b/packages/functions-compiler/src/version.ts @@ -0,0 +1,15 @@ +/** + * This package's version, as a literal. + * + * It goes into every compiled shard's banner, which means it is part of the + * emitted bytes and therefore part of a deployed version's identity. Reading it + * from `package.json` at runtime cannot work: a host that bundles this module + * into an artifact of its own — which the CLI does — ships no `package.json` + * beside it, and the read silently degraded to "unknown", so the same functions + * compiled to different bytes in the CLI and in the service. + * + * A literal resolves in every host. `version.test.ts` fails the build if it + * drifts from `package.json`, so bumping the package still means editing two + * files but cannot mean forgetting one. + */ +export const COMPILER_VERSION = "0.1.0"; diff --git a/packages/functions-compiler/src/worker-entry.ts b/packages/functions-compiler/src/worker-entry.ts index 857a4442..98af789f 100644 --- a/packages/functions-compiler/src/worker-entry.ts +++ b/packages/functions-compiler/src/worker-entry.ts @@ -10,6 +10,7 @@ import type { AppFunctionInput } from "./contracts.js"; import { DenoCompatError } from "./errors.js"; import { RUNTIME_CONTEXT_SPECIFIER } from "./esbuild/runtime-context-virtual.js"; import { TELEMETRY_PATCH, TELEMETRY_STORE_FIELDS } from "./telemetry.js"; +import { COMPILER_VERSION } from "./version.js"; // Pre-built by scripts/build-shim.ts; regenerate it after changing the shim. // Read LAZILY, not at module load: build-shim.ts transitively imports this @@ -44,6 +45,8 @@ const ACTOR_ENTRY_FILENAME = "__base44_actor_entry.mjs"; export interface PreparedWorker { entry: string; files: Record; + /** Prepended verbatim to the compiled module. Only the app path sets it. */ + banner?: string; } export function workerRuntimeFiles(): Record { @@ -133,6 +136,33 @@ export async function prepareFunction( /** One app function paired with the stable key its files and diagnostics are * namespaced under (`fn_`). The index is the function's original * position so attribution stays correct across an exclude-and-rebuild. */ +/** Bumped only when the payload's shape changes, never for a new field. */ +const BANNER_FORMAT = 1; + +/** The bundle's self-description, as its first line: `//!b44: `. + * A fixed sentinel so `head -1` finds it and a reader can version the format, + * and JSON so it parses in one call. Before this, a compiled module named its + * functions only as scattered `registerLazy` literals in minified output. + * + * Nothing volatile belongs in here. A version's identity is the hash of these + * bytes, so a timestamp, a build id or anything else that moves on its own + * would re-mint a version for code that did not change; the app id would make + * the same functions compile differently per app; the shard's position would + * make two identical shards differ. Names are sorted for the same reason — the + * module below is assembled in caller order, this line is not. */ +function buildBanner( + entries: AppFunctionEntry[], + telemetry: boolean, + runtimeSecrets: boolean, +): string { + return `//!b44:${BANNER_FORMAT} ${JSON.stringify({ + functions: entries.map(({ fn }) => fn.name).sort(), + telemetry, + runtimeSecrets, + compiler: COMPILER_VERSION, + })}`; +} + export interface AppFunctionEntry { index: number; fn: AppFunctionInput; @@ -172,7 +202,11 @@ export function prepareApp( postResponseTelemetry, runtimeSecrets, ); - return { entry: ENTRY_FILENAME, files }; + return { + entry: ENTRY_FILENAME, + files, + banner: buildBanner(entries, postResponseTelemetry, runtimeSecrets), + }; } // Activation prelude for runtime-secrets bundles: gate on the encrypted diff --git a/packages/functions-compiler/test/assembly.test.ts b/packages/functions-compiler/test/assembly.test.ts new file mode 100644 index 00000000..a49f6cd9 --- /dev/null +++ b/packages/functions-compiler/test/assembly.test.ts @@ -0,0 +1,237 @@ +/** + * Parity fixtures for the source assembly, ported one-for-one from apper's + * backend/tests/unit/cloudflare_functions/test_function_bundle.py. + * + * The Python walks the import graph with tree-sitter and these cases pin what it + * collects. The port asks esbuild instead, so matching them is the evidence that + * swapping the parser did not change which files a function is built with. + */ + +import { describe, expect, it } from "vitest"; +import { cfwBundleInput, collectReachableFiles } from "../src/assembly"; +import { bundle } from "../src/bundler"; + +const ENTRY = "base44/functions/crossshared/entry.ts"; + +const reached = async (entry: string, files: Record) => + Object.keys(await collectReachableFiles(entry, files)).sort(); + +describe("collectReachableFiles", () => { + it("collects a shared module reached across functions", async () => { + expect( + await reached(ENTRY, { + [ENTRY]: + 'import { greet } from "../../shared/greeting.ts";\nDeno.serve(() => new Response(greet("x")));', + "base44/shared/greeting.ts": + "export const greet = (n: string) => `hi ${n}`;", + // Another function's subtree must not be pulled in. + "base44/functions/other/entry.ts": + 'import { z } from "../../shared/other.ts";', + "base44/shared/other.ts": "export const z = 1;", + }), + ).toEqual([ENTRY, "base44/shared/greeting.ts"]); + }); + + it("collects a helper inside the function's own directory", async () => { + const entry = "base44/functions/withinshared/entry.ts"; + expect( + await reached(entry, { + [entry]: + 'import { greet } from "./greeting.ts";\nconsole.log(greet());', + "base44/functions/withinshared/greeting.ts": + "export const greet = () => 1;", + }), + ).toEqual([entry, "base44/functions/withinshared/greeting.ts"]); + }); + + it("follows shared imports transitively and leaves unreached files out", async () => { + expect( + await reached(ENTRY, { + [ENTRY]: 'import { a } from "../../shared/a.ts";', + "base44/shared/a.ts": + 'import { b } from "./b.ts";\nexport const a = b;', + "base44/shared/b.ts": "export const b = 1;", + "base44/shared/unused.ts": "export const u = 2;", + }), + ).toEqual([ENTRY, "base44/shared/a.ts", "base44/shared/b.ts"]); + }); + + it("does not collect an escape into the frontend tree", async () => { + // `../../../src/lib/x.ts` resolves to `src/lib/x.ts`, which the caller keeps + // out of the backend set — so it is never collected and stays forbidden. + const files = await reached(ENTRY, { + [ENTRY]: + 'import { x } from "../../../src/lib/x.ts";\nimport { greet } from "../../shared/greeting.ts";', + "base44/shared/greeting.ts": "export const greet = () => 1;", + }); + expect(files).toEqual([ENTRY, "base44/shared/greeting.ts"]); + expect(files.some((f) => f.startsWith("src/"))).toBe(false); + }); + + it("leaves a relative import with no target unresolved", async () => { + expect( + await reached(ENTRY, { + [ENTRY]: 'import { greet } from "../../shared/missing.ts";', + }), + ).toEqual([ENTRY]); + }); + + it("ignores non-relative specifiers", async () => { + expect( + await reached(ENTRY, { + [ENTRY]: + 'import { createClientFromRequest } from "npm:@base44/sdk";\n' + + 'import { base44 } from "@/api/base44Client";\n' + + 'import { greet } from "../../shared/greeting.ts";', + "base44/shared/greeting.ts": "export const greet = () => 1;", + }), + ).toEqual([ENTRY, "base44/shared/greeting.ts"]); + }); + + it("keeps an import whose bindings are never used", async () => { + // TypeScript elides an unused import as a presumed type. That would drop a + // file the function genuinely ships, so the walk must not inherit it. + expect( + await reached(ENTRY, { + [ENTRY]: + 'import { unusedButPresent } from "../../shared/side-effect.ts";', + "base44/shared/side-effect.ts": "export const unusedButPresent = 1;", + }), + ).toEqual([ENTRY, "base44/shared/side-effect.ts"]); + }); +}); + +describe("cfwBundleInput", () => { + it("stays flat for a single-file function", async () => { + // No relative imports: byte-identical to how it is submitted today, so the + // function's compiled output does not move. + const content = + 'import Stripe from "npm:stripe";\nDeno.serve(() => new Response("x"));'; + expect(await cfwBundleInput(ENTRY, content, {})).toEqual({ + entry: "main.ts", + files: { "main.ts": content }, + }); + }); + + it("expands to real paths once a shared module is reached", async () => { + const content = + 'import { greet } from "../../shared/greeting.ts";\nDeno.serve(() => new Response(greet("x")));'; + const input = await cfwBundleInput(ENTRY, content, { + "base44/shared/greeting.ts": "export const greet = (n: string) => n;", + }); + expect(input.entry).toBe(ENTRY); + expect(Object.keys(input.files).sort()).toEqual([ + ENTRY, + "base44/shared/greeting.ts", + ]); + }); + + it("stays flat when the escape target is absent, keeping the specifier intact", async () => { + const content = + 'import { x } from "../../../src/lib/x.ts";\nDeno.serve(() => new Response(x));'; + const input = await cfwBundleInput(ENTRY, content, {}); + expect(input).toEqual({ entry: "main.ts", files: { "main.ts": content } }); + // Preserved verbatim, so the compiler refuses it rather than the deploy + // quietly succeeding against frontend code. + expect(input.files["main.ts"]).toContain("../../../src/lib/x.ts"); + }); + + it("stays flat when a relative target does not exist", async () => { + const content = + 'import { greet } from "../../shared/nope.ts";\nDeno.serve(() => new Response(greet("x")));'; + const input = await cfwBundleInput(ENTRY, content, { + "base44/shared/greeting.ts": "export const greet = (n: string) => n;", + }); + expect(input).toEqual({ entry: "main.ts", files: { "main.ts": content } }); + expect(input.files["main.ts"]).toContain("../../shared/nope.ts"); + }); +}); + +describe("the assembled input compiles", () => { + it("hands the compiler a set whose shared import resolves", async () => { + // The assembly is only correct if its output is buildable — the Python + // fixtures can assert the file set but never this. + const input = await cfwBundleInput( + ENTRY, + 'import { greet } from "../../shared/greeting.ts";\nDeno.serve(() => new Response(greet("x")));', + { + "base44/shared/greeting.ts": + "export const greet = (n: string) => `hi ${n}`;", + }, + ); + const result = await bundle(input); + expect(result.ok).toBe(true); + }); + + it("hands the compiler a flat set that still fails on an escape", async () => { + const input = await cfwBundleInput( + ENTRY, + 'import { x } from "../../../src/lib/x.ts";\nDeno.serve(() => new Response(x));', + {}, + ); + const result = await bundle(input); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.errors.map((e) => e.message).join("\n")).toContain( + "bundled with this function", + ); + }); +}); + +describe("a source the walk cannot parse", () => { + // apper cannot reach this state: its walk reads each file on its own, so a + // file it cannot parse contributes no edges while the rest of the set still + // assembles. esbuild's walk is one build over the whole graph, so anything + // unparseable in it rejects. These two pin what happens instead of an + // exception leaving assembly with nowhere to be reported. + const BROKEN = "export const oops = ("; + + it("falls back to the entry alone, so the compiler reports the error", async () => { + const input = await cfwBundleInput(ENTRY, BROKEN, { + "base44/shared/greeting.ts": "export const greet = () => 1;", + }); + expect(input).toEqual({ entry: "main.ts", files: { "main.ts": BROKEN } }); + + const result = await bundle(input); + expect(result.ok).toBe(false); + }); + + it("falls back for a broken shared module too, unlike apper", async () => { + // The divergence worth knowing: apper would submit the broken shared file + // and the compiler would name it. Here the function is submitted flat, so + // the error it reports is the unresolved import instead. Both refuse the + // deploy; only the message differs. + expect( + await reached(ENTRY, { + [ENTRY]: + 'import { greet } from "../../shared/greeting.ts";\nDeno.serve(() => new Response(greet()));', + "base44/shared/greeting.ts": BROKEN, + }), + ).toEqual([ENTRY]); + }); +}); + +describe("code the compiler accepts, the walk must accept", () => { + it("follows imports in a function that uses top-level await", async () => { + // The walk compiled as iife, which rejects top-level await — so a legal + // Deno function using it failed the walk, fell back to the flat + // submission, and reported its shared import as unreachable. + const source = [ + 'import { greet } from "../../shared/greeting.ts";', + 'const cfg = await Promise.resolve("x");', + "Deno.serve(() => new Response(greet(cfg)));", + ].join("\n"); + const shared = { + "base44/shared/greeting.ts": + "export const greet = (n: string) => `hi ${n}`;", + }; + + expect(await reached(ENTRY, { ...shared, [ENTRY]: source })).toEqual([ + ENTRY, + "base44/shared/greeting.ts", + ]); + + const result = await bundle(await cfwBundleInput(ENTRY, source, shared)); + expect(result.ok).toBe(true); + }); +}); diff --git a/packages/functions-compiler/test/bundle-banner.e2e.test.ts b/packages/functions-compiler/test/bundle-banner.e2e.test.ts new file mode 100644 index 00000000..5ecb2a5b --- /dev/null +++ b/packages/functions-compiler/test/bundle-banner.e2e.test.ts @@ -0,0 +1,110 @@ +/** + * The bundle's first line — the only place a compiled module says what is + * inside it. Every case here is a real compile, because the question is whether + * the banner survives minification and lands where `head -1` finds it. + */ + +import { describe, expect, it } from "vitest"; +import { bundle } from "../src/bundler"; +import { compileFunctionShards } from "../src/shards/build"; +import type { ShardPolicy } from "../src/shards/plan"; + +const policy = (over: Partial = {}): ShardPolicy => ({ + shardSize: 5, + globalShardSize: 5, + maxShards: 4, + gzipCapBytes: 9_500_000, + ...over, +}); + +const fn = (name: string) => ({ + name, + entry: "main.ts", + files: { "main.ts": `Deno.serve(() => new Response("${name}"));` }, +}); + +const bannerOf = (module: string) => { + const first = module.split("\n")[0]; + const match = /^\/\/!b44:(\d+) (.*)$/.exec(first); + if (!match) { + throw new Error(`no banner on the first line: ${first.slice(0, 80)}`); + } + return { format: Number(match[1]), payload: JSON.parse(match[2]) }; +}; + +describe("the bundle names its own functions", () => { + it("puts a parseable banner on the first line, past minification", async () => { + const result = await compileFunctionShards( + [fn("sendReminder"), fn("health")], + policy(), + ); + expect(result.ok).toBe(true); + if (!result.ok) return; + + const { format, payload } = bannerOf(result.shards[0].module); + expect(format).toBe(1); + // Sorted, not in caller order: the module below is assembled in caller + // order and this line deliberately is not. + expect(payload.functions).toEqual(["health", "sendReminder"]); + expect(payload.telemetry).toBe(false); + expect(payload.runtimeSecrets).toBe(false); + expect(payload.compiler).toMatch(/^\d+\.\d+\.\d+/); + }); + + it("names only the functions in its own shard", async () => { + const result = await compileFunctionShards( + [fn("alpha"), fn("beta"), fn("gamma")], + policy({ shardSize: 2 }), + ); + expect(result.ok).toBe(true); + if (!result.ok) return; + + expect(result.shards).toHaveLength(2); + for (const shard of result.shards) { + expect(bannerOf(shard.module).payload.functions).toEqual( + [...shard.functions].sort(), + ); + } + }); + + it("reports the wrapper flags it was compiled with", async () => { + // Both change the emitted bytes and the secrets delivery a deploy must + // pair with, so the artifact has to carry them rather than be asked. + const result = await compileFunctionShards([fn("alpha")], policy(), { + postResponseTelemetry: true, + }); + expect(result.ok).toBe(true); + if (!result.ok) return; + + expect(bannerOf(result.shards[0].module).payload.telemetry).toBe(true); + }); + + it("is byte-identical across two compiles of one set", async () => { + // A version's identity is the hash of these bytes, so a banner carrying + // anything volatile — a timestamp, a build id — would re-mint a version for + // code that did not change. + const functions = [fn("alpha"), fn("beta")]; + const first = await compileFunctionShards(functions, policy()); + const second = await compileFunctionShards(functions, policy()); + expect(first.ok && second.ok).toBe(true); + if (!first.ok || !second.ok) return; + + expect(bannerOf(second.shards[0].module)).toEqual( + bannerOf(first.shards[0].module), + ); + }); + + it("leaves the single-function lane's bytes alone", async () => { + // `bundle()` is the legacy per-function path, and "which functions are + // inside" has one answer there. Keeping it bannerless keeps that lane + // byte-comparable with the engine apper still runs in production. + const result = await bundle({ + entry: "main.ts", + files: { "main.ts": 'Deno.serve(() => new Response("x"));' }, + }); + expect(result.ok).toBe(true); + if (!result.ok) return; + + expect(result.module.startsWith("//!b44:")).toBe(false); + }); +}); diff --git a/packages/functions-compiler/test/diagnostics-hooks.test.ts b/packages/functions-compiler/test/diagnostics-hooks.test.ts index 06a96c5e..7d29b760 100644 --- a/packages/functions-compiler/test/diagnostics-hooks.test.ts +++ b/packages/functions-compiler/test/diagnostics-hooks.test.ts @@ -7,13 +7,15 @@ */ import { afterEach, describe, expect, it, vi } from "vitest"; - import { bundle } from "../src/bundler"; import * as publicSurface from "../src/index"; import { type Field, type Level, logEvent, setLogSink } from "../src/log"; import { type CompilerTracer, setCompilerTracer } from "../src/tracing"; -const HELLO = { entry: "main.ts", files: { "main.ts": 'Deno.serve(() => new Response("ok"));' } }; +const HELLO = { + entry: "main.ts", + files: { "main.ts": 'Deno.serve(() => new Response("ok"));' }, +}; afterEach(() => { setLogSink(null); @@ -112,21 +114,36 @@ describe("published surface", () => { it("exports what the two consumers import", () => { // A rename here silently breaks apper's service at its next upgrade, and // nothing else in the suite imports through the package entry point. - expect(Object.keys(publicSurface).sort()).toEqual([ - "DenoCompatError", - "STATIC_EGRESS_ARTIFACT_MARKER", - "appFunctionSchema", - "bundle", - "bundleAppRequestSchema", - "bundleApp", - "bundleRequestSchema", - "classifyAppErrors", - "createGuardedFetch", - "importsConflictingPackage", - "installFetchGuard", - "setCompilerTracer", - "setLogSink", - ].sort()); + expect(Object.keys(publicSurface).sort()).toEqual( + [ + "DenoCompatError", + "STATIC_EGRESS_ARTIFACT_MARKER", + "appFunctionSchema", + "bundle", + "bundleAppRequestSchema", + "bundleApp", + "bundleRequestSchema", + "classifyAppErrors", + "createGuardedFetch", + "importsConflictingPackage", + "installFetchGuard", + "setCompilerTracer", + "setLogSink", + "cfwBundleInput", + "collectReachableFiles", + "compileFunctionShards", + "ShardCapacityError", + "assertWithinCapacity", + "planFreshShards", + "targetShardCount", + "BUNDLE_GZIP_LEVEL", + "WORKER_RAW_SIZE_CEILING_BYTES", + "judgeBundleSize", + "measureBundleBytes", + "workerGzipCapBreach", + "workerRawSizeBreach", + ].sort(), + ); }); it("keeps the static-egress marker in step with the Python constant", () => { diff --git a/packages/functions-compiler/test/shard-build.e2e.test.ts b/packages/functions-compiler/test/shard-build.e2e.test.ts new file mode 100644 index 00000000..5f8e6311 --- /dev/null +++ b/packages/functions-compiler/test/shard-build.e2e.test.ts @@ -0,0 +1,315 @@ +/** + * The compile → measure → split loop, and the rule that separates a whole-app + * build from what the HTTP service does: a partial result is a failure here. + * + * The size fixtures calibrate themselves — they compile the same functions once + * unbounded and set the cap from what came out — so they stay meaningful as the + * injected shim and runtime change size. + */ + +import { describe, expect, it } from "vitest"; +import { compileFunctionShards } from "../src/shards/build"; +import type { ShardPolicy } from "../src/shards/plan"; +import { measureBundleBytes } from "../src/shards/size"; +import { runInWorkerd } from "./workerd"; + +const policy = (over: Partial = {}): ShardPolicy => ({ + shardSize: 5, + globalShardSize: 5, + maxShards: 4, + gzipCapBytes: 9_500_000, + ...over, +}); + +const fn = (name: string, body = `"${name}"`) => ({ + name, + entry: "main.ts", + files: { "main.ts": `Deno.serve(() => new Response(${body}));` }, +}); + +const broken = (name: string) => ({ + name, + entry: "main.ts", + files: { + "main.ts": + 'import { x } from "./absent.ts";\nDeno.serve(() => new Response(x));', + }, +}); + +describe("a whole-app build", () => { + it("puts every function in exactly one shard and reports its size", async () => { + const result = await compileFunctionShards( + [fn("alpha"), fn("beta"), fn("gamma")], + policy({ shardSize: 2 }), + ); + expect(result.ok).toBe(true); + if (!result.ok) return; + + expect(result.shards).toHaveLength(2); + expect(result.shards.flatMap((s) => s.functions).sort()).toEqual([ + "alpha", + "beta", + "gamma", + ]); + for (const shard of result.shards) { + expect(shard.rawBytes).toBeGreaterThan(0); + expect(shard.gzipBytes).toBeGreaterThan(0); + expect(shard.gzipBytes).toBeLessThan(shard.rawBytes); + expect(shard.mainModule).toBe("_bundled.mjs"); + } + }); + + it("produces shards that actually route their functions", async () => { + const result = await compileFunctionShards( + [fn("alpha"), fn("beta")], + policy(), + ); + expect(result.ok).toBe(true); + if (!result.ok) return; + + const [shard] = result.shards; + const routed = await runInWorkerd(shard.module, { + headers: { "Base44-Function-Name": "beta" }, + }); + expect(routed).toMatchObject({ status: 200, text: "beta" }); + }); + + it("fails the whole build when one function fails to compile", async () => { + // The service ships the peers and attributes the failure. A whole-app build + // cannot: the app would deploy missing a handler. + const result = await compileFunctionShards( + [fn("alpha"), broken("beta"), fn("gamma")], + policy(), + ); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.failures.map((f) => f.function)).toContain("beta"); + expect(result.failures[0].errors?.[0].message).toContain("./absent.ts"); + }); + + it("refuses duplicate function names before compiling anything", async () => { + const result = await compileFunctionShards( + [fn("alpha"), fn("alpha")], + policy(), + ); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.failures[0].message).toContain("Duplicate"); + }); +}); + +describe("splitting on size", () => { + it("halves an oversized shard instead of emitting it", async () => { + // Calibrate against the pair itself: two functions share one injected shim, + // so a two-function module is only marginally larger than a one-function + // module. Setting the cap one byte under the pair makes it breach, and each + // half is necessarily smaller and fits. + const pair = [fn("alpha"), fn("beta")]; + const unbounded = await compileFunctionShards( + pair, + policy({ shardSize: 2 }), + ); + expect(unbounded.ok).toBe(true); + if (!unbounded.ok) return; + expect(unbounded.shards).toHaveLength(1); + const cap = unbounded.shards[0].gzipBytes - 1; + + const result = await compileFunctionShards( + pair, + policy({ shardSize: 2, gzipCapBytes: cap }), + ); + expect(result.ok).toBe(true); + if (!result.ok) return; + + // Planned as one shard, emitted as two — and every function still lands once. + expect(result.shards).toHaveLength(2); + expect(result.shards.flatMap((s) => s.functions).sort()).toEqual([ + "alpha", + "beta", + ]); + for (const shard of result.shards) { + expect(shard.gzipBytes).toBeLessThanOrEqual(cap); + } + expect(result.shards.map((s) => s.index)).toEqual([0, 1]); + }); + + it("fails when a single function alone is over the ceiling", async () => { + // Splitting is exhausted; there is nothing left to halve. + const result = await compileFunctionShards( + [fn("solo")], + policy({ gzipCapBytes: 64 }), + ); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.failures[0].function).toBe("solo"); + expect(result.failures[0].message).toContain("gzipped"); + }); + + it("measures what it emits", async () => { + const result = await compileFunctionShards([fn("alpha")], policy()); + expect(result.ok).toBe(true); + if (!result.ok) return; + const [shard] = result.shards; + expect(await measureBundleBytes(shard.module)).toEqual({ + rawBytes: shard.rawBytes, + gzipBytes: shard.gzipBytes, + }); + }); +}); + +describe("the same input compiles to the same bytes", () => { + // Load-bearing beyond tidiness: a version's identity is the hash of the + // compiled artifacts, never of the sources. Anything that shifts the emitted + // bytes mints a new version of unchanged code — so the compile has to be + // reproducible, and its inputs have to be the only thing that moves it. + it("emits identical modules for two runs of one set", async () => { + const functions = [fn("alpha"), fn("beta"), fn("gamma")]; + const first = await compileFunctionShards( + functions, + policy({ shardSize: 2 }), + ); + const second = await compileFunctionShards( + functions, + policy({ shardSize: 2 }), + ); + expect(first.ok && second.ok).toBe(true); + if (!first.ok || !second.ok) return; + + expect(second.shards.map((s) => s.module)).toEqual( + first.shards.map((s) => s.module), + ); + }); + + it("is sensitive to caller order inside a single shard", async () => { + // The single-shard path builds in caller order; only a multi-shard plan + // sorts by name. So the caller owns a stable order, and this is the test + // that says so out loud rather than leaving it to a comment. + const forward = await compileFunctionShards( + [fn("alpha"), fn("beta")], + policy({ maxShards: 1 }), + ); + const reversed = await compileFunctionShards( + [fn("beta"), fn("alpha")], + policy({ maxShards: 1 }), + ); + expect(forward.ok && reversed.ok).toBe(true); + if (!forward.ok || !reversed.ok) return; + + expect(forward.shards).toHaveLength(1); + expect(reversed.shards).toHaveLength(1); + expect(reversed.shards[0].module).not.toBe(forward.shards[0].module); + }); +}); + +/** A payload that gzip cannot collapse, so a shard's compressed size grows with + * the number of functions in it — which is what makes a multi-level split + * reachable. Deterministic (a fixed-seed xorshift), because a test that + * calibrates its own cap must measure the same bytes on every run. */ +const incompressible = (chars: number): string => { + let state = 0x9e3779b9; + let out = ""; + while (out.length < chars) { + state ^= state << 13; + state ^= state >>> 17; + state ^= state << 5; + out += (state >>> 0).toString(16).padStart(8, "0"); + } + return out.slice(0, chars); +}; + +const bulky = (name: string, chars = 40_000) => ({ + name, + entry: "main.ts", + files: { + "main.ts": `const payload = "${incompressible(chars)}";\nDeno.serve(() => new Response(payload.slice(0, 8) + "${name}"));`, + }, +}); + +describe("splitting more than once", () => { + // The recursion was only ever exercised one level deep, so the nested flatMap + // of recursive results went unchecked. apper's own test drives 8 → 4 → 2. + const eight = [ + "alpha", + "beta", + "gamma", + "delta", + "epsilon", + "zeta", + "eta", + "theta", + ].map((name) => bulky(name)); + + it("halves again when a half is still over the ceiling", async () => { + // Calibrate on the real thing: measure a two-function shard and a + // four-function shard, then set the cap between them. Eight functions then + // breach, each four still breaches, and only the pairs fit. + const measure = async (group: typeof eight) => { + const built = await compileFunctionShards( + group, + policy({ shardSize: group.length, maxShards: 1 }), + ); + expect(built.ok).toBe(true); + if (!built.ok) throw new Error("calibration compile failed"); + expect(built.shards).toHaveLength(1); + return built.shards[0].gzipBytes; + }; + const [pairBytes, quadBytes] = [ + await measure(eight.slice(0, 2)), + await measure(eight.slice(0, 4)), + ]; + // If this ever stops holding, the payload has become compressible and the + // calibration below is meaningless rather than wrong. + expect(quadBytes).toBeGreaterThan(pairBytes); + const cap = Math.floor((pairBytes + quadBytes) / 2); + + const result = await compileFunctionShards( + eight, + policy({ shardSize: 8, maxShards: 1, gzipCapBytes: cap }), + ); + expect(result.ok).toBe(true); + if (!result.ok) return; + + // One planned shard became four, which takes two levels of halving. + expect(result.shards).toHaveLength(4); + for (const shard of result.shards) { + expect(shard.functions).toHaveLength(2); + expect(shard.gzipBytes).toBeLessThanOrEqual(cap); + } + expect(result.shards.flatMap((s) => s.functions).sort()).toEqual( + eight.map((f) => f.name).sort(), + ); + expect(result.shards.map((s) => s.index)).toEqual([0, 1, 2, 3]); + }, 60_000); + + it("fails the whole build when one function survives every halving", async () => { + // The halves that fit are not a smaller success: a module missing a handler + // is a broken app. One function is bulky enough to breach alone, so the + // recursion reaches it and the build must fail rather than emit its + // siblings. + const group = [ + bulky("small-a", 2_000), + bulky("small-b", 2_000), + bulky("small-c", 2_000), + bulky("monster", 400_000), + ]; + const pairBytes = (async () => { + const built = await compileFunctionShards( + group.slice(0, 2), + policy({ shardSize: 2, maxShards: 1 }), + ); + if (!built.ok) throw new Error("calibration compile failed"); + return built.shards[0].gzipBytes; + })(); + const cap = (await pairBytes) + 1; + + const result = await compileFunctionShards( + group, + policy({ shardSize: 4, maxShards: 1, gzipCapBytes: cap }), + ); + expect(result.ok).toBe(false); + if (result.ok) return; + + expect(result.failures.map((f) => f.function)).toContain("monster"); + expect(result.failures[0].message).toContain("gzipped"); + }, 60_000); +}); diff --git a/packages/functions-compiler/test/shard-plan.test.ts b/packages/functions-compiler/test/shard-plan.test.ts new file mode 100644 index 00000000..14c2d0dd --- /dev/null +++ b/packages/functions-compiler/test/shard-plan.test.ts @@ -0,0 +1,140 @@ +/** + * Ported from apper's shard_planning.py semantics and the capacity refusal in + * deploy_app. Only the fresh path exists here; the incremental branch needs the + * previous deployment map and stays with the service. + */ + +import { describe, expect, it } from "vitest"; +import { + assertWithinCapacity, + planFreshShards, + ShardCapacityError, + type ShardPolicy, + targetShardCount, +} from "../src/shards/plan"; + +const policy = (over: Partial = {}): ShardPolicy => ({ + shardSize: 5, + globalShardSize: 5, + maxShards: 4, + gzipCapBytes: 9_500_000, + ...over, +}); + +describe("targetShardCount", () => { + it("is always one shard when sharding is off", () => { + // maxShards === 1 means the whole app bundles into a single Worker whatever + // the function count — the legacy per-app behaviour. Dividing instead would + // trip the capacity check for a flag-off app. + expect(targetShardCount(50, 5, 1)).toBe(1); + }); + + it("divides and rounds up otherwise", () => { + expect(targetShardCount(1, 5, 4)).toBe(1); + expect(targetShardCount(5, 5, 4)).toBe(1); + expect(targetShardCount(6, 5, 4)).toBe(2); + expect(targetShardCount(11, 5, 4)).toBe(3); + }); +}); + +describe("capacity", () => { + it("refuses a set over the product ceiling", () => { + expect(() => assertWithinCapacity(21, policy())).toThrow( + ShardCapacityError, + ); + expect(() => assertWithinCapacity(21, policy())).toThrow(/capacity of 20/); + }); + + it("accepts a set exactly at the ceiling", () => { + expect(() => assertWithinCapacity(20, policy())).not.toThrow(); + }); + + it("judges capacity at the global size, not a ratcheted-down one", () => { + // A lowered shardSize changes packing only. Judging capacity at it would + // lock an app out of deploying precisely because an earlier deploy had to + // pack smaller — the recovery path would be closed by the recovery itself. + expect(() => + assertWithinCapacity(20, policy({ shardSize: 2, globalShardSize: 5 })), + ).not.toThrow(); + }); +}); + +describe("planFreshShards", () => { + it("keeps the caller's order in a single shard", () => { + // The single-shard path does not sort in apper, and function order inside a + // combined module changes the emitted bytes. + expect(planFreshShards(["zebra", "alpha", "mango"], policy())).toEqual([ + ["zebra", "alpha", "mango"], + ]); + }); + + it("sorts by name before chunking a multi-shard plan", () => { + expect( + planFreshShards(["d", "b", "a", "c"], policy({ shardSize: 2 })), + ).toEqual([ + ["a", "b"], + ["c", "d"], + ]); + }); + + it("packs the remainder into a final short shard", () => { + expect(planFreshShards(["a", "b", "c"], policy({ shardSize: 2 }))).toEqual([ + ["a", "b"], + ["c"], + ]); + }); + + it("may plan more shards than maxShards when packing is ratcheted down", () => { + // Bounded by the function count, which is no more Workers than the legacy + // per-function topology used. A "shard count <= maxShards" assertion here + // would refuse an app that deploys fine today. + const plan = planFreshShards( + ["a", "b", "c", "d", "e", "f", "g", "h"], + policy({ shardSize: 1, globalShardSize: 5, maxShards: 2 }), + ); + expect(plan).toHaveLength(8); + }); + + it("puts everything in one shard when sharding is off", () => { + const names = ["a", "b", "c", "d", "e", "f"]; + expect(planFreshShards(names, policy({ maxShards: 1 }))).toEqual([names]); + }); + + it("refuses before planning when the set is over capacity", () => { + expect(() => + planFreshShards( + new Array(21).fill(0).map((_, i) => `f${i}`), + policy(), + ), + ).toThrow(ShardCapacityError); + }); +}); + +describe("an unusable policy is refused, not survived", () => { + // `shardSize: 0` made the chunking loop never advance: the capacity check + // passed at the global size and planning then hung. Python raises on the + // same input. + it("refuses a shard size that cannot advance", () => { + expect(() => + planFreshShards(["a", "b", "c"], policy({ shardSize: 0 })), + ).toThrow(/shardSize must be an integer of at least 1/); + expect(() => planFreshShards(["a"], policy({ shardSize: -1 }))).toThrow( + /at least 1/, + ); + }); + + it("refuses a fractional count", () => { + expect(() => + planFreshShards(["a", "b"], policy({ shardSize: 2.5 })), + ).toThrow(/at least 1/); + }); + + it("refuses the other two counts as well", () => { + expect(() => + planFreshShards(["a"], policy({ globalShardSize: 0 })), + ).toThrow(/globalShardSize/); + expect(() => planFreshShards(["a"], policy({ maxShards: 0 }))).toThrow( + /maxShards/, + ); + }); +}); diff --git a/packages/functions-compiler/test/shard-size.test.ts b/packages/functions-compiler/test/shard-size.test.ts new file mode 100644 index 00000000..0738ef64 --- /dev/null +++ b/packages/functions-compiler/test/shard-size.test.ts @@ -0,0 +1,160 @@ +/** + * Ported from apper's TestBundleSizeMeasurement in + * backend/tests/unit/app/cloudflare_functions/test_worker_bundle_size_limits.py. + * The constants and the verdict order are the contract; the rest of that file + * is deploy machinery and stays with the service. + */ + +import { randomBytes } from "node:crypto"; +import { gzipSync } from "node:zlib"; +import { describe, expect, it } from "vitest"; +import { + BUNDLE_GZIP_LEVEL, + judgeBundleSize, + measureBundleBytes, + WORKER_RAW_SIZE_CEILING_BYTES, + workerGzipCapBreach, + workerRawSizeBreach, +} from "../src/shards/size"; + +describe("measurement", () => { + it("counts raw as UTF-8 bytes and gzip as the compressed size", async () => { + const module = `const א = 1;${"x".repeat(5000)}`; + const { rawBytes, gzipBytes } = await measureBundleBytes(module); + + expect(rawBytes).toBe(Buffer.byteLength(module, "utf8")); + expect(rawBytes).toBeGreaterThan(module.length); // the non-ASCII identifier + expect(gzipBytes).toBe( + gzipSync(Buffer.from(module, "utf8"), { level: BUNDLE_GZIP_LEVEL }) + .byteLength, + ); + expect(gzipBytes).toBeLessThan(rawBytes); + }); +}); + +describe("ceilings", () => { + it("passes a module under both", () => { + expect(workerRawSizeBreach(1_000_000)).toBeNull(); + expect(workerGzipCapBreach(200_000, 9_000_000)).toBeNull(); + }); + + it("puts the raw ceiling exactly where Cloudflare says it is", () => { + // Cloudflare's own 10027 reads "exceeded the uncompressed size limit of + // 64 MiB", so the constant is 64 MiB in CF's units — decimal 64_000_000 + // would refuse 3 MiB early for no reason. + expect(WORKER_RAW_SIZE_CEILING_BYTES).toBe(67_108_864); + expect(workerRawSizeBreach(67_108_864)).toBeNull(); + expect(workerRawSizeBreach(67_108_865)).not.toBeNull(); + }); + + it("refuses the one real production rejection without uploading it", async () => { + // App 6a8c58a3, 2026-08-24: a single function whose module reached 97.4 MiB. + // Unsplittable, and it spent 30 s uploading 97 MB to be told the same thing. + const breach = workerRawSizeBreach(102_160_553); + expect(breach).not.toBeNull(); + expect(breach).toContain("uncompressed"); + expect(breach).toContain("64 MiB"); + }); + + it("refuses over the compressed cap too", () => { + expect(workerGzipCapBreach(9_500_001, 9_500_000)).not.toBeNull(); + expect(workerGzipCapBreach(9_500_000, 9_500_000)).toBeNull(); + }); + + it("reports the uncompressed breach when both are over", async () => { + // The exact one wins: it is the verdict Cloudflare would give. + const verdict = await judgeBundleSize( + "x".repeat(WORKER_RAW_SIZE_CEILING_BYTES + 1), + 1, + ); + expect(verdict.breach).toContain("uncompressed"); + }); + + it("returns the sizes and no breach for a module within both", async () => { + const verdict = await judgeBundleSize("export default 1;", 9_500_000); + expect(verdict.breach).toBeNull(); + expect(verdict.rawBytes).toBe(17); + expect(verdict.gzipBytes).toBeGreaterThan(0); + }); +}); + +describe("the headroom the cap was chosen for", () => { + // The regression net for the cap: real production figures it has to keep + // clearing. Ported from apper's TestBundleSizeMeasurement, and they matter + // more in this lane than in that one — Node's gzip reads ~0.7% heavier than + // Python's on identical input, so we sit that much closer to the cap. + it("clears the biggest bundle in production", () => { + // 8,495,351 B gzipped from 14.4 MB raw — app 6a04bc98, ~40 uploads a day, + // the largest compressed module over n=166,471 bundles. The ~1.0 MB (11.8%) + // it clears by IS the safety margin. Re-check this number before assuming a + // firing cap is wrong. + expect(workerRawSizeBreach(14_446_791)).toBeNull(); + expect(workerGzipCapBreach(8_495_351, 9_500_000)).toBeNull(); + }); + + it("clears a barely compressible bundle too", () => { + // Second-largest: 7,935,787 B from 11.8 MB raw, a 1.49x ratio against a + // measured floor of 1.33x. Raw size predicts compressed size loosely, which + // is why the verdict is taken on the compressed figure and never + // extrapolated from raw. + expect(workerRawSizeBreach(11_809_821)).toBeNull(); + expect(workerGzipCapBreach(7_935_787, 9_500_000)).toBeNull(); + }); + + it("does not refuse the largest module production ever uploaded", async () => { + // 35.3 MiB raw, uploaded successfully five times. A cap that refuses it is + // wrong however defensible the arithmetic looked. + const parts: string[] = []; + for (let i = 0; i < 700_000; i++) { + parts.push(`const v${i}=1;function f${i}(){return v${i}};`); + } + const { rawBytes, gzipBytes } = await measureBundleBytes(parts.join("")); + + expect(rawBytes).toBeGreaterThan(20_000_000); + expect(workerRawSizeBreach(rawBytes)).toBeNull(); + expect(workerGzipCapBreach(gzipBytes, 9_000_000)).toBeNull(); + }, 60_000); +}); + +describe("compression stays off the event loop", () => { + it("keeps the loop servicing while a large module compresses", async () => { + // Level 6 on the largest module production has uploaded costs ~650 ms in + // apper's measurement. Held on the loop it stalls everything else in the + // process, and concurrent shard builds queue behind each other. + const module = randomBytes(6_000_000).toString("hex"); + let ticks = 0; + const beat = setInterval(() => { + ticks += 1; + }, 1); + try { + await measureBundleBytes(module); + } finally { + clearInterval(beat); + } + + // Blocking would leave this at 0; offloaded to the zlib threadpool, the loop + // keeps firing timers. + expect(ticks).toBeGreaterThan(3); + }, 60_000); +}); + +describe("the Python and Node gzips do not agree", () => { + it("keeps the level pinned, because the two lanes measure differently", async () => { + // Measured on a real 122,324-byte compiled module: Python's gzip.compress + // at level 6 gives 42,765 bytes, Node's gzipSync gives 43,057 — Node reads + // ~0.7% heavier on identical input. So a module within ~0.7% of the cap can + // pass one lane and fail the other. + // + // The direction is the safe one: the local gate refuses slightly earlier + // than the service would, so nothing doomed slips through locally. It does + // mean a function very near the cap could build in apper and be refused + // here. Pinning the level is what keeps the gap this small and stable. + expect(BUNDLE_GZIP_LEVEL).toBe(6); + + const module = "export const x = 1;".repeat(2000); + const { gzipBytes } = await measureBundleBytes(module); + expect(gzipBytes).toBe( + gzipSync(Buffer.from(module, "utf8"), { level: 6 }).byteLength, + ); + }); +}); diff --git a/packages/functions-compiler/test/version.test.ts b/packages/functions-compiler/test/version.test.ts new file mode 100644 index 00000000..d8d7c7f5 --- /dev/null +++ b/packages/functions-compiler/test/version.test.ts @@ -0,0 +1,18 @@ +/** + * The banner carries this package's version, so the literal in src/version.ts + * is part of every compiled shard's bytes. If it drifts from package.json, the + * artifacts claim a version that was never published. + */ + +import { readFileSync } from "node:fs"; +import { describe, expect, it } from "vitest"; +import { COMPILER_VERSION } from "../src/version"; + +describe("COMPILER_VERSION", () => { + it("matches package.json", () => { + const pkg = JSON.parse( + readFileSync(new URL("../package.json", import.meta.url), "utf8"), + ); + expect(COMPILER_VERSION).toBe(pkg.version); + }); +});