Repository navigation
feat(vrowzer): build libraries in the browser with vrowzer.build() - #75
Merged
Merged
Conversation
…transformer chunk The single-file Web Worker aggregate already renames the bundled `__vite__injectQuery` helper, because the host Vite injects its own `__vite__injectQuery` import into a module that has a dynamic import of a variable. The chunked build (transformer.js, vite.js and their chunks) did not. A chunk that declares the helper and also gets the host's import fails to load with "Identifier '__vite__injectQuery' has already been declared". A Worker config that imports `vite` then stops the preview's Web Worker with an error that has no message. Nothing breaks today, because the chunk that holds the helper gets no such import. Which chunk holds it depends on how rolldown groups the modules, though, and the build code of #36 changes that grouping. - Rename the helper in every chunk that declares it, as the aggregate does. - Stop the build when a chunk still declares `__vite__injectQuery`, or when no chunk holds the renamed helper.
…tream does The bundles of vite-dev-server replaced `process.env.NODE_ENV` with "development", so `resolveConfig()` resolved `isProduction` to false even for a build. The build Worker of #36 runs the same code and needs "production". - The bundles keep `process.env.NODE_ENV` as a reference. rolldown replaces it for the browser platform by default, so it is defined to itself. - `resolveConfig()` handles `NODE_ENV` as upstream does again. It reads and sets `process.env.NODE_ENV`, which is the process polyfill of each Worker, and reads `process.env.VITE_USER_NODE_ENV`, instead of `import.meta.env`. The lookup through Vite Task stays commented out. - A bundle check stops the build when `isProduction` is fixed again. The dev Web Worker still gets "development", because `resolveConfig()` sets it before the code that reads it runs. The new unit tests pin the upstream behavior of `resolveConfig()`. Under vitest, `import.meta.env` reflects `process.env`, so they pass with the old code too, and only the bundle check catches it.
Port the build core of Vite 8.3.1 (`build.ts`), so that the build Worker of #36 can run upstream's `build()`. It builds with `@vrowzer/rolldown`, the browser build of rolldown. - build.ts: `build()`, `createBuilder()`, `resolveConfigToBuild()`, `resolveRolldownOptions()`, `buildEnvironment()`, the library helpers (`resolveLibFilename()`, `resolveBuildOutputs()`), the log handling (`onRollupLog()`, `enhanceRollupError()`), `ChunkMetadataMap` and the hook wrappers that inject the environment. - Commented out with `NOTE(kazupon)`: the watch mode (`build.watch` throws), `esmExternalRequirePlugin` for SSR webworker builds, the DevTools, and the build plugins that the browser build does not support yet (preload, the esbuild and terser minifiers, license, manifest and Worker builds). - `config.tsconfig` and `build.chunkImportMap` (Vite 8.3) are read without their types, because the fork's config does not have them yet. - The classes of `@vrowzer/rolldown` are not assignable to the `rolldown` types, so the bundle is cast. - `resolveBuildPlugins()`: `vite:prepare-out-dir` (ported; it does nothing with `write: false`), `vite:rollup-options-plugins`, the native reporter (ported) and the native load fallback. - `resolvePlugins()` registers `vite:define` and the `pre` build plugins for builds. The dev Web Worker keeps handling `define` in clientInjections. - types/metadata.d.ts is synced with upstream (`AssetMetadata`). - The Worker aggregate loads `./define` and `@vrowzer/rolldown` eagerly. The unit tests port upstream's tests of the library helpers, the option resolution and the log handling. They build with the Node rolldown of the same version in place of `@vrowzer/rolldown`. The tests of HTML, CSS, SSR and watch builds are left for later stages.
Builds now resolve modules with rolldown's native resolve plugin
(`viteResolvePlugin`), as upstream Vite does. The JavaScript resolve
plugin returns `packageJsonPath: null` for a module without a
package.json, and rolldown 1.2.12 rejects it in a build ("Failed to
convert JavaScript value `Null` into rust type `String`").
- Port upstream's `oxcResolvePlugin()`. The resolved config does not have
`tsconfig` (Vite 8.3) yet, so it is not passed.
- `resolvePlugins()` uses it for builds. The dev Web Worker keeps the
JavaScript resolve plugin for now, so that the preview stays as it is.
- The unit tests that load `resolve.ts` mock the native plugin. A new
test builds a library that imports packages through `exports`,
`browser` and `module`.
`@vrowzer/vite-dev-server/builder` builds a snapshot of the project files with the ported `build()`. The build Worker of vrowzer (#36) loads it, and gets the outputs and the warnings, or the errors of the build. - `buildProject(files, inlineConfig)` writes the snapshot to the virtual file system of the Worker, and builds it with `root: '/'`, `configFile: false`, `build.write: false`, `build.emptyOutDir: false` and `logLevel: 'warn'`. It returns the outputs, keyed by the path from the output root (binary assets as ArrayBuffers), and the warnings. It also adds the public files, which Vite copies only when it writes. - A plugin of the builder defaults `build.lib.formats` to `['es']`, and rejects what the browser build does not support yet: HTML app builds (no `build.lib`), formats other than `es`, multiple library entries, `build.cssMinify`, the terser and esbuild minifiers, `build.watch`, `build.ssr`, `build.manifest`, `build.ssrManifest` and `build.license`. - A failure throws `BuildProjectError`. Its `errors` keep the code, plugin, id, location and frame of each rolldown error, without colors. An output outside the output directory, and a public file with the path of an output, are errors too. - The builder is bundled into one file, like the transformer of the dev Web Worker. Its entry sets `globalThis.Buffer` before the rolldown binding initializes, and loads `config.ts` before `build.ts`. A bundle check keeps the first order.
`vrowzer.build(options)` builds the project for production in a build Worker, and resolves with the outputs and the warnings (part of #36). Only library builds (`build.lib`) in the `es` format are supported for now. - The runtime keeps a copy of the project files: the files of `ready()` with the default `/index.html`, and the changes of `addFile()`, `updateFile()` and `deleteFile()`, recorded before they are sent. `build()` takes a snapshot of it when it is called, so later changes do not reach the running build. ArrayBuffers are copied. - Each build creates a build Worker (`build-worker.ts`). It loads the builder of `@vrowzer/vite-dev-server`, and merges the options over the Worker config with Vite's `mergeConfig()`. The runtime terminates the Worker when the build ends, whatever the outcome, so nothing stays in memory between builds. - One build at a time. `build()` rejects before `ready()` resolves to true, after `dispose()`, while another build is running, and when `@vrowzer/vite-plugin` does not enable it (`__VROWZER_INTERNAL_BUILD__`). - A failed build rejects with `VrowzerBuildError`, whose `errors` have the code, plugin, id, location and frame of each error. A build that does not finish within the new `buildTimeout` option (120 seconds by default), an aborted `signal`, an error of the build Worker and `dispose()` reject the build, and terminate the build Worker. - The messages of the build Worker are `V_BW_READY`, `V_BW_BUILD` and `V_BW_RESULT`. The results of other builds are ignored. The `build` option of the plugin, which enables it, comes in the next commit.
`Vrowzer({ build: true })` enables `vrowzer.build()` (#36). It is
disabled by default.
- The plugin injects `__VROWZER_INTERNAL_BUILD__`, as it does for the
other runtime settings, so that `build()` rejects with guidance when
the option is off. The value is checked like the other reserved
defines.
- With the option, the Worker config is also prebundled for builds,
with `process.env.NODE_ENV` set to "production", to
`node_modules/.vrowzer/config.build.bundled.mjs`. It has the same
inputs as the dev config, which the dev watch already covers.
- The build Worker entry (`vrowzer/build-worker`) gets that config, both
in the host dev server and in host builds.
- Without the option, the entry is replaced with a stub, so the host
output does not include the builder (about 2.7 MB of rolldown and the
Vite build). The e2e projects now emit a stub of 211 bytes.
- README: the `build` option.
…in errors
The native plugins of rolldown, e.g. the transform of TypeScript
(`builtin:vite-transform`), report an error without `plugin` and `id`.
The plugin is only the prefix of the message, and the module is only in
the header of the code frame ("╭─[ src/index.ts:1:23 ]"). A syntax error
in a TypeScript file is reported this way, so `vrowzer.build()` could not
tell which file and which plugin failed.
The builder now reads them from the message when rolldown does not set
them: the plugin of a `PLUGIN_ERROR`, and the module of the code frame,
whose path is relative to the root of the build Worker.
The new fixture `integration/build/` enables the `build` option of the plugin, and has two test plugins. One adds exports that show the plugin instance and the `process.env.NODE_ENV` of the Worker config. The other never finishes the build of a module that contains `HANG_BUILD`. `build.integration-test.ts` builds a library in the browser, and imports the output in Node: - the outputs (the entry, a lazy chunk, the CSS and the source maps), the exports, JSON, an inlined SVG, `define`, `import.meta.env`, tree shaking and a dynamic import. The plugin hooks run in production mode, with new plugin instances in a new build Worker for each build. - one file (`codeSplitting: false`), and with and without minification - the snapshot: a change made while a build runs is not built, and added, updated and deleted files reach the next build - a syntax error reports the module, the plugin and the location, and the build works again once it is fixed - a second build is rejected while one runs - the timeout, an aborted signal and `dispose()` reject a build and terminate the build Worker. The next build works with a new one. - the build Workers and the Workers that they start are released - unsupported options are rejected: HTML app builds, the `umd` format and `cssMinify` - the previews keep working after successful, failed and timed-out builds The restart fixture does not enable the option, so its test checks that `build()` rejects without creating a Worker.
- README: `build()` and its options, results and errors, `buildTimeout`, what the build does not support yet, what `dispose()` does to a build, and the build Worker in the architecture. - The API references are generated again for the new types, and the constructor of `VrowzerBuildError` documents its parameters.
Short builds one after another stalled. In Chromium, the twelfth build in a row never loaded the builder, and waited for `buildTimeout`. A terminated build Worker stays for about 2 seconds with one thread of rolldown, which Chromium stops by force only after 2 seconds. A build of a small library took about 0.14 seconds, so the closed build Workers piled up, and with about 11 of them the WebAssembly of a new build Worker did not start. Disposing the WASI binding of rolldown before the Worker is terminated did not help. The runtime now remembers when it closed each build Worker. When 4 of them closed within the last 2.5 seconds, a build waits until the oldest of them has had that long to stop, before it creates its build Worker. Longer builds do not wait. The wait does not count toward `buildTimeout`, and the signal and `dispose()` cancel it. The timeout tests no longer keep a pending assertion while they advance the fake timers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #36. Stacked on #74 (rolldown 1.2.12).
vrowzer.build()builds the project for production in the browser. This PR is stage 1 of #36: the foundation (a build Worker, a snapshot of the project files and the Vite build core of upstream) and library builds.What it does
build(options)takes a subset of the Vite config (base,mode,defineandbuild), and resolves with{ files, warnings }.filesis keyed by the path from the output root. JavaScript, CSS, source maps and text assets are strings, and binary assets areArrayBuffers./publicare added unlessbuild.copyPublicDirisfalse.build.lib) in theesformat:rolldownOptions.output.codeSplitting: falsedefine,mode,import.meta.envand the plugin hooks of the Worker configbuild()is called: the files ofready(), with the default/index.html, and the file operations called before. Later changes do not reach the running build.VrowzerBuildError. Itserrorshave the message, code, plugin, module, location and frame of each error, without colors.buildTimeout(new, 120 s by default),signalanddispose()reject a build, and terminate its build Worker.Vrowzer({ build: true })of@vrowzer/vite-pluginenables it. Without it,build()rejects with guidance, and the host output does not include the builder.How it works
vrowzer/build-worker). It loads the builder (@vrowzer/vite-dev-server/builder), and runs upstream'sbuild()withwrite: falseon a virtual file system. The runtime terminates the build Worker when the build ends.process.env.NODE_ENVis"production", innode_modules/.vrowzer/config.build.bundled.mjs. Each build evaluates it in a new Worker, so the plugins start fresh./dist/client/*.mjs) are not in it.root: '/',configFile: false,build.write: false,build.emptyOutDir: falseandlogLevel: 'warn'. The options ofbuild()are merged over the Worker config with Vite'smergeConfig().Ported from upstream Vite 8.3.1
build.ts:build(),createBuilder(),resolveConfigToBuild(),resolveRolldownOptions(),buildEnvironment(), the library helpers, the log handling,ChunkMetadataMapand the hook wrappersplugins/prepareOutDir.ts, andplugins/reporter.ts(the native reporter)oxcResolvePlugin(), the native resolve plugin, used by builds onlypackageJsonPath: null, which rolldown 1.2.12 rejects in a build.resolveConfig()decidesNODE_ENVat runtime, as upstream does. The bundles of vite-dev-server no longer fix it to"development". The dev Web Worker still gets"development".NOTE(kazupon):config.tsconfigandbuild.chunkImportMap(Vite 8.3) are read without their types, until the config of the fork has them.Differences from the issue
terminate(), and Chromium stops it by force after 2 s.buildTimeout.Not supported yet
These reject with
VrowzerBuildError(VROWZER_UNSUPPORTED_OPTION):es, and libraries with more than one entrybuild.cssMinify), and the terser and esbuild minifiersbuild.watch, SSR builds, manifests and license files.envfiles, CSS Modules, tsconfig and production CJS dependencies come in stage 3. The builds are tested in Chromium only.Other changes
__vite__injectQueryhelper in every chunk, as the single-file aggregate does, and a bundle check guards it.vite.isProductiondecided at runtime, and the builder'sglobalThis.Bufferset before rolldown's binding.pluginorid, e.g. a TypeScript syntax error frombuiltin:vite-transform. The builder reads them from the message.There are no breaking changes. With
build: false(the default), the host output only gets a 142-byte stub of the build Worker.Measurements
macOS arm64, Chromium, a small library:
build(): 218 ms for the first build, 137–143 ms afterwards. Each includes creating the build Worker and loading the builder.build: true:web-worker-builder-*.js), and a build Worker entry of 1,664 bytesrolldown-worker.jsare shared with the previewsTests
All on macOS (arm64):
vp run build, the type checks, oxlint, knip and the format check pass.Unit tests: 2098 passed, 3 skipped (124 new).
NODE_ENVresolveConfig()registersbuild.spec.tstestsbuild()with a fake Worker, covering the state checks, the snapshot, closing the Worker on every outcome, other build ids,VrowzerBuildErrorand the pacing.Integration tests:
build.integration-test.tsbuilds a library in the browser, and imports it in Node. It covers:dispose()build()rejects without the option.e2e, with the Vite caches cleared first: