Skip to content

Latest commit

 

History

History
219 lines (137 loc) · 30.5 KB

development-and-packaging.md

File metadata and controls

219 lines (137 loc) · 30.5 KB

Source Development and Windows Delivery

English | 简体中文

Source-run examples use production. The production license service is https://update.opentest.com, with no separate test server. Packaging retains test and production modes for local configuration and native materials. See License Configuration for selection rules and offline cache isolation.

Run the following commands at the host repository root, opentest/, using already installed locked dependencies. Plugins build independently. These commands do not publish or upload releases.

Source Development

# Run in three separate terminals
pnpm dev:frontend
pnpm dev:backend
pnpm dev:desktop

# Or start all three with one command
pnpm dev

The frontend uses Vite's source server and hot reload. Before each backend startup, Maven runs clean package -DskipTests, then java -jar starts the current reactor's executable cry-daq.jar. The desktop runs src/main.js. A prior complete pnpm build is unnecessary. Frontend and desktop commands prepare required SDK artifacts; the desktop also runs the existing ensure:native. Restart the corresponding command after desktop or backend changes.

Standalone commands each manage their own processes. The desktop connects to the external frontend and backend; closing it does not stop a standalone backend. pnpm dev manages the three components it creates and cleans them up on exit or component failure. Port conflicts are reported without taking over or terminating existing services.

Development and production use the same plugin registration lifecycle. After successful backend loading, the host waits for frontend registration. Waiting beyond two minutes produces one message without entering recovery. When a backend starts first or restarts independently, fully refresh the page after it is ready to obtain current startup candidates and register again. Backend loading failures, frontend activation failures or timeouts, and registration content mismatches are isolated to the affected external plugin, whose capabilities remain unavailable. Healthy plugins can still complete their required registration and bring the application to APP_READY. Global failures such as state-commit or resource-cleanup failures trigger recovery for the whole startup; individual requests retain bounded waits. Use the same runtime directory, ports, and configuration in all three terminals. See Frontend Registration Wait Rules.

pnpm dev and pnpm dev:desktop open DevTools by default; other entry points keep them closed. Set OPEN_DEVTOOLS=false to disable automatic opening; manual opening remains available. After updating the registration flow, restart the backend and desktop to load current service interfaces and the preload bridge. Refreshing the page alone does not update the desktop bridge.

pnpm dev:backend follows the launch command's lifetime. Ctrl+C, IPC disconnection, or exit of an outer pnpm/wrapper process triggers cleanup of this invocation. On Windows, a PowerShell process-chain watchdog holds real process handles before startup. Watchdog failure prevents or stops this service. Shutdown first asks Java through a dedicated pipe to stop acquisition and release resources. It waits 45 seconds by default, then terminates this process tree and reports failure on timeout.

Managed Java monitors parent-process exit before Spring initialization. Even if the launcher is forcibly terminated or Java is not ready or is blocked during cleanup, Java exits within 45 seconds of the parent's disappearance. Forced termination does not establish that acquisition results were saved completely. Launchers pass OPENTEST_PARENT_PID automatically; standalone Java debugging should not enable OPENTEST_DESKTOP_MANAGED manually.

Configuration selection is: an explicit OPENTEST_ENV_FILE, then .local/runtime.env, then the root runtime.env for compatibility. Only one file is loaded; files are not merged, and a missing explicit path does not fall back. Relative paths in runtime environment settings resolve from the host root even when launched from a subdirectory. Existing process values take priority for ordinary settings. Set JAVA_HOME and MAVEN_HOME explicitly in the current terminal when refreshing tool paths; environment files cannot replace stale inherited terminal values.

pnpm dev:frontend does not require the file or read or validate backend license selections. It does not check the JDK, gRPC port, or native materials, or create backend runtime/project directories. If the file exists, frontend settings such as ports are still loaded. The default frontend port is 19010, with proxy target 127.0.0.1:15021; override through FRONTEND_PORT and BACKEND_PORT. Startup still builds the public SDK before launching Vite. Complete development, standalone backend, and desktop entry points retain their license-configuration requirements.

Chart Artifacts

Charts use the public npmjs.com package @crysound/crychart@1.0.1 through pnpm. pnpm install --frozen-lockfile installs the exact locked version and verifies the download against the lockfile. The frontend build checks the installed package and records digests for packaging to verify. No chart runtime license or license environment variables are needed. See the CRYChart integration guide for installation and upgrades. vendor/crychart/ retains historical artifacts and validation records.

pnpm test:chart:browser uses real Chrome to check host drawing calls, data retention, viewport behavior, export, and disposal. Reports reside in .runtime/crychart-replacement/browser/. This does not replace Windows installer, performance, or one-hour continuous-run acceptance.

Runtime Paths and Dependencies

Item Default
Frontend URL http://127.0.0.1:19010; override with FRONTEND_PORT
Backend HTTP 127.0.0.1:15021; override with BACKEND_PORT
Backend gRPC 127.0.0.1:19021; override with GRPC_PORT
Runtime data .runtime/app; override with OPENTEST_RUNTIME_DIR
Projects and results projects under the runtime directory; override with OPENTEST_PROJECTS_DIR
Native material root apps/backend/crydaq-server/src/main/resources, containing native/<platform>; override with OPENTEST_NATIVE_MATERIALS_ROOT
Supplemental backend configuration apps/backend/.runtime/application-local.properties; override with OPENTEST_BACKEND_CONFIG
Desktop local configuration apps/desktop/.runtime/local-settings.json; override with OPENTEST_LOCAL_SETTINGS_PATH

The backend requires a full JDK 21 and Maven; a standalone frontend requires no JDK. The selected platform's runtime directory must contain native-libs-load-order.txt and all libraries it references. Prepare authentication, licenses, and precompiled runtime materials according to Framework Runtime.

Developers install base tools themselves. Run pnpm deps:check first; it does not check Python, which may still be needed for native module compilation. The project does not download or activate Node/JDK/Maven/Python/C++ tools. See Build Environment for requirements and configuration.

Public C++ source and build projects reside in apps/native. Precompiled backend runtime materials reside in apps/backend/crydaq-server/src/main/resources/native/<platform>, including libraries, device support files, and the main load manifest. <platform> is windows-amd64, linux-amd64, or macos-arm64; production overrides reside in its prd. Ordinary Java compilation does not generate these native materials or include them in the JAR. OPENTEST_NATIVE_MATERIALS_ROOT points to the resources root containing native/<platform>, with relative paths resolved from the host root.

Source startup selects the runtime platform and architecture. WSL explicitly launching Windows Java still selects windows-amd64. Mac/Linux packaging checks precompiled Java Native Interface (JNI) libraries, device host programs, and algorithm libraries for the current com.opentest namespace and prepareManagedRuntime interface. Supply compatible versions when materials are missing; JNI source is not in this repository, so packaging does not compile it. Runtime and device compatibility still require target-platform verification.

pnpm start and pnpm dev:backend check materials before Java startup. test uses the selected platform's base runtime directory directly; production copies base libraries and applies that platform's prd overrides. Base copying and native search paths exclude the prd subtree. Each production startup uses a separate staging directory, removes only that directory on normal shutdown, and leaves original materials unchanged. Missing matching production overrides stop startup. Verify material and device compatibility on the target machine.

The Windows base directory requires a matching License.dat; production additionally requires native/windows-amd64/prd/cry_analyst.dll and prd/License.dat. Both default Windows licenses are tracked with their matching libraries. Linux/macOS licenses are optional, but supplied licenses must be nonempty regular files. Production staging uses only licenses provided by production overrides. User entitlement certificates and logs are managed separately.

The plugin bridge no longer generates, stores, passes, or validates a bootstrap key. Plugin management writes still require a valid online or offline session. Install plugins through normal packages first, then follow Active Installation Development to update installed content during development.

pnpm start starts already built applications; pnpm build compiles them. The old node tools/start.mjs --dev invocation redirects to source development.

For a source desktop started through pnpm start, restart actions in plugin management and service monitoring wait for Java exit, then rerun tools/start.mjs using the original Node executable. Each startup creates separate native staging, and the old launcher cleans only its own directory. Installed applications restart themselves. Standalone development backends require a manual full restart.

Building the Backend from Source

To build and install backend Java artifacts into the local Maven repository, stop the source backend and run at the host root:

pnpm prepare:backend-deps
pnpm dev:backend

pnpm prepare:backend-deps runs mvn clean install -DskipTests directly in apps/backend: it rebuilds every backend module, compiles tests without executing them, then installs successful artifacts into the local repository through Maven's normal lifecycle. It uses terminal JDK 21, Maven from PATH, and Maven settings/defaults, normally %USERPROFILE%/.m2/repository. It does not load additional project environment configuration or filter modules. For complete test execution, run mvn clean install directly in apps/backend.

This entry point builds and installs backend artifacts only, without reading plugin source, generating plugin packages, or updating development kits. Stop the source backend first to avoid concurrent target writes. Maven failures propagate unchanged; already installed modules are not rolled back. Subsequent dev:backend still runs Maven clean package. Existing caches are not moved or deleted automatically, and build and dev do not invoke this preparation command automatically.

Run pnpm dev:backend from a configured host root. Before each startup, it calls Maven, clears old outputs, and rebuilds all modules. This standalone command starts neither frontend nor Electron. pnpm dev uses the same backend strategy while retaining required SDK and desktop native-module preparation.

Backend development always executes mvn -B --no-transfer-progress -DskipTests clean package, clearing each module's target before rebuilding, without an additional build-mode option. pnpm start remains the entry point for starting built applications.

Every development backend startup uses Maven clean to remove old classes and resources rather than reusing previous compilation outputs. Serialize builds within the same workspace.

Only a successful Maven exit permits startup of crydaq-server/target/cry-daq.jar. Failed or canceled builds stop startup without falling back to an old JAR. The launcher adds no special Maven RC, configuration, or clean-target restrictions and does not detect source changes during compilation. Stop the old source backend before restarting and serialize builds in the same source directory to avoid concurrent target writes.

Development launches the generated executable JAR with java -jar. pnpm dev and pnpm dev:backend explicitly allow mutable installed content; normal pnpm start and Java startup without explicit settings retain strict validation. Native-material checks, HTTP/gRPC port protection, health checks, and shutdown cleanup remain active. Default service ports are 15021/19021; backend console logs reside at .runtime/app/logs/backend/backend-console.log, following any runtime directory override.

Installers and Upgrade Packages

See Cross-Platform Packaging for Windows/macOS/Linux commands, native integration, and Mac notarization. The EXE and NSIS details below apply only to Windows.

The host builds, runs, and packages independently by default, without building or bundling external business plugins automatically. The reference project is OpenTest-Plugins. default-plugins.json lists only the public plugin fft-tools. To prepare it, explicitly run pnpm prepare:plugins. OPENTEST_PUBLIC_PLUGINS_SOURCE selects the OpenTest-Plugins source directory and defaults to ../OpenTest-Plugins. Relative paths resolve from the host root; the optional entry point requires all sources and dependencies specified by the catalog.

To deliver plugins in the application package, explicitly provide built .otplugin archives with --plugins-dir <directory>. Validation requires every plugin in default-plugins.json, with one package per identity; no additional package metadata JSON is needed. --edition community bundles only catalog plugins and ignores other archives. --edition full (the default) also accepts other compatible business plugin packages from --plugins-dir. Before building the host, packaging reads archive identities, versions, and compatibility ranges and rejects missing or duplicate entries. Bundled archives and index.json reside in the final resources/bundled-plugins/. For a single plugin, import a compatible package through normal plugin management after starting the host.

When plugins are explicitly bundled, first startup in a fresh runtime directory installs and registers them. Later startup, reinstallation, and upgrades retain user choices: uninstalled plugins are not reinstalled automatically, disabled plugins receive package updates while staying disabled, and newer installed versions are not downgraded. If a bundled plugin has the same version as an installed plugin but different contents, the host logs a warning, keeps the installed plugin, and continues startup. Increase the plugin version to deliver an automatic update. For manual reinstallation, use Open Bundled Plugin Directory in plugin management, then follow normal import and full restart. Uninstalling a plugin does not remove its bundled archive. Without bundled packages, use normal plugin import.

Installers and upgrades use maximum compression. Electron language resources retain only en-US, zh-CN, zh-TW, ru, de, fr, ja, ko, and es, matching the nine supported languages. Installer agreements and frontend translations retain their respective configurations.

Packaging mode, package type, and signing are independent. The common entry point is pnpm package --platform windows|mac|linux --mode test|production --kind installer|upgrade|both [--sign]. Platform defaults to the current system; an omitted mode still defaults to test, producing an unsigned installer. Both modes connect to the same production service. production selects production configuration and native materials without implying signing or both package types. Explicitly pass --mode production for production delivery.

Command Output
pnpm package --platform windows --kind installer --mode production --check Check production delivery requirements only
pnpm package --platform windows --kind installer --mode production Unsigned production installer EXE
pnpm package --platform windows --kind installer --mode production --sign Signed production installer EXE
pnpm package --platform windows --kind upgrade --mode production --sign Signed complete production upgrade EXE + update metadata
pnpm package:windows:both:prod:signed Signed production installer EXE + complete upgrade EXE + update metadata

Only the five common shortcuts in Cross-Platform Packaging are retained. Other combinations use the shared pnpm package entry point.

With both, SDK, Java, frontend, and desktop artifacts are prepared once and shared by both packages, with the same version and configuration. delivery.json is written only after all packaging, signing, and validation succeeds. Any failure returns a nonzero exit code; partial files are not a complete delivery.

Windows --sign signs host native files. When explicitly bundling plugins, first generate signed final .otplugin packages using shared plugin repository tools, then provide them with --plugins-dir. The host checks archive integrity, indexes, and publisher signatures on DLL, EXE, and NODE files both outside and inside JARs, using archives unchanged. Missing or invalid signatures cause failure. Independent plugins use pnpm build:signed <plugin> --platform windows; existing compiled artifacts can use pnpm package:signed. Original DLLs and JARs remain unchanged; signed copies and caches are ignored by Git. Pure Java/frontend plugins gain neither an extra Java signature nor a plugin signature.json. See Native Signing.

pnpm build:backend always runs Maven clean package -DskipTests, clearing module target directories before compilation so removed or renamed classes and resources cannot remain in delivery JARs. pnpm build and every installer/upgrade entry point reuse it. Java test source is compiled but tests are not executed.

Backend Maven downloads translations from Crowdin before resource copying by default. This requires an installed CLI and local .local/crowdin/backend.yml. Skip downloads with pnpm build:backend "-Dskip.crowdin.download=true", or pass the same Maven property through MAVEN_ARGS for desktop packaging. The default is false; failed downloads stop the build. See Localization for configuration and manual synchronization.

External plugins build and maintain translations separately. Their build and build:signed accept -Dskip.crowdin.download=true; plain plugin package reuses existing resources. Host packaging accepts built packages, so its MAVEN_ARGS controls the host backend build without downloading or compiling bundled plugins again. See Plugin Translation Builds and Maintenance.

Windows x64 upgrades are complete NSIS EXEs using a one-click flow, while installers allow directory selection. Both packaging modes use host identity org.opentest.framework, the same installation location, and the current user's projects, results, plugins, and settings. Upgrades preserve data and must be validated with increasing versions. They do not overwrite old com.crysound.opentest installations or guarantee arbitrary downgrade compatibility.

Packaging specifically excludes node_modules/@opentest/native-modules/build/** so MSBuild intermediates such as .lastbuildstate and .tlog do not produce excessively long paths when the old uninstaller moves files into temporary storage. Runtime dist files and package.json remain included; other dependencies' build directories are unaffected. This rule affects newly built packages without changing existing installation contents.

Related regression: node --test tools/__tests__/packaging.test.mjs checks installer and upgrade dependency collection excludes that build directory while retaining runtime native files and other dependencies' build outputs.

Windows main executables from both modes retain requireAdministrator, requesting administrator rights at startup. Installers and upgrades use the same configuration.

Installation agreements use the nine standard-edition license_*.txt files in apps/desktop/assets/, preserving original text and UTF-8 BOM. electron-builder recognizes English, Simplified Chinese, Traditional Chinese, Russian, German, French, Japanese, Korean, and Spanish agreements from that resource directory and displays the installation language. Upgrades launched with --updated skip the agreement page. Installer fonts retain MS Shell Dlg 2, 9pt, with Korean Malgun Gothic, 9pt.

The installer writes its language code, such as ko-KR, to installation app-cache/locale.txt as the default shared by all users of that installation. It also writes current-user userData/app-cache/locale.txt and locale.user-set. Language changes in the application update the same user record. For one Windows user, installer choices and application changes have equal priority; the last choice wins, and standard reinstall overwrites the previous user-language selection. Startup reads valid user settings first, then the installation default, then falls back to English. Upgrades without language selection preserve both records. The main process resolves installation defaults relative to the actual EXE directory, independently of the app.asar code path.

Installed applications load Java, backend, frontend, protocols, and native modules from their own resources, without requiring a source workspace or development JDK. Windows installation and upgrades always select native/windows-amd64 and apply its prd overrides for production. Only the selected material set is included, directly under resources/backend/native, without platform or prd subtrees. Mutable data defaults to %LOCALAPPDATA%/OpenTest; development .runtime/app data is not copied into installed applications automatically.

Production installed startup selects distinct available frontend HTTP, backend HTTP, and gRPC ports before window, interprocess communication, and plugin initialization. apps/desktop/src/utils/network.js provides address enumeration, isPortAvailable, sequential searching, and get-port@7.1.0 fallback. The default search starts at 3000 with up to 300 attempts; out-of-range starting values fall back to 1024. Actual startup begins at each service's configured port.

Port selection excludes ports already selected for this startup and rechecks get-port fallback results up to 300 times, avoiding duplicates and listeners on other addresses. The tool loads only when selecting ports. Final ports and health-check addresses are synchronized before backend startup. Services listen only on 127.0.0.1. Final ports are logged without rewriting configuration, and recreated windows in the same process reuse them. test packages and source runs use fixed ports. If a port becomes occupied after checking, startup reports failure and cleans up its own services without taking over existing processes.

Default Configuration and Overrides

The default file is root packaging.config.json, defining common configuration and test/production modes. Both modes use https://update.opentest.com; there is no separate test server. Commands select packaging modes without reading license fields from runtime.env. Supply any missing runtime materials yourself.

# Check required external materials and signing conditions only
pnpm package --platform windows --mode production --kind installer --check
pnpm package:windows:both:prod:signed --check

# Override default configuration
pnpm package:windows:both:prod:signed --config .local/packaging.json

Configuration applies default common, platform, mode, then platform-specific mode settings, followed by the same hierarchy from the supplied file. Relative paths resolve from the file defining each field. Fields not overridden retain their values and path origins. Do not put credentials in version-controlled defaults.

Example override; adapt paths to the local machine:

{
  "javaRuntime": "D:/OpenTest-Release/java",
  "outputDir": "D:/OpenTest-Release/output",
  "modes": {
    "production": {
      "backendNativeMaterialsRoot": "D:/OpenTest-Release/materials",
      "backendConfig": "D:/OpenTest-Release/application-production.properties",
      "desktopConfig": "D:/OpenTest-Release/local-settings-production.json"
    }
  }
}
Setting Meaning
javaRuntime Windows x64 Java 21 tracked by ordinary Git, including release, bin/java.exe, and the complete runtime; defaults to vendor/java/windows-x64
backendNativeMaterialsRoot Resources root containing native/windows-amd64 and its prd; defaults to apps/backend/crydaq-server/src/main/resources; Windows delivery does not select materials from the build machine's platform
modes.*.licenseUrl License service URL for the corresponding packaging mode
backendConfig Backend configuration delivered in the package; production defaults to application-production.properties
desktopConfig Desktop configuration delivered in the package; production defaults to local-settings-production.json
backendProfile Spring backend profile name: qa for test, prod for production; these names do not imply separate servers
signing Certificate fingerprint, certificate store, SignTool, and timestamp settings
outputDir Delivery output root; defaults to dist/releases

Explicit --sign reads WIN_SIGN_CERT_SHA1 or signing.thumbprint. The selected certificate must have a private key, allow code signing, and be unexpired. The default store is CurrentUser; use signing.store: "LocalMachine" if needed. SignTool prefers SIGNTOOL_PATH or a configured path, otherwise searches Windows SDK installations. SIGNTOOL_TIMESTAMP_URL overrides the timestamp service.

Signing signs and verifies individual files, then verifies the final EXE. It records the selected certificate's publisher in update configuration, preserving in-app update publisher checks. Unsigned builds require no certificate and record no signed publisher identity. Signed clients still validate future update publishers, so do not supply unsigned packages for their in-app upgrades. Validate manual installation and in-app upgrades separately. The backend remains a standard JAR without Xjar.

Output is fixed at dist/releases/<platform>-<arch>/<mode>/<signed|unsigned>/, with installer/ and upgrade/ subdirectories. Windows upgrades include latest.yml, Mac includes latest-mac.yml, and Linux TAR.GZ produces no automatic-update metadata. After material checks pass, only selected package directories and the current signing variant's old delivery.json are cleared. Other package types, modes, and signing variants remain. --check does not clear output. Only full success writes delivery.json, recording this invocation's version, mode, platform, architecture, signed/notarized state, and SHA-256 digests. It does not establish versions of retained artifacts from other invocations. Serialize packaging commands to avoid concurrent source-build writes. Move historical packages out yourself; old output layouts are not deleted automatically. Staging resides in Git-ignored .runtime/packaging/. Commands do not publish to the update server; explicit Mac --notarize uploads to Apple.

Explicit pnpm sign:backend:native copies base native/windows-amd64 materials into .runtime/native-signing/test before signing. With --mode production, it applies that platform's prd overrides and writes .runtime/native-signing/production. Staging omits the prd subtree. verify:backend:native reads the staged signed artifacts for the same mode and reports missing materials if signing has not run. Original DLLs are not modified. Production packaging signs independently in its delivery staging directory and does not depend on these manual signing artifacts.

Distribution Java

A complete Windows x64 Java 21 runtime is tracked by ordinary Git at vendor/java/windows-x64, currently Azul Zulu 21.0.11+10-LTS. Cloning supplies these packaging materials without copying from another machine, downloading, or restoration. The directory contains release, bin/java.exe, conf, lib, legal, and vendor documentation. .gitattributes disables line-ending conversion to preserve original vendor bytes. See Java Materials for versions and updates.

The distribution runtime is separate from compilation's JAVA_HOME; packaging does not fall back to the local JDK. Override javaRuntime through --config for another material set. Run pnpm package --platform windows --mode production --kind installer --check before complete packaging. Prepare development tools, dependencies, local configuration, and release signing certificates as described above.

See Local Runtime Directories for storing and cleaning runtime data, build caches, and temporary files.

Verification

pnpm test:delivery
pnpm --filter @opentest/desktop test:run
pnpm --filter @opentest/desktop test:integration
pnpm check:boundaries

Packaging compiles source, but backend builds use -DskipTests; this does not mean Java tests passed. Configuration, orchestration, and path tests do not replace actual certificate signing, installation, upgrade data retention, plugin import and restart, login, or hardware acceptance. Prechecks report missing private materials or certificates.

Windows native licenses reside beside matching DLLs: resources/native/windows-amd64/License.dat accompanies base materials and resources/native/windows-amd64/prd/License.dat accompanies production overrides. pnpm dev, pnpm dev:backend, and pnpm start reject missing or empty Windows licenses. Linux/macOS permit absence, but supplied licenses must be nonempty regular files. Reads accept License.dat and license.dat; multiple case variants in one directory cause errors. Staging normalizes the name to License.dat, using only the selected platform's prd license for production. Source examples select production and the production service. Entry points check configuration and material structure; material providers confirm actual compatibility.