Repository navigation
Conversation
This was referenced Nov 18, 2026
Compiling the same source twice gave different module bytes, for two reasons: - Each module was stamped with the wall-clock time of its compile. Nothing reads that timestamp, but it made every compile's output new. The compiler now stamps the epoch, as FileStructure already allows for reproducible output. - A variable named _ got a name numbered by a counter shared by every compile in the JVM, so a compile's names depended on what else the JVM had compiled before it. A method now numbers within itself, as a property outside a method already did. The timestamp is what made the build's compiles cascade: any recompile of a module, even for a comment, gave new bytes, so every module compiled against it was out of date, and so were their tests and the javatools tests that use the XDK. Now a change that does not change a module's bytes stops at that module. A trailing comment in lib_ecstasy costs one compile (7s) instead of a rebuild of all 22 XDK modules and their tests (96s). The counter matters for compiles that share a JVM: the embedding API, the language server and compiles of several modules in one process.
javatools' build-info.properties records the commit and branch it was built from, for xtc --version. That file is in the javatools jar, which every task that compiles or runs modules takes as an input, so each commit, branch switch or rebase made every XDK compile, every XDK test and the javatools tests out of date, and no build cache entry from another commit could be used: 106s for a build after a commit that changed nothing. Nothing that compiles or runs modules reads the commit or the branch, and the version test compares the output with the same file. So the XTC plugin's tasks now take the javatools jar as a classpath, which it is, and the plugin's projects and javatools compare classpaths without those two properties, with Gradle's runtime classpath normalization. A different commit now rebuilds only the jar and the distribution, which should carry it: 11s. The version numbers in the same file are still compared, and a change to any class or other resource in the jar still reruns everything.
XtcRunTask took the unpacked XDK directory as an input of its own, besides the XDK's modules and its jars, which XtcLauncherTask already takes. The directory holds only those, so it tracked nothing more, but it compared javatools.jar as a plain file, with the commit and branch that its build-info.properties records, which the classpath comparison ignores. So every commit reran the tests of a project that consumes the XDK, such as manualTests (9s). The task still depends on the task that unpacks the XDK, which it names explicitly.
The javatools tests took the sources of lib_net, lib_json and lib_web as inputs, because ModuleInfoTest parses them, so any edit to those libraries, even a comment, reran all of the tests (62s) for the sake of one class that takes 0.2s. ModuleInfoTest now runs in a task of its own, moduleInfoTest, which check runs, with those sources as its inputs. A comment in lib_json now costs a build of 8s instead of 70s.
…ainst ModuleApi writes a module file without its methods' code, the source text of their bodies, its timestamp and its source directory, followed by whether each method is abstract, native or has code. The compiler reads nothing else from the modules it compiles against, so a module compiles to the same bytes against two versions of an import with the same API: a build can then skip compiling the importers after a change inside a method body. ModuleApiTest compiles a library in two versions that differ only inside a method body, and checks that their APIs and an importer's bytes are the same; and that a changed signature, an interface method that gains a default body, a changed constant and a new method each change the API. ModuleApiStripTest checks that no method of the XDK's modules keeps code or source text in the API.
A compile task took the whole files of the modules it imports as input, so any change to ecstasy, such as one string inside a method body, compiled all 21 modules that import it again, though each came out byte for byte the same: 117 s of the 170 s of task time in that build. An artifact transform now turns each imported module into its API (ModuleApi), and a compile task takes those as its input in place of the modules. The compile still reads the modules themselves, and the tasks that run or ship them, tests and the distribution among them, still take the whole modules as input. The transform loads javatools for each run and closes the class loader after it, so that no class loader keeps a jar open that the build rewrites; with a javatools that has no ModuleApi it copies the modules, which keeps the old behavior. The same body change in ecstasy now compiles 1 module instead of 22 and reruns the tests: 37 s instead of 59 s.
… one ConfigurationCacheCompatibilityTest failed on CI: its project depends on a file that is not a module and has no javatools, and the transform failed the build where the compile had only compared the file. Wherever the transform cannot compute an API, it now copies the file, so that a compile depends on all of it, as before: with no javatools, with a javatools that has no ModuleApi, for a file it cannot read as a module, or in a JVM too old to load javatools, which the compile itself runs in a JVM of its own. The API files have the extension .api: they are only compared, never loaded, and are not valid module files.
lagergren
force-pushed
the
lagergren/deterministic-xtc
branch
from
November 18, 2026 11:24
de8e0d4 to
739d0f3
Compare
lagergren
marked this pull request as ready for review
November 18, 2026 12:09
The compiler stamps every module with the epoch, so that the same source
compiles to the same bytes. To record a time instead, xtc build
--timestamp stamps the modules it compiles with the time of the compile
("now") or with an ISO-8601 instant, such as a release commit's time. The
Gradle plugin passes it from the compiler extension's timestamp setting,
which is unset by default. A module that is up to date is not compiled
again and keeps its stamp; --rebuild restamps it.
A module's description shows the timestamp only when a time was set,
instead of always showing the epoch, which says nothing.
ggleyzer
reviewed
Nov 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A commit that changes no source, a comment in a library, or a change inside one method rebuilt and retested most of the XDK. With these commits a build reruns only what the change affects. Part of #665: the compiler no longer stamps the time of the compile; the absolute source directory and file times remain.
Note: The ModelApi is a well understood and well tested concept that only has existed in the language server as a kotlin class for quite a while. I just ported that to Java and put it in Javatools so that the Gradle Plugin can also utilize it for its dependency analysis for the XtcCompile tasks.
What changes
Make the compiler's output depend only on its input
xtc build --timestampstays available for compatibility, although nothing uses it. It takesnow, or an ISO-8601 instant such as a release commit's time. The module description shows a timestamp only when one was set.--timestamponly if a build setsxtcCompile { timestamp = ... }. The XDK build must not set it: it would bring back the rebuilds and the build cache misses this PR removes._variables, instead of using a counter shared by the whole JVM.DeterministicOutputTestcompiles a module twice in one JVM and compares the bytes.Stop a commit that changes no source from rebuilding the XDK
javatools.jarrecords the git commit and status inbuild-info.propertiesforxtc --version, and every XDK compile and test took the jar as an input.@Classpath, and runtime classpath normalization ignores justgit.commitandgit.status. A real change to javatools still reruns everything.javatools.jar, the installed XDK and the distribution zip all have the new one, andxtc build --versionprints it. Normalization only changes how Gradle compares the jar, never what is in it.Stop a commit from rerunning the run and test tasks of XDK consumers
XtcRunTaskhad a duplicate input: the whole unpacked XDK directory,javatools.jarincluded, compared as plain files.manualTests' tests rerun on every commit. The modules and jars are still tracked by the existing inputs.Run
ModuleInfoTeston its own, with the XDK sources it readslib_net,lib_jsonandlib_websources as inputs, for the one test class that reads them.moduleInfoTesttask, whichcheckruns.Define the API of a module: what a module that imports it compiles against
ModuleApiwrites a module without its methods' code, their source text, its timestamp and its source directory, followed by whether each method is abstract, native or has code.ModuleApiTest: a change inside a method body keeps the API and the importer's bytes, while a changed signature, an interface method that gains a default body, a changed constant and a new method each change the API.ModuleApiStripTest: no method of the XDK's modules keeps code or source text.Compile a module again only when the API of a module it imports changes
ecstasy, even one string inside a method, compiled the 21 modules that import it again, and each came out byte for byte the same: 117 s of the 170 s of task time in that build.ModuleApi, a file that isn't a readable module, and a Gradle JVM too old to load javatools (the compile itself runs in a JVM of its own). TheConfigurationCacheCompatibilityTestfailure on this PR's first run was the no-javatools case.<module>.apiand live only in Gradle's transform cache. Nothing loads them, and they aren't valid module files: the module format is unchanged.Build times
./gradlew buildafter a full build, with the lang build off andmanualTestsnot attached, as they are by default. The master column is the last commit before this PR. The changes inside a method were measured on this branch before and after commit 6.lib_ecstasylib_jsonmoduleInfoTestecstasymethodecstasy's API (a new class)With
manualTestsattached, its tests no longer rerun after a commit or a comment. They rerun when a library they use really changes. This was measured on another branch with commits 1 to 4.Checked in a new checkout with an empty
GRADLE_USER_HOMEecstasymethodecstasy.xtcchangedecstasyclean, then buildAfter the round trip through the cache, all 69 outputs were byte for byte what the build had made: every module, the javatools jar, the distribution zip and the installed XDK.
Things to know
git.commitandgit.statusin abuild-info.propertiesat the root of a classpath entry. It's narrow, but third-party projects get it too.javatools_utilstasks came from the cache, and a lang marker skipped the lang validation. Once this is merged, a pull request that doesn't change javatools, the plugin or the XDK sources can reuse the XTC compiles and tests too.javatools_jitbridge's javadoc still reruns on a commit. It's cheap.lib-runnermakesrunnerClient,webcliandcliinputs to the javatools tests, though they never load them.Testing
./gradlew buildpasses on this branch, and in a new checkout with an emptyGRADLE_USER_HOME.DeterministicOutputTest,CompilerCounterAtomicTest,moduleInfoTest,ModuleApiTestandModuleApiStripTest../gradlew spotlessCheckis clean.CI,RUN_INTEGRATION_TESTSandmanualTestsattached (all 62 plugin tests, none skipped), themanualTestsCI tasks, andxtc-app-template greetagainst the published Maven repository.