Cleanup of expression-based version selectors with explicit prerelease control

README.md

@@ -93,12 +93,12 @@ llvm.toolchain(
 )
 ```
 
-Prereleases must be requested explicitly. Following the common SemVer range
-convention, they are excluded from `latest`, `first`, and stable-only version
-requirements. An exact `llvm_version = "23.1.0-rc3"` selects that release, and
-a requirement opts in by naming a prerelease with the same
-major/minor/patch tuple. Thus `latest:>=23.1.0-rc1` can select `23.1.0-rc3`,
-while `latest:>=23` and `latest:>22` cannot select it.
+Automatic `latest` and `first` selection excludes prereleases by default, including
+when a requirement explicitly mentions a prerelease. Set `llvm_version_allow_prereleases = True`
+to make matching LLVM prereleases eligible, or
+`linker_version_allow_prereleases = True` for catalogued linkers, including mold. Exact pins
+such as `llvm_version = "23.1.0-rc3"` select that release directly regardless of the
+option.
 
 `extra_llvm_distributions` is not limited to official LLVM artifacts. Its keys
 may also be complete download URLs or absolute local paths, so internally
@@ -236,25 +236,105 @@ for how CC toolchains work in general.
 
 ### Requirements
 
-Version attributes can be requirements of the form `first`, `first:<condition>`,
-`latest` or `latest:<condition>`.
+Version requirements let projects bind automatic LLVM upgrades to deliberate `toolchains_llvm`
+release upgrades. A `toolchains_llvm` release can add new LLVM release identifiers to its
+checksummed catalogue. Upgrading the pinned module then makes those releases eligible without a
+separate edit to an exact `llvm_version`. Requirements restrict which of those additions a project
+will adopt, for example by keeping LLVM within major version 23:
 
-In case of `latest`, the latest distribution matching the optional `condition`
-will be selected.
+```starlark
+llvm.toolchain(
+    name = "llvm_toolchain",
+    llvm_version = "latest:major==23",
+)
+```
 
-In case of `first`, the first distribution matching the optional `condition`
-will be selected.
+With this policy, upgrading `toolchains_llvm` to a release that adds a newer stable LLVM 23
+version also upgrades the selected compiler, when an archive is available for the execution host.
+Even a module release that only adds LLVM release identifiers can therefore update LLVM as part
+of the same dependency upgrade. Adding LLVM 24 to the catalogue leaves this selection within
+LLVM 23; adopting LLVM 24 requires an explicit selector change. Bare `latest` permits major
+upgrades introduced by module upgrades as well. Exact versions remain useful when an LLVM
+release must match another artifact, such as an instrumented libc++ build.
+
+Selection uses the catalogue bundled with the pinned `toolchains_llvm` module plus explicitly
+configured extra distributions. Builds do not query upstream for newly published LLVM releases.
+With the same module, configured catalogue and execution host, the selected release stays the
+same. A module upgrade or an explicit catalogue change is what makes additional releases eligible.
+
+Version attributes accept exact versions or selectors of the form `first`, `first:<conditions>`,
+`latest`, or `latest:<conditions>`. `latest` picks the newest matching release available for the
+execution host; `first` picks the oldest. The same clause syntax applies to LLVM and catalogued
+linker versions, and lets one configuration accommodate hosts whose available archives differ.
+
+Conditions are comma-separated and combined with AND. toolchains_llvm supplies `major`, `minor`,
+and `patch` as numeric release-component constants, so `latest:major==23,minor>=2` stays within
+LLVM 23 and requires a minor version of at least 2. Comparisons and function expressions use
+[mboworks_bzl's SemVer mode](https://mboworks.github.io/bzl/site/tag/0.7.0/docs/versions.html#semver).
+
+<!-- prettier-ignore -->
+| Selector                    | Selection policy                               |
+| --------------------------- | ---------------------------------------------- |
+| `latest:major==23`          | Newest stable LLVM 23 release for the host     |
+| `latest:major==23,minor==1` | Newest stable 23.1 release for the host        |
+| `latest:~=23.1.0,!=23.1.2`  | Newest stable compatible release except 23.1.2 |
+| `first:>=22,<23`            | Oldest stable LLVM 22 release for the host     |
+
+For all comparison operators, wildcard matching and compatible-release boundaries, see
+[Compatible and matching requirements](https://mboworks.github.io/bzl/site/tag/0.7.0/docs/versions.html#compatible-and-matching-requirements).
+Function clauses such as `major()==23` and more advanced inspection are documented under
+[Requirement expressions and constants](https://mboworks.github.io/bzl/site/tag/0.7.0/docs/versions.html#requirement-expressions-and-constants).
+
+`llvm_version_allow_prereleases` and `linker_version_allow_prereleases` each
+default to `False`. They independently control bare and constrained
+`first`/`latest` selectors for LLVM and catalogued linkers, respectively. Requirements always
+apply; enabling prereleases does not bypass ranges or host availability. Naming
+an RC in a requirement does not implicitly enable prereleases. Exact version pins
+are unaffected.
 
-The condition consists of a comma separated list of semver version comparisons
-supporting `<`, `<=`, `>`, `>=`, `==`, `!=`. Examples:
+```starlark
+llvm.toolchain(
+    name = "llvm_rc",
+    llvm_version = "latest:>=23.1.0-rc1,<23.1.0",
+    llvm_version_allow_prereleases = True,
+)
+```
+
+This selects the newest matching release candidate available for the execution
+host. With `llvm_version_allow_prereleases = False`, that RC-only range has no matching release.
+With the option enabled, expressions such as `is_prerelease()==True` can further
+restrict selection to prereleases; `is_prerelease()==False` can restrict a
+particular selector to stable releases.
+
+Linker selectors use the same bzl SemVer comparisons, requirement expressions,
+projection constants, and first/latest selection as LLVM. For example:
+
+```starlark
+llvm.toolchain(
+    name = "llvm_with_mold_rc",
+    llvm_version = "latest:major==23",
+    llvm_version_allow_prereleases = False,
+    linker = {"": "mold"},
+    linker_version = "latest:major==3,is_prerelease()==True",
+    linker_version_allow_prereleases = True,
+)
+```
 
-- `latest`
-- `latest:>=20.1.0`
-- `latest:>17.0.4,!=19.1.7,<=20.1.0`
-- `first:>=15.0.6,<16`
+This selects stable LLVM and a mold prerelease independently. A matching mold
+prerelease must exist in the configured linker catalogue (for example, through
+`extra_linker_distribution_files`); selection does not discover live releases.
+Linker candidates are restricted to the execution platform before version selection,
+just as LLVM candidates are restricted to compatible archives. An exact
+`linker_version = "3.0.0-rc1"` does not require enabling prereleases. Bare `mold`
+without a version derives an exact pin from the injected mold module. The linker
+flag has no effect on bundled linkers, `auto`, or absolute executable paths.
+Both flags also work in feature overrides; changing only the linker flag can
+reuse the base LLVM archive.
 
 It is further possible to provide the version or requirement from an environment
-variable with a fallback version or requirement. In this case it is important to
+variable with a fallback version or requirement. This works for both LLVM and
+linker versions, for example `linker_version = "getenv(MOLD_VERSION,latest:major==3)"`
+with `--repo_env=MOLD_VERSION=3.0.0`. In this case it is important to
 also use the bazel flag `--repo_env=LLVM_VERSION=version_or_requirement`. It is
 important to use both correctly because otherwise the resulting builds are not
 reproducible. The main purpose of using an environment variable to encode the

toolchain/deps.bzl

@@ -59,7 +59,6 @@ def bazel_toolchain_dependencies():
             url = "https://github.com/bazel-contrib/bazel_features/releases/download/v1.52.0/bazel_features-v1.52.0.tar.gz",
         )
 
-    # Load mboworks_bzl for version comparisons.
     if not native.existing_rule("mboworks_bzl"):
         http_archive(
             name = "mboworks_bzl",

toolchain/internal/BUILD.bazel

@@ -15,12 +15,15 @@
 load("@bazel_skylib//rules:diff_test.bzl", "diff_test")
 load(":feature_overrides_test.bzl", "feature_overrides_test_suite")
 load("linker_distributions.bzl", "linker_version_test_writer")
-load("llvm_distributions.bzl", "distributions_test_writer", "prerelease_test_writer", "requirements_test_writer")
+load(":llvm_distributions_test.bzl", "distributions_test_writer", "prerelease_test_writer", "requirements_test_writer")
+load(":version_selectors_test.bzl", "version_selectors_test_suite")
 
 exports_files(["template.modulemap"])
 
 feature_overrides_test_suite(name = "feature_overrides_tests")
 
+version_selectors_test_suite(name = "version_selectors_tests")
+
 linker_version_test_writer(
     name = "linker_version_test_output",
     testonly = True,

toolchain/internal/feature_overrides_test.bzl

@@ -113,6 +113,19 @@ def _archive_sharing_test_impl(ctx):
     asserts.false(env, can_share_llvm_archive(base, {}, reset = ["distribution"]))
     asserts.false(env, can_share_llvm_archive(base, dict(toolchain_roots = {"": "/opt/llvm"})))
 
+    asserts.true(env, can_share_llvm_archive(base, dict(linker_version_allow_prereleases = True)))
+    asserts.true(env, can_share_llvm_archive(base, {}, reset = ["linker_version_allow_prereleases"]))
+    asserts.false(env, can_share_llvm_archive(base, dict(llvm_version_allow_prereleases = True)))
+    asserts.false(env, can_share_llvm_archive(base, {}, reset = ["llvm_version_allow_prereleases"]))
+    for llvm_allow in [False, True]:
+        for linker_allow in [False, True]:
+            settings = dict(llvm_version_allow_prereleases = llvm_allow, linker_version_allow_prereleases = linker_allow)
+            override = llvm_feature_override(features = ["test"], **settings)
+            merged = merge_override(dict(base, llvm_version_allow_prereleases = not llvm_allow, linker_version_allow_prereleases = not linker_allow), settings)
+            asserts.equals(env, llvm_allow, merged["llvm_version_allow_prereleases"])
+            asserts.equals(env, linker_allow, merged["linker_version_allow_prereleases"])
+            asserts.equals(env, settings, override["settings"])
+
     # Resetting an external root must create a new download: the base has no
     # generated LLVM archive repository that the variant could share.
     external_base = dict(base, toolchain_roots = {"": "/opt/llvm"})

toolchain/internal/linker_distributions.bzl

@@ -17,7 +17,9 @@
 load("@mboworks_bzl//bzl/versions:versions.bzl", "versions")
 load("//toolchain/internal:common.bzl", "is_absolute_path")
 load("//toolchain/internal:distributions_repo.bzl", "load_jsonc")
-load("//toolchain/internal:llvm_distributions.bzl", "resolve_version")
+load("//toolchain/internal:version_selectors.bzl", "version_selectors")
+
+_semver = versions.with_mode("semver")
 
 def catalogued_linker_reference(selection, version):
     """Return the catalogue reference for a linker selection, or None."""
@@ -31,7 +33,7 @@ def llvm_23_macos_uses_bundled_linker(llvm_version, linker_dict):
     """Whether any Darwin target retains a bundled LLVM 23 linker without arm64e.x1 support."""
 
     # LLVM 23.1.3 backports arm64e.x1 support to ld64.lld (llvm/llvm-project#224185).
-    if not llvm_version.startswith("23.") or not versions.lt(llvm_version, "23.1.3"):
+    if not llvm_version.startswith("23.") or not _semver.lt(llvm_version, "23.1.3"):
         return False
     default = linker_dict.get("", "")
     return any([
@@ -80,7 +82,7 @@ def _merge_catalogue(catalogue, extra):
                 merged_versions[version] = merged_platforms
             merged_platforms.update(platforms)
 
-def resolve_linker_distribution(catalogue, linker, version_selection, platform):
+def resolve_linker_distribution(catalogue, linker, version_selection, platform, *, allow_prereleases = False):
     """Resolve a version against releases available for an execution platform."""
     versions = catalogue.get(linker)
     if not versions:
@@ -92,7 +94,7 @@ def resolve_linker_distribution(catalogue, linker, version_selection, platform):
     ]
     if not available_versions:
         fail("linker '{}' has no executable for the {} execution platform".format(linker, platform))
-    version = resolve_version(version_selection, available_versions)
+    version = version_selectors.resolve(version_selection, available_versions, allow_prereleases = allow_prereleases)
     return version, versions[version][platform]
 
 def _mold_version(rctx):
@@ -124,6 +126,7 @@ def _linker_distributions_repository_impl(rctx):
             linker, version_selection = "mold", _mold_version(rctx)
         else:
             linker, version_selection = _parse_reference(reference)
+        version_selection = version_selectors.resolve_environment(rctx, version_selection)
         versions = catalogue.get(linker)
         if not versions:
             fail("unknown linker '{}'; known linkers: {}".format(linker, ", ".join(sorted(catalogue.keys()))))
@@ -136,7 +139,7 @@ def _linker_distributions_repository_impl(rctx):
             exec_os = platform.split("-")[0]
             if all([target and target.split("-")[0] != exec_os for target in references[reference]]):
                 continue
-        _, distribution = resolve_linker_distribution(catalogue, linker, version_selection, platform)
+        _, distribution = resolve_linker_distribution(catalogue, linker, version_selection, platform, allow_prereleases = rctx.attr.linker_version_allow_prereleases)
 
         extraction_dir = "_extract/{}".format(index)
         rctx.download_and_extract(
@@ -167,6 +170,7 @@ exports_files(["linkers.json"])
 linker_distributions_repository = repository_rule(
     implementation = _linker_distributions_repository_impl,
     attrs = {
+        "linker_version_allow_prereleases": attr.bool(default = False),
         "catalogue": attr.label(
             allow_single_file = [".json", ".jsonc"],
             default = Label("//toolchain/distributions:linkers.jsonc"),
@@ -191,7 +195,7 @@ def _linker_version_test_writer_impl(ctx):
         "latest:>=2.40.0,!=2.42.1",
     ]
     version_results = [
-        "{} -> {}".format(selection, resolve_version(selection, available))
+        "{} -> {}".format(selection, version_selectors.resolve(selection, available))
         for selection in selections
     ]
     reference_results = [