Skip to content

policy(rust): enforce cached cross-platform Dylint and make it a setup-soldr default for every Rust app #9

Closed

Description

@zackees

Context

Under #5 (RUST-008), cross-target Dylint is the cheap way for a Rust app to catch platform bugs on every PR: check-only passes on Linux for each OS family, with no native runners. It is only affordable if it is cached. Dylint lints workspace crates only, but every pass must first type-check the entire dependency graph with Dylint's pinned nightly, and the stable build caches cannot stand in for that. Uncached, dependency checking is most of the job, and the cost multiplies with every target.

This issue asks to enforce cached cross-platform Dylint as fleet policy, and to make it automatic in setup-soldr. A Rust app should get warm host + cross-target Dylint by setting dylint: true and declaring its targets, with no per-repository cache plumbing. Every Rust app in the fleet has this problem.

It builds on #1 (Dylint cache identity mechanics), #5 (cross-target checks, cache budget, toolchain sourcing order) and #7 (kill-switched layers, green runs that save nothing). It does not duplicate them.

Evidence (zackees/clud, measured 2026-09-28)

All numbers come from job logs via gh api .../jobs/<id>/logs.

Uncached baseline

clud#1541 adds Windows MSVC and macOS arm64 cross passes to clud's single Linux Dylint job. PR run 36467947289:

Pass Dependency units Workspace crates start at Pass ends
Host x86_64-unknown-linux-gnu 405 113 s 144 s
x86_64-pc-windows-msvc 336 113 s 147 s
aarch64-apple-darwin 278 31 s 60 s

About 75–80% of every pass is dependency type-checking. The Dylint job took 7m45s.

Why nothing was cached

There are three independent causes, and any one of them is enough to defeat caching:

  1. A kill switch in the consumer. clud's _dylint.yml carried ZCCACHE_DISABLE: "1" from its first version (clud #487, 2026-07-06). setup-soldr still reported layers=...zccache-unit..., restored a 22-byte archive (produced an unusable payload), and saved a 253-byte one. This is the CACHE-010 class in policy(cache): detect kill-switched layers and green default-branch runs that save nothing (fleet-wide) #7.
  2. soldr never writes the Dylint success marker on the soldr dylint path. After a green soldr dylint --all run, the file named by SOLDR_DYLINT_SUCCESS_MARKER does not exist. A diagnostic step in dispatch run 36471128435 printed (marker missing).
    • The marker is written only by the cargo front door (cargo_front_door/run.rs around L888, when dylint_entrypoint).
    • Because the marker is missing, setup-soldr's post.ts (around L2330) skips both Dylint layers on every run, including green main runs: dylint-cache: no matching successful Dylint marker - skipping save and dylint-output-cache: Dylint did not complete successfully - skipping save. See main run 36463271709.
    • Even if the marker were written, the identity strings disagree, as fix: detect Dylint recompilation and cache identity drift in fast CI #1 noted:
      • soldr writes channel|release|commit with a host-qualified channel (nightly-2026-05-28-x86_64-unknown-linux-gnu).
      • setup-soldr compares against the short nightly-2026-05-28.
    • Both are present in soldr 0.9.25 and setup-soldr 67ed401.
  3. The output-cache key includes the commit. resolve-setup.ts hashes source_revision: githubSha into setup-soldr-dylint-output-v1-*, and the key is restored exact-only. Every new commit misses by design, so even a working save would never help a PR.

Measured fix

The experiment re-enabled zccache and used a fresh suffix, dylint-zccache, because the empty entry on main is immutable under its key. It ran as two workflow_dispatch runs of the same workflow on a branch; save-cache: auto saves on dispatch:

Pass Before (uncached PR run) Cold seed 36471128435 Warm 36472266045 Warm zccache hits
Host 2m45s 3m06s 28s 551 HIT / 0 MISS
Windows MSVC 2m41s 3m22s 49s 417 HIT / 0 MISS
macOS arm64 1m21s 1m38s 69s 322 HIT / 0 MISS
Whole Dylint job 7m45s 9m13s 3m31s —

What a warm cross pass still pays: toolchain downloads every run

Pass Downloaded each run Time
Windows target rust-std, LLVM 21.1.5, xwin-cache MSVC sysroot, 4 system libraries ~16 s
macOS target rust-std, llvm-tools 20.1.7 (~25 s alone), Apple SDK 14.5, 4 system libraries ~40 s

That is sourcing tier 2 (downloaded) in #5's order, when it should be tier 1 (vendored or cached in setup-soldr). With it cached, the warm macOS pass would drop from 69s to about 30s.

Proposal

Policy rules

Candidate ID Mechanical signal
RUST-013 A Rust app's quick-gate Dylint does not run with a dependency layer that is keyed on the nightly compiler identity + Cargo.lock + target set, restored across commits (not keyed on the source SHA), and saved from the default branch. Evidence required: a green default-branch run saves a non-trivial layer, and the next run on a new commit shows at least 90% dependency-unit hits for every declared Dylint target. A kill switch (ZCCACHE_DISABLE, soldr --no-cache cargo dylint) or a SHA-only key is a violation.
CACHE-013 A green Dylint run leaves setup-soldr's success marker absent, or with an identity that differs from setup-soldr's, so the foundation/output layers skip saving. Detect no matching successful Dylint marker / did not complete successfully - skipping save on a green default-branch run.

setup-soldr: make it the default, not per-repo plumbing

  1. Declared cross targets. In Dylint mode, accept the targets (or derive them from soldr's target declarations) and run the host plus per-target check-only passes. soldr lint rust --cross-dylint --target ... (soldr#3428) already exists.
  2. Dependency layer on by default. In Dylint mode, zccache-unit (or soldr dylint cook, Linux-only, default-branch save per policy(rust): make coverage-per-compile-minute the objective; require cheap cross-target checks, a cache budget, and a toolchain sourcing order #5 RUST-010) holds the nightly check units:
  3. Fix the success marker. The soldr dylint path writes SOLDR_DYLINT_SUCCESS_MARKER, and the identity is normalized between the short and host-qualified channel in one owner: soldr, setup-soldr, or a shared contract.
  4. Output cache. Remove source_revision from the key or add a lockfile-scoped restore prefix. Alternatively, retire this layer in favor of the dependency layer, if the workspace-crate part is cheap. It is (about 25–30s per pass here).
  5. Cross sysroots and rust-std. Cache each declared target's nightly rust-std with the Dylint foundation. Reuse or vendor the prepare cache family for LLVM, llvm-tools, the MSVC xwin sysroot and the Apple SDK, so a warm pass downloads nothing.
  6. Report layer size and hit rate per target in the final cache summary, so RUST-013 can be checked from logs.

Acceptance criteria

  • RUST-013 and CACHE-013 are added to policy-rust.md / policy-general.md, each with its signal and fixtures.
  • RED fixtures:
    • clud PR run 36467947289 (cold, kill switch)
    • clud main run 36463271709 (green, nothing saved)
    • dispatch run 36471128435 (marker missing)
  • GREEN fixture: dispatch run 36472266045 (100% hits, 28s/49s/69s passes).
  • setup-soldr acceptance:
    • A consumer that sets only dylint: true plus its Dylint targets gets, on a new commit after one default-branch seed:
      • at least 90% dependency hits per target
      • no marker skip
      • no toolchain or sysroot downloads on the warm run
    • The Dylint layers stay within a declared budget. The 589 MB for three targets is the reference point.
  • A fleet sweep lists every Rust app with Dylint and reports per target: cached or cold, hit rate, and layer size.

Decisions

Open questions

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions