pre-generated test keys for faster CI#587
Open
anshalshuklabot wants to merge 17 commits intoblockblaz:mainfrom
Open
pre-generated test keys for faster CI#587anshalshuklabot wants to merge 17 commits intoblockblaz:mainfrom
anshalshuklabot wants to merge 17 commits intoblockblaz:mainfrom
Conversation
- Add keygen command to zeam-tools for generating XMSS key pairs - Add test-keys submodule (anshalshuklabot/zeam-test-keys) with 32 pre-generated keys - Update getTestKeyManager to load pre-generated keys from disk (near-instant) - Falls back to runtime generation if test-keys submodule not initialized - build.zig: add xmss and types imports to tools target Each XMSS key takes ~1 min to generate at runtime. Loading 32 pre-generated keys from SSZ files is near-instant, significantly speeding up CI and tests.
Collaborator
|
This PR is end to end created using openclaw, I'm just trying it out using a separate bot account. Will review if later. |
- Search upward for test-keys/ directory (handles test runners in subdirs) - Load pre-generated keys for first 32 validators, generate rest at runtime - If 40 validators needed: loads 32 from disk + generates 8 at runtime - Fixes CI failure where keys weren't found
Ensures test-keys submodule is available in every CI job. Fixes CACHE MISS in Dummy prove jobs where pre-generated keys weren't found.
tools now imports @zeam/xmss which requires libhashsig_glue.a. Without this dependency, the tools binary and tests fail to link on CI where Rust libs aren't pre-built in the cache.
The tools executable was missing proper Rust library linking (libhashsig_glue.a, libmultisig_glue.a, liblibp2p_glue.a) and platform-specific frameworks (macOS). Replace manual step.dependOn with addRustGlueLib() which handles object files, linkLibC, linkSystemLibrary(unwind), and macOS frameworks.
gballet
changed the title
addRustGlueLib only adds object files and link flags but not the step dependency that ensures Rust libs are built first. Both dependOn AND addRustGlueLib are needed, matching how all other targets are wired.
The relative path search for test-keys/hash-sig-keys failed because Zig test binaries run from cache directories, not the repo root. Fix by injecting the absolute repo path as a build option (test_keys_path) from build.zig using pathFromRoot(), and add build_options import to key-manager module. All 5 build commands verified locally: - zig build all - zig build test - zig build simtest - zig build spectest:generate - zig build spectest
Two fixes: 1. Track actually_loaded count instead of assuming all num_preloaded keys were inserted. On partial failure (corrupt SSZ, addKeypair error), the fallback loop now starts from the correct index, so no validators are left without keys. 2. Replace boolean owns_keypairs with num_owned_keypairs counter. Pre- generated keys (index < num_owned_keypairs) are freed on deinit while cached runtime keys (index >= num_owned_keypairs) are skipped. This prevents leaking ~20MB private keys when the mixed ownership path is taken.
Replace index-threshold ownership check with a per-key owned_keys hashmap. addKeypair() marks keys as owned (freed on deinit), addCachedKeypair() does not (for borrowed/cached runtime keys). This fixes the regression where CLI-loaded keys (arbitrary validator indices) were never freed because num_owned_keypairs stayed 0. Now any caller using addKeypair() gets correct cleanup regardless of index order.
Collaborator
|
CI test time reduction summary. |
ch4r10t33r
reviewed
Mar 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Each XMSS key takes ~1 min to generate at runtime. Loading 32 pre-generated keys from SSZ files is near-instant, significantly speeding up CI and tests.