Repository navigation
feat(arc): add a node-cache capability that shares a node directory between a profile's pods - #90
Merged
Merged
Conversation
…etween a profile's pods A node-cache capability mounts one hostPath directory per profile on the runner container and prepares the subdirectories it names as the runner's user, setting a variable for each to its path, so a package manager's content-addressed store survives the pod and the next job starts with it filled. A pruner DaemonSet empties a profile's cache when it outgrows its limit and no runner pod of the profile runs on the node.
|
This security review couldn’t start because the paying account has exhausted its Codex allowance and has insufficient available credits. The payer can check their balance and add credits in the Codex usage dashboard, then try again. If you do not manage the paying account, contact this repository's admins. |
|
🎉 This PR is included in version 1.31.0 🎉 The release is available on:
Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a fourth capability kind,
node_cache, for #83. It gives a profile's runner pods on a node one shared directory of that node, so a package manager's store survives the pod and a job on a warm node starts with it filled.The runner pod gets a
hostPathvolume at<host path>/<namespace>/<release>, mounted once on the runner container. An init container prepares the subdirectories the capability names as the runner's user, and eachdirsvariable is set to its subdirectory's path, sopnpm_config_store_dir: pnpmputs pnpm's store in/node-cache/pnpm. The workflow's own cache step stays the fallback, since a workflow only uses the path when it exists. A pruner DaemonSet (from the autoscaler image) empties a profile's whole cache when it passesgithub_runner_arc_node_cache_size_limit_gib, and only while no runner pod of that profile is running on the node; it never trims a cache, because a running job may be linked into any of it.The issue suggested getting hard links by putting the work directory on the same volume as the store. I tested that on a Linux host:
link(2)fails withEXDEVbetween two bind mounts of one directory and between a directory and a bind mount of one of its subdirectories (which is whatsubPathis), and only works inside a single mount. The runner's workspace path is fixed and a workspace shared by pods would collide on_tempand_actions, so the workspace cannot live on the shared mount. The design therefore expects a package manager that links with symlinks: with pnpm's global virtual store, a project'snode_modulesholds symlinks into<store>/v11/linksplus a few metadata files, and a second project installing the same dependencies writes no package files (checked with a real install against a shared store directory).The cache is shared writable state, so the capability is opt-in per profile and the README says not to give it to a profile that runs fork pull requests. Cluster Pod Security levels that refuse
hostPathwill refuse it too.