feat: Add disk cache garbage collection to setup-bazel-cache#10
Conversation
The caches saved by setup-bazel-cache can at the moment grow without limits. This puts a limit to the disk cache.
|
This limits the size of bazels disk cache, but it has to be somehow known the maximum size of disk cache. I got the value by getting rid of all caches and doing a clean build of communication using the host build, which resulted in 3.6GB. I increased it to 5GB as safety margin. But there is no guarantee that this value is the right one for other jobs or modules, which may work much better with smaller or bigger cache sizes. Another problem is that the repository cache can still grow beyond infinity. Back in the old days I dealt with similar situations by wiping the cache and all build artifacts and then rebuilding everything to get a newly populated cache. This was done using the nightly job. With that approach we could live with the situation that caches can grow a lot, because they will be gotten rid of in the nightly anyway. Logic would be as follows:
During the day the jobs use, write and delete caches as they would already do with this action. The nightly should only be triggered, if there has been activity on the repository during the day. Otherwise it might waste CI resources. |
|
I have now setup a nightly which wipes the caches and rebuilds them: It build on You can also see it in my test repo: If that works, I would remove the garbage collector settings. At least for inc_someip_gateway the repository cache is way bigger than the disk cache and we should not rely on an option marked experimental. Instead we should keep an eye that the workflow name is not part of the cache name. So far it looks good to me. In my test repo the caches are recreated as they should. In the S-CORE that is not happening because caches are only stored if the action runs on the main branch. |
|
Discussion continues at https://github.com/orgs/eclipse-score/discussions/3016#discussioncomment-17431750 |
The caches saved by setup-bazel-cache can at the moment grow without limits. This puts a limit to the disk cache.
Corresponding documentation: https://bazel.build/reference/command-line-reference#common_options-flag--experimental_disk_cache_gc_idle_delay