Repository navigation
Remove the binary archive cache - #1009
Merged
Merged
Conversation
Metal already caches native code per user, keyed on the library contents, and our metallibs are byte-stable across sessions. Replaying the 2,805 kernels of a full test suite run, linking takes 0.14 ms/kernel from a warm system cache, against 0.85 ms for an archive hit and 22.6 ms for an archive miss, whose harvest is a second back-end compile. Archives only win once the system cache has been evicted, which one full test suite run does, and even there they save under 1% of CPU time. Every harvest also makes Metal leave behind a gpuarchiver-* directory of a few hundred KB in the user cache, which has filled disks on CI and developer machines. Removing those safely while archives are in use takes a private method. Keep the MTLBinaryArchive wrappers, but drop the per-kernel cache built on them, along with its preferences and the Scratch dependency.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1009 +/- ##
==========================================
- Coverage 87.29% 87.16% -0.14%
==========================================
Files 92 91 -1
Lines 6816 6675 -141
==========================================
- Hits 5950 5818 -132
+ Misses 866 857 -9 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
Metal Benchmarks
Details
| Benchmark suite | Current: e45dee0 | Previous: c51cdc2 | Ratio |
|---|---|---|---|
array/accumulate/Float32/1d |
392042 ns |
393209 ns |
1.00 |
array/accumulate/Float32/dims=1 |
370083 ns |
370125 ns |
1.00 |
array/accumulate/Float32/dims=1L |
8859084 ns |
8891250 ns |
1.00 |
array/accumulate/Float32/dims=2 |
432333 ns |
437125 ns |
0.99 |
array/accumulate/Float32/dims=2L |
2791292 ns |
2721125 ns |
1.03 |
array/accumulate/Int64/1d |
847042 ns |
848000 ns |
1.00 |
array/accumulate/Int64/dims=1 |
909083 ns |
907708 ns |
1.00 |
array/accumulate/Int64/dims=1L |
9569583 ns |
9559042 ns |
1.00 |
array/accumulate/Int64/dims=2 |
1209750 ns |
1205958 ns |
1.00 |
array/accumulate/Int64/dims=2L |
6558000 ns |
6545792 ns |
1.00 |
array/broadcast |
234000 ns |
240208 ns |
0.97 |
array/construct |
2333 ns |
2375 ns |
0.98 |
array/permutedims/2d |
454958 ns |
458750 ns |
0.99 |
array/permutedims/3d |
1033125 ns |
1033750 ns |
1.00 |
array/permutedims/4d |
1020083 ns |
1130792 ns |
0.90 |
array/private/copy |
224708 ns |
224541 ns |
1.00 |
array/private/copyto!/cpu_to_gpu |
214875 ns |
214917 ns |
1.00 |
array/private/copyto!/gpu_to_cpu |
213958 ns |
216250 ns |
0.99 |
array/private/copyto!/gpu_to_gpu |
222708 ns |
220167 ns |
1.01 |
array/private/iteration/findall/bool |
1053625 ns |
1057500 ns |
1.00 |
array/private/iteration/findall/int |
1214500 ns |
1212709 ns |
1.00 |
array/private/iteration/findfirst/bool |
1011959 ns |
972583 ns |
1.04 |
array/private/iteration/findfirst/int |
1019750 ns |
1020583 ns |
1.00 |
array/private/iteration/findmin/1d |
1080000 ns |
1088625 ns |
0.99 |
array/private/iteration/findmin/2d |
1026250 ns |
1021375 ns |
1.00 |
array/private/iteration/logical |
1547625 ns |
1547084 ns |
1.00 |
array/private/iteration/scalar |
1398667 ns |
1403375 ns |
1.00 |
array/random/rand/Float32 |
420375 ns |
417583 ns |
1.01 |
array/random/rand/Int64 |
497500 ns |
500375 ns |
0.99 |
array/random/rand!/Float32 |
403542 ns |
403625 ns |
1.00 |
array/random/rand!/Int64 |
426250 ns |
426250 ns |
1 |
array/random/randn/Float32 |
391250 ns |
390584 ns |
1.00 |
array/random/randn!/Float32 |
376042 ns |
375375 ns |
1.00 |
array/reductions/mapreduce/Float32/1d |
273125 ns |
274667 ns |
0.99 |
array/reductions/mapreduce/Float32/dims=1 |
349959 ns |
354750 ns |
0.99 |
array/reductions/mapreduce/Float32/dims=1L |
614083 ns |
609625 ns |
1.01 |
array/reductions/mapreduce/Float32/dims=2 |
355917 ns |
354292 ns |
1.00 |
array/reductions/mapreduce/Float32/dims=2L |
1240000 ns |
1240917 ns |
1.00 |
array/reductions/mapreduce/Int64/1d |
459167 ns |
455583 ns |
1.01 |
array/reductions/mapreduce/Int64/dims=1 |
634792 ns |
635500 ns |
1.00 |
array/reductions/mapreduce/Int64/dims=1L |
1009750 ns |
1015750 ns |
0.99 |
array/reductions/mapreduce/Int64/dims=2 |
789500 ns |
787291 ns |
1.00 |
array/reductions/mapreduce/Int64/dims=2L |
2206666 ns |
2191209 ns |
1.01 |
array/reductions/reduce/Float32/1d |
273042 ns |
274084 ns |
1.00 |
array/reductions/reduce/Float32/dims=1 |
355083 ns |
355625 ns |
1.00 |
array/reductions/reduce/Float32/dims=1L |
622208 ns |
620375 ns |
1.00 |
array/reductions/reduce/Float32/dims=2 |
247459 ns |
251208 ns |
0.99 |
array/reductions/reduce/Float32/dims=2L |
488583 ns |
471792 ns |
1.04 |
array/reductions/reduce/Int64/1d |
460667 ns |
458333 ns |
1.01 |
array/reductions/reduce/Int64/dims=1 |
641042 ns |
641333 ns |
1.00 |
array/reductions/reduce/Int64/dims=1L |
1006792 ns |
1010167 ns |
1.00 |
array/reductions/reduce/Int64/dims=2 |
260375 ns |
257625 ns |
1.01 |
array/reductions/reduce/Int64/dims=2L |
659417 ns |
670834 ns |
0.98 |
array/shared/copy |
127500 ns |
130250 ns |
0.98 |
array/shared/copyto!/cpu_to_gpu |
37208 ns |
37291 ns |
1.00 |
array/shared/copyto!/gpu_to_cpu |
36459 ns |
37875 ns |
0.96 |
array/shared/copyto!/gpu_to_gpu |
36750 ns |
38209 ns |
0.96 |
array/shared/iteration/findall/bool |
1058084 ns |
1048458 ns |
1.01 |
array/shared/iteration/findall/int |
1221583 ns |
1223209 ns |
1.00 |
array/shared/iteration/findfirst/bool |
977333 ns |
976875 ns |
1.00 |
array/shared/iteration/findfirst/int |
988250 ns |
981250 ns |
1.01 |
array/shared/iteration/findmin/1d |
1049958 ns |
1050875 ns |
1.00 |
array/shared/iteration/findmin/2d |
1022666 ns |
1027416 ns |
1.00 |
array/shared/iteration/logical |
1512875 ns |
1525334 ns |
0.99 |
array/shared/iteration/scalar |
75.35040983606558 ns |
75.89014373716633 ns |
0.99 |
array/sorting/1d |
1916500 ns |
1921125 ns |
1.00 |
array/sorting/2d |
8270958 ns |
8319917 ns |
0.99 |
integration/byval/reference |
1120167 ns |
1120625 ns |
1.00 |
integration/byval/slices=1 |
1124208 ns |
1122291 ns |
1.00 |
integration/byval/slices=2 |
2028334 ns |
2029542 ns |
1.00 |
integration/byval/slices=3 |
6821750 ns |
6806458 ns |
1.00 |
integration/metaldevrt |
396875 ns |
396167 ns |
1.00 |
kernel/indexing |
213625 ns |
207625 ns |
1.03 |
kernel/indexing_checked |
395458 ns |
395208 ns |
1.00 |
kernel/launch |
2115.777777777778 ns |
2101.8888888888887 ns |
1.01 |
kernel/rand |
318875 ns |
408958 ns |
0.78 |
latency/import |
1609990875 ns |
1862270958 ns |
0.86 |
latency/precompile |
30858224084 ns |
31631620958 ns |
0.98 |
latency/ttfp |
2067661667 ns |
2295776542 ns |
0.90 |
metal/synchronization/context |
520.0680628272252 ns |
525.958115183246 ns |
0.99 |
metal/synchronization/stream |
627.9239766081871 ns |
657.375796178344 ns |
0.96 |
This comment was automatically generated by workflow using github-action-benchmark.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1004. Rather than cleaning up after Metal's
gpuarchiver-*leak with a private method, this removes the per-kernelMTLBinaryArchivecache from #930. Metal's own per-user shader cache ($(getconf DARWIN_USER_CACHE_DIR)com.apple.metal) is keyed on the library contents, and our metallibs are byte-stable across sessions, so that cache already handles the common case, and does so faster.I measured this on the full test suite (
--all, M1, macOS 27). I dumped every metallib it linked (2,805 kernels), then replayed only the back-end link, each case in a fresh process:A harvest isn't served from the system cache, so every archive miss costs a second full compile, plus a leaked directory of ~350 KB. The system cache does evict: it keeps about two generations of ~280 MB, and one full test-suite run cycles through them. That is the only case where archives help, and even there they save ~50 s of CPU on a 116 CPU-minute run, which matches the absence of a CI speedup since #930. Keeping archives as an opt-in would mean maintaining ~600 lines of cache and tests that hardly anyone would enable.
The
MTLBinaryArchivewrappers inlib/mtlstay. Thebinary_archivesandbinary_archives_max_sizepreferences, theJULIA_METAL_BINARY_ARCHIVES*environment variables, theversioninfo()kernel-cache section and the Scratch dependency go.Leftovers from earlier releases are not cleaned up automatically:
rm -rf "$(getconf DARWIN_USER_CACHE_DIR)com.apple.gpuarchiver"/gpuarchiver-*(while no Julia session is running);rm -rf ~/.julia/scratchspaces/dde4c033-4e86-420c-a63e-0dd931031962/binary_archives.