Add Gradle-level verification of dependency SHAs - #22859
Conversation
PR Code Analyzer ❗AI-powered 'Code-Diff-Analyzer' found issues on commit 680a462. ⛔ Hard block: Issues at Medium severity or above will block this PR from merging.
The table above displays the top 10 most important findings. Pull Requests Author(s): Please update your Pull Request according to the report above. Repository Maintainer(s): You can Thanks. |
f91d2d3 to
2d25cca
Compare
This provides an extra level of defense against supply-chain attacks, as it verifies SHAs of buildSrc dependencies, by using a built-in feature provided by Gradle. Signed-off-by: Michael Froh <m_froh@apple.com>
Signed-off-by: Michael Froh <m_froh@apple.com>
2d25cca to
f9638a8
Compare
These artifacts will need to be updated on every release, but that is not a huge burden. Signed-off-by: Michael Froh <m_froh@apple.com>
msfroh
added
the
skip-diff-analyzer
PR Reviewer Guide 🔍(Review updated until commit 02e0157)Here are some key observations to aid the review process:
|
|
❌ Gradle check result for 680a462: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Michael Froh <m_froh@apple.com>
|
Persistent review updated to latest commit b12e5bb |
These are pulled in as part of :server:detachedConfiguration1, so they're not easy to ask Gradle to emit. I ended up just adding them by hand. Signed-off-by: Michael Froh <m_froh@apple.com>
|
Persistent review updated to latest commit 3cb37ef |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #22859 +/- ##
============================================
+ Coverage 71.53% 71.59% +0.05%
- Complexity 77249 77322 +73
============================================
Files 6170 6170
Lines 359710 359710
Branches 52460 52460
============================================
+ Hits 257318 257520 +202
+ Misses 81965 81805 -160
+ Partials 20427 20385 -42 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Signed-off-by: Michael Froh <m_froh@apple.com>
|
Persistent review updated to latest commit 7c6d484 |
|
❌ Gradle check result for 7c6d484: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Michael Froh <m_froh@apple.com>
|
Persistent review updated to latest commit 02e0157 |
| ./gradlew <task that resolves the changed dependency> -Dorg.gradle.dependency.verification=off | ||
| ./gradlew --write-verification-metadata sha256 <same task> --offline | ||
| ``` |
There was a problem hiding this comment.
This two-step process to update dependencies can save a lot of time, since the default "online" mode for --write-verification-metadata seems to read from the various artifactories for every single task (as I understand it).
The :qa:os subprojects iterate through every listed OS and every version of OpenSearch defined in Version.java and fetches Jackson + JUnit dependencies. On my laptop, the online mode takes over 6 minutes (though I've seen it go as high as 9 minutes), while the offline mode is closer to 1 minute.
Removing the :qa:os subprojects and removing most of the unused constants in Version.java cuts the online metadata regeneration time to 1m30s, which is much closer to our old updateSHAs task.
The two-step process exists as a workaround until then, but it didn't work for me with -Dsandbox-enabled=true, because it insisted that arrow-compression-18.1.0.jar was not cached for offline verification (though I could clearly see it in my Gradle cache).
So, to summarize:
- If you're updating dependencies, not in sandbox, this two step process works and is fast.
- If you're updating sandbox dependencies, you need to do the online regeneration with something like
./gradlew --write-verification-metadata sha256 precommit -Denable.sandbox=true. Go grab a coffee. - If we remove the
:qa:ostasks and remove some cruft fromVersion.java, then this whole section can be a lot shorter and just document the online regeneration step (since it will be decently fast).
2 participants
Description
This provides an extra level of defense against supply-chain attacks, as it verifies SHAs of buildSrc dependencies, by using a built-in feature provided by Gradle.
When I opened #22770, I was kind of surprised that I didn't have to update/add the SHA for apache-rat.
Related Issues
N/A
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.