Skip to content

Support new-style digest functions - #18731

Closed
tylerwilliams wants to merge 1 commit into
bazelbuild:masterfrom
tylerwilliams:b3_remote_exec
Closed

Support new-style digest functions#18731
tylerwilliams wants to merge 1 commit into
bazelbuild:masterfrom
tylerwilliams:b3_remote_exec

Conversation

@tylerwilliams

@tylerwilliams tylerwilliams commented Jun 21, 2023

Copy link
Copy Markdown
Contributor

Support new-style digest functions.

This PR adds support for new-style digest functions to the remote execution
library code. The remote-apis spec says:

// * `digest_function` is a lowercase string form of a `DigestFunction.Value`
//   enum, indicating which digest function was used to compute `hash`. If the
//   digest function used is one of MD5, MURMUR3, SHA1, SHA256, SHA384, SHA512,
//   or VSO, this component MUST be omitted. In that case the server SHOULD
//   infer the digest function using the length of the `hash` and the digest
//   functions announced in the server's capabilities.

This is a partial commit for #18658.

@tylerwilliams
tylerwilliams marked this pull request as ready for review June 21, 2023 05:35
@tylerwilliams
tylerwilliams requested a review from a team as a code owner June 21, 2023 05:35
@github-actions github-actions Bot added awaiting-review PR is awaiting review from an assigned reviewer team-Remote-Exec Issues and PRs for the Execution (Remote) team labels Jun 21, 2023
coeuvre
coeuvre requested changes Jun 21, 2023
View reviewed changes

@coeuvre coeuvre left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we please add a test case for URL of the new style digest function?

@tylerwilliams
tylerwilliams force-pushed the b3_remote_exec branch 2 times, most recently from 65db605 to 10f1965 Compare June 21, 2023 22:03
@tylerwilliams

tylerwilliams commented Jun 21, 2023 •
edited by brentleyjones
Loading

Copy link
Copy Markdown
Contributor Author

Can we please add a test case for URL of the new style digest function?

Done.

coeuvre
coeuvre reviewed Jun 22, 2023
View reviewed changes
Comment thread src/main/java/com/google/devtools/build/lib/remote/ByteStreamUploader.java
return digestFunction.getNumber() <= 7;
}

private String buildUploadResourceName(

@coeuvre coeuvre Jun 22, 2023 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may also want to change GrpcCacheClient#getResourceName, maybe in another PR.

@tylerwilliams tylerwilliams Jun 22, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may also want to change GrpcCacheClient#getResourceName, maybe in another PR.

Yeah it's a bit cyclical; I'm happy to add support for it here so that download and upload are both kept in sync -- which I've done.

Adding another test for the download path will be simple once there is a DigestHashFunction for BLAKE3. Will send that in a followup after adding the hasher.

@coeuvre coeuvre added awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally and removed awaiting-review PR is awaiting review from an assigned reviewer labels Jun 22, 2023
@tylerwilliams tylerwilliams mentioned this pull request Jun 26, 2023
Closed
@tylerwilliams

tylerwilliams commented Jun 27, 2023

Copy link
Copy Markdown
Contributor Author

@coeuvre friendly ping on this -- do I need to do anything more to get this merged? Thanks!

@coeuvre

coeuvre commented Jun 27, 2023

Copy link
Copy Markdown
Member

It has been imported but is still under review internally, should be merged soon. No actions required on your side! Sorry for the delay.

@github-actions github-actions Bot removed the awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally label Jun 27, 2023
@brentleyjones
brentleyjones deleted the b3_remote_exec branch June 27, 2023 12:47
@brentleyjones brentleyjones mentioned this pull request Aug 7, 2023
Merged
@iancha1992

iancha1992 commented Sep 21, 2023

Copy link
Copy Markdown
Member

The changes in this PR have been included in Bazel 6.4.0 RC1. Please test out the release candidate and report any issues as soon as possible. If you're using Bazelisk, you can point to the latest RC by setting USE_BAZEL_VERSION=last_rc.
Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@coeuvre coeuvre coeuvre approved these changes

Assignees

No one assigned

Labels

team-Remote-Exec Issues and PRs for the Execution (Remote) team

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@tylerwilliams @coeuvre @iancha1992