Skip to content

Upload download_file outputs to the CAS; closes #1571 - #1572

Open
vlad-ivanov-name wants to merge 1 commit into
meta:mainfrom
vlad-ivanov-name:vlad/upload-downloads-to-cas
Open

vlad-ivanov-name wants to merge 1 commit into
meta:mainfrom
vlad-ivanov-name:vlad/upload-downloads-to-cas

Conversation

@vlad-ivanov-name

@vlad-ivanov-name vlad-ivanov-name commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

What

  • Add [buck2] upload_downloads_to_cas (bool, default true).
  • After download_file fetched a file with a checksum of the CAS digest algorithm, upload the file to the CAS.
  • --no-remote-cache skips the upload unless --write-to-cache-anyway is set.
  • A failed upload is a quiet soft error and never fails the download.
  • Docs paragraph and one e2e test.

Why

download_file already probes the CAS for the declared checksum before it fetches the origin, but nothing stored a downloaded file there. The probe only hit when a remote action had used the file as an input. With the upload, every later machine and remote worker gets the file from the CAS instead of the origin server.

Notes

  • Only a real download uploads. The path where the file is already on disk with the right content does not upload, and downloads without a checksum of the CAS algorithm do not upload.
  • Size 0 blobs are not uploaded; every CAS holds the empty blob.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 14, 2026
@meta-codesync

meta-codesync Bot commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

This pull request has been imported. If you are a Meta employee, you can view this in D125193316. (Because this pull request was imported automatically, there will not be any future comments.)

@vlad-ivanov-name
vlad-ivanov-name marked this pull request as ready for review October 14, 2026 12:49
@vlad-ivanov-name
vlad-ivanov-name force-pushed the vlad/upload-downloads-to-cas branch from 766a646 to 5806f06 Compare November 3, 2026 20:05
What:
- Add `[buck2] upload_downloads_to_cas` (default true).
- After a download with a checksum of the CAS digest algorithm, upload the file to the CAS.
- Skip the upload with `--no-remote-cache` unless `--write-to-cache-anyway` is set.
- A failed upload is a quiet soft error.

Why:
- `download_file` probes the CAS for the declared checksum, but nothing stored a downloaded file there.
- With the upload, other machines and remote workers get the file from the CAS instead of the origin.
@vlad-ivanov-name
vlad-ivanov-name force-pushed the vlad/upload-downloads-to-cas branch from 5806f06 to 13c11d8 Compare November 4, 2026 21:17

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant