Skip writing unchanged generated files to preserve mtimes#4396
Open
iainmcgin wants to merge 1 commit intobufbuild:mainfrom
Open
Skip writing unchanged generated files to preserve mtimes#4396iainmcgin wants to merge 1 commit intobufbuild:mainfrom
iainmcgin wants to merge 1 commit intobufbuild:mainfrom
Conversation
buf generate previously wrote output files unconditionally, touching mtimes even when content was identical to what was already on disk. For mtime-based build systems (cargo, make, ninja), a no-op generate would cascade into unnecessary downstream rebuilds. This change replaces the unconditional storage.Copy in the directory output path with a compare-then-write loop. Files whose content matches what is already on disk are skipped, preserving their mtimes. Read errors during comparison (e.g. file does not exist, permissions) fall through to write, preserving existing behavior. Scope is limited to directory output; zip/jar output continues to write unconditionally. Interaction with --clean is already correct: clean deletes first, so comparison finds nothing and writes everything. Closes bufbuild#3126
emcfarlane
approved these changes
Mar 19, 2026
Contributor
There was a problem hiding this comment.
Thanks for this contribution! The skip-unchanged behavior is a good improvement.
However, this doesn't fully close #3126. That issue is about making --clean a per-plugin "smart" clean, including stale file deletion. This PR is a useful step toward that, but the core of the issue remains open.
Could you update the PR description to say "Contributes to #3126" instead of "Closes #3126" so the issue stays open?
|
|
||
| - Add support for `--rbs_out` as a `protoc_builtin` plugin (requires protoc v34.0+). | ||
| - Add relevant links from CEL LSP hover documentation to either <celbyexample.com> or <protovalidate.com> | ||
| - `buf generate` now skips writing output files when the content matches what's already on disk, preserving modification times for mtime-based build systems like cargo and make. |
Contributor
There was a problem hiding this comment.
Suggested change
| - `buf generate` now skips writing output files when the content matches what's already on disk, preserving modification times for mtime-based build systems like cargo and make. | |
| - Skip writing unchanged output files in `buf generate` to preserve modification times |
doriable
approved these changes
Mar 19, 2026
Member
doriable
left a comment
There was a problem hiding this comment.
Looks good to me, thanks for putting this up! +1 to Ed's comments on fixing the PR description.
Author
|
Changed to "contributes towards" as requested |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Contributes towards #3126.
Summary
buf generatepreviously wrote output files unconditionally, touching mtimes even when content was identical to what was already on disk. For mtime-based build systems (cargo, make, ninja, file-watching dev servers), a no-opbuf generatewould cascade into unnecessary downstream rebuilds.The protoc plugin protocol gives plugins no visibility into the output directory—they emit
File{name, content}on stdout and the invoking tool handles filesystem writes—so this fix lives in buf's write path rather than in individual plugins.Approach
Replaces the unconditional
storage.Copyin the directory output path (bufprotopluginos.writeDirectory) with a compare-then-write loop. For each generated file:Read errors during comparison (file doesn't exist, permissions, etc.) fall through to write, so worst-case behavior is identical to before.
Scope
--cleaninteraction is already correct. Clean deletes first, so comparison finds nothing and writes everything.Testing
response_writer_test.gocovering unchanged/changed/new/mixed file scenariosgenerate_test.gothat runsbuf generatetwice and verifies the output mtime is preserved