release scripts: support reproducible builds via SOURCE_DATE_EPOCH

[PtJade-Ceramic]

Add reproducible-build support: when SOURCE_DATE_EPOCH is set, the release scripts pin file/directory mtimes and the 7z/ZIP paths use -mta- -mtc- so the installer, portable, MinGit and tar archives are byte-identical across repeated builds on the same SDK snapshot. This is a no-op when SOURCE_DATE_EPOCH is unset.

[dscho]

I fear that this is a lot of code (which would need to be simplified before it could be merged, anyway) for an incomplete solution.

The real problem is of course that it depends on much more than just the build-extra repository which bits and parts are combined into a full Git from Windows release. Specifically a full Git for Windows release is usually done starting with a specific commit in the git-sdk-* repositories, and then building Git and then installing it into that checkout. So there's really no single commit from which this is built. And therefore it would really be a challenge to reproduce any release, and setting the epoch here is just maybe one percent of the entire effort that would be required.

I don't think reproducible builds are feasible in Git from Windows.

[PtJade-Ceramic]

I fear that this is a lot of code (which would need to be simplified before it could be merged, anyway) for an incomplete solution.

The real problem is of course that it depends on much more than just the build-extra repository which bits and parts are combined into a full Git from Windows release. Specifically a full Git for Windows release is usually done starting with a specific commit in the git-sdk-* repositories, and then building Git and then installing it into that checkout. So there's really no single commit from which this is built. And therefore it would really be a challenge to reproduce any release, and setting the epoch here is just maybe one percent of the entire effort that would be required.

I don't think reproducible builds are feasible in Git from Windows.

Thanks for the review -- both points are fair.

I agree that this cannot, on its own, make a full Git for Windows release reproducible: a release is assembled from a specific git-sdk-* snapshot plus a freshly built Git, and there is no single commit that pins all of that down. Setting SOURCE_DATE_EPOCH in the packaging scripts is indeed only a small slice of the effort.

The scope I am aiming for is deliberately narrower: given one and the same SDK snapshot, the packaging step itself should be deterministic. Building the installer, portable, MinGit or tar archive twice on that snapshot should yield byte-identical output instead of silently embedding the build time. That property is verifiable on its own and useful for debugging -- when artifact bytes differ, it is no longer an unactionable mystery -- but it makes no claim about reproducing a release from scratch.

On the code-size concern: the per-script ~37-line blocks are now a single shared helper (pin-mtimes.sh, ~70 lines including comments), the duplicated hardcoded source list in installer/release.sh is gone, and the parent-directory walk is deduplicated. While reworking this I also found that the installer path had never actually pinned anything: the Perl one-liner's print (x), "\n" form drops the trailing newline, so the whole file list collapsed into one line and every lookup failed. With that fixed, the installer reproducibility claim is now actually exercised, not just the archive paths.

[dscho]

I am afraid that I still do not follow how this would be useful for the Git for Windows project. In the official release process, the proposed premise "building from the same snapshot" simply does not exist.

I understand that you want this. Yet I do not understand why the Git for Windows project should accept a complex (and, frankly a bit fragile, given that building the Git executables and code-signing them with a current timestamp is a crucial part of the release process) and substantial amount of code (the added pin-mtimes.sh is almost half the size of mingit/release.sh, which itself is not exactly light on -- necessary! -- complexity), when Git for Windows itself won't benefit from it. It sounds like a net negative on the project to me: carry more, and complex, and fragile code, for reasons that lie outside the project and its direct users.

In other words, from where I sit the most plausible course of action would be to keep this code in your fork, and leave it out of this here repository.

Also, I wonder whether you have considered extracting the mtimes from an existing installer/archive you wish to reproduce and forcing the respective files' mtimes in your git-sdk-* checkout to those values? That should achieve the same reproducibility goal without burdening build-extra with code that does not benefit the Git for Windows project.