[WIP] feat: unstable SHA256 support by weihanglo · Pull Request #1206 · rust-lang/git2-rs

weihanglo · 2026-01-12T23:44:54Z

This adds an unstable-sha256 Cargo feature, as a follow-up of #1201.
Also adds some smoke tests for affected operations/types.

Part of #1090

Insta-stable

NEW Index::with_object_format to create with different format
NEW Oid::object_format to access underlying oid type

Behind `unstable-sha256`

NEW ObjectFormat::Sha256 enum variant
NEW RepositoryInitOptions::object_format() method to set hash algo
NEW Remote::object_format method to get hash algo on a remote
NEW Oid::raw_bytes to return the full underlying bytes
CHANGED Diff::from_buffer to accept an extra object format argument
CHANGED Index::open to accept an extra object format argument
CHANGED Indexer::new to accept an extra object format argument
CHANGED Oid::from_str to accept an extra object format argument
CHANGED Oid::hash_{object,file} to accept an extra object format argument
CHANGED Oid::as_bytes to return the logic bytes (SHA1 -> 20 bytes; SHA256 -> 32 bytes)
CHANGED impl Hash for Oid to hash git_oid->kind
REMOVED Index::new to avoid misuse.
REMOVED impl std::FromStr for Oid to avoid misuse
REMOVED impl std::FromStr for Oid to avoid misuse

libgit2 1.9 compatibility changes

This PR also includes fixes for libgit2 1.9 breaking changes:

Renamed build macros (GIT_SHA1_COLLISIONDETECT -> GIT_SHA1_BUILTIN, HTTPS backend macros)
Added refdb_type field to git_repository_init_options
Updating sha256 bindings accordingly

See libgit2/libgit2#6994 and libgit2/libgit2#7102 for upstream changes.

These changes can be split out if needed.

jorendorff · 2026-02-24T22:44:28Z

+    /// View this OID as a byte-slice.
+    ///
+    /// * 20 bytes in length if the feature `unstable-sha256` is not enabled.
+    /// * 32 bytes in length if the feature `unstable-sha256` is enabled.
    pub fn as_bytes(&self) -> &[u8] {
        &self.raw.id
    }


I think this should return 20 bytes if unstable-sha256 is enabled but the object format is SHA-1.

One reason for this is that crate features should be additive; turning on the feature shouldn't unnecessarily affect how the crate behaves with already-supported repositories.

Thanks for calling this out. That's a fair point, but I'm not sure "feature additivity" is the core issue here.

I think the real question is what Oid::as_bytes() is supposed to mean in this crate: the logical object ID bytes, or the raw backing storage from libgit2.

I'm leaning toward the logical interpretation. That seems like the less surprising API for callers, as you mentioned.

The reason I wasn’t fully convinced yet is that this type has historically been a fairly thin wrapper over libgit2, so I wanted to check whether anyone might rely on the raw representation or the backing buffer. Or maybe we should just mark this with a big ‼️ in the changelog, so we can transit to your proposal. Do you know any use of this functions in the wild?

There is the option to have Oid::as_bytes() give you the logical representation and introduce a new Oid::raw_bytes() for the (probably few) users that actually want the entire buffer.

Addressed the as_bytes part and also added the raw_bytes method. Thanks for the feedback, folks!

* GIT_SHA1_COLLISIONDETECT -> GIT_SHA1_BUILTIN * GIT_WINHTTP -> GIT_HTTPS_WINHTTP * GIT_SECURE_TRANSPORT -> GIT_HTTPS_SECURETRANSPORT * GIT_OPENSSL -> GIT_HTTPS_OPENSSL See <libgit2/libgit2#6994>

This was added in <libgit2/libgit2#7102>

* Add `GIT_REPOSITORY_INIT_RELATIVE_GITLINK` at libgit2/libgit2@bc737620d * Remove `GIT_REPOSITORY_INIT_NO_DOTGIT_DIR` at libgit2/libgit2@ca2a241e4

* Remove `GIT_OBJECT_OFS_DELTA` * Remove `GIT_OBJECT_REF_DELTA` See libgit2/libgit2@23da3a8f3

libgit2 simplified SHA256 API to use `_ext` suffixes instead of changing base function signatures. This avoids breaking changes for users who don't need SHA256 support. Updated bindings: - git_oid_from_raw - git_oid_from_prefix - git_oid_from_string - git_diff_from_buffer_ext - git_index_new_ext - git_index_open_ext - git_odb_new_ext - git_odb_open_ext - git_repository_new_ext See libgit2/libgit2@56e2a85643f (libgit2/libgit2#6975)

This adds an `unstable-sha256` Cargo feature, as a follow-up of rust-lang#1201 Also adds some smoke tests for affected operations/types. ## Insta-stable - **NEW** `Index::with_object_format` to create with different format - **NEW** `Oid::object_format` to access underlying oid type ## Behind `unstable-sha256` - **NEW** `ObjectFormat::Sha256` enum variant - **NEW** `RepositoryInitOptions::object_format()` method to set hash algo - **NEW** `Remote::object_format` method to get hash algo on a remote - **NEW** `Oid::raw_bytes` to return the full underlying bytes - **CHANGED** `Diff::from_buffer` to accept an extra object format argument - **CHANGED** `Index::open` to accept an extra object format argument - **CHANGED** `Indexer::new` to accept an extra object format argument - **CHANGED** `Oid::from_str` to accept an extra object format argument - **CHANGED** `Oid::hash_{object,file}` to accept an extra object format argument - **CHANGED** `Oid::as_bytes` to return the logic bytes (SHA1 -> 20 bytes; SHA256 -> 32 bytes) - **CHANGED** `impl Hash for Oid` to hash `git_oid->kind` - **REMOVED** `Index::new` to avoid misuse. - **REMOVED** `impl std::FromStr for Oid` to avoid misuse

This also run systest first, so we can dicover bindding mismatch before

lorenzleutgeb · 2026-04-25T18:24:51Z

I believe to have found something that you missed in this PR. Note that git2::util::zeroed_raw_oid does not take any arguments:

git2-rs/src/util.rs

Lines 271 to 275 in e6919b7

    
           /// Creates a zeroed git_oid structure. 
        
           #[inline] 
        
           pub(crate) fn zeroed_raw_oid() -> raw::git_oid { 
        
               unsafe { std::mem::zeroed() } 
        
           }

I believe that this means that the kind: c_uchar (where c_uchar is defined as u8) member of the returned libgit2_sys::git_oid will have the value 0u8.

The function git2::util::zeroed_raw_oid is called by git2::Oid::zero, which also takes no arguments. If the function git2::Oid::object_format is called on the result of git2::Oid::zero, the call

git2-rs/src/oid.rs

Line 231 in e6919b7

unsafe { Binding::from_raw(self.raw.kind as raw::git_oid_t) }

will hit the panic! at

git2-rs/src/oid.rs

Line 29 in e6919b7

_ => panic!("Unknown git oid type"),

This is because 0u8 as u32 (which we got from std::mem::zeroed) is neither GIT_OID_SHA1 = 1, nor GIT_OID_SHA256 = 2.

I discovered this experimenting in code that is downstream of git2, building against this PR. In one of my tests I did essentially git2::Oid::zero().object_format().

Then I started to think about how much unsafe really is necessary here. Maybe zeroed_raw_oid could look like

#[inline]
pub(crate) fn zeroed_raw_oid(format: ObjectFormat) -> raw::git_oid {
  raw::git_oid {
    kind: format.raw(),
    id: [0; raw::GIT_OID_MAX_SIZE],
  }
}

In fact one might even ask why not to have

impl Oid {
  pub const ZERO_SHA1: Self = Self {
    raw: raw::git_oid {
      kind: raw::GIT_OID_SHA1,
      id: [0; raw::GIT_OID_MAX_SIZE],
    }
  };
  
  pub const ZERO_SHA256: Self = Self {
    raw: raw::git_oid {
      kind: raw::GIT_OID_SHA256,
      id: [0; raw::GIT_OID_MAX_SIZE],
    }
  };
  
  // NOTE: Maybe call this `zero_with_object_format` to not break `zero`.
  pub fn zero(format: ObjectFormat) -> Self {
    // TODO: Match on `format` and return `Oid::ZERO_SHA1` or `Oid::ZERO_SHA256`.
  }
}

This way, users that know at compile time which kind of zero they want can just use the constant, while those that decide at run time would call zero. And there is no unsafe involved.

If you do not want git2::Oid::zero to take an argument for backwards compatibility, then you could have it return a SHA-1 object identifier always, and mark it deprecated in favor of these constants and a new zero_with_object_format (if I have not missed a reason why the constants cannot be used).

For oid.rs I also had the gut feeling that some unsafe could be removed, but I do not have any constructive comments here, sorry.

lorenzleutgeb · 2026-04-25T18:47:18Z

Another comment regarding FromStr: I understand you opted to remove it to "avoid misuse", but I would argue that having FromStr actually is quite convenient. Instead of removing it, you could consider implementing FromStr in a way that it checks the length of the argument and then calls into the more specialized function that takes a string and an object format. The caller can then still inspect the object format of the result. You leave all options open for your users, and keep backwards compatibility.

DanielEScherzer

just some thoughts, not going to pretend that I understand this fully

View changes since this review

DanielEScherzer · 2026-04-27T22:24:15Z

+    /// * [`Repository::index`] if you have repository context
+    ///
+    /// </div>
+    #[cfg(not(feature = "unstable-sha256"))]


why is this function removed when sha256 support is enabled? If the idea is that we should migrate to with_object_format() anyway, maybe add that function in a separate PR that doesn't need to wait for SHA-256 to be ready?

DanielEScherzer · 2026-04-27T22:28:00Z

+    ///
+    /// Note: Most users should obtain an ODB from [`Repository::odb`](crate::Repository::odb),
+    /// which automatically inherits the repository's object format.
+    #[cfg(feature = "unstable-sha256")]


this can match Index and introduce the new method without the feature gate, so that code can start to use it without requiring the unstable feature

DanielEScherzer · 2026-04-27T22:28:45Z

+    /// characters (or 64 for SHA256), or contains any non-hex characters.
+    pub fn from_str(
+        s: &str,
+        #[cfg(feature = "unstable-sha256")] format: crate::ObjectFormat,


this parameter could also be ungated so that callers can start passing it

DanielEScherzer · 2026-04-27T22:33:42Z

+        #[cfg(not(feature = "unstable-sha256"))]
+        {
+            if bytes.len() != raw::GIT_OID_SHA1_SIZE {
+                return Err(Error::from_str("raw byte array must be 20 bytes"));


maybe a clearer error for trying to accidentally use 32 bytes when the feature isn't enabled?

Also, why have different ways of creating the SHA1 oid depending on if the SHA256 feature is available?

let oid_type = match bytes.len() { raw::GIT_OID_SHA1_SIZE => raw::GIT_OID_SHA1, #[cfg(feature = "unstable-sha256")] raw::GIT_OID_SHA256_SIZE => raw::GIT_OID_SHA256, #[cfg(not(feature = "unstable-sha256"))] raw::GIT_OID_SHA256_SIZE => { return Err(Error::from_str( "raw byte array of 32 bytes requires unstable-sha256 feature to be enabled", )) } _ => { return Err(Error::from_str( "raw byte array must be 20 bytes (SHA1) or 32 bytes (SHA256, if enabled)", )) } }; unsafe { try_call!(raw::git_oid_from_raw(&mut raw, bytes.as_ptr(), oid_type)); } Ok(Oid { raw })

DanielEScherzer · 2026-04-27T22:34:06Z

+    pub fn hash_object(
+        kind: ObjectType,
+        bytes: &[u8],
+        #[cfg(feature = "unstable-sha256")] format: crate::ObjectFormat,


another parameter that can already be added immediately - not going to call them out each time though

DanielEScherzer · 2026-04-27T22:37:08Z

+        }
+        #[cfg(not(feature = "unstable-sha256"))]
+        {
+            let _ = oid_type;


hmm, even if the git2-rs bindings don't have SHA256 enabled, if the oid_type corresponds to SHA256 we should say that and probably return an error, rather than just claiming that the repository is formatted with SHA1?

weihanglo force-pushed the sha256-git2 branch 7 times, most recently from dd35da4 to a332c64 Compare January 13, 2026 18:53

weihanglo force-pushed the sha256-git2 branch from a332c64 to fd975c7 Compare February 10, 2026 06:46

bentenjamin mentioned this pull request Feb 16, 2026

[Feature]: Add support for SHA256 object format ViscousPot/GitSync#360

Open

jorendorff reviewed Feb 24, 2026

View reviewed changes

weihanglo force-pushed the sha256-git2 branch from fd975c7 to 23c005f Compare March 16, 2026 14:24

weihanglo mentioned this pull request Mar 16, 2026

SHA-256 #1090

Open

weihanglo force-pushed the sha256-git2 branch from 23c005f to fc6f290 Compare March 16, 2026 15:27

weihanglo force-pushed the sha256-git2 branch from fc6f290 to c590157 Compare April 5, 2026 13:21

weihanglo force-pushed the sha256-git2 branch 2 times, most recently from c07b123 to 5c04d59 Compare April 16, 2026 12:29

This was referenced Apr 19, 2026

fix(clone): propagate object format in local clone libgit2/libgit2#7226

Merged

Support SHA256 Git repositories rust-lang/cargo#14942

Open

weihanglo added 10 commits April 24, 2026 09:04

chore(sys): bump libgit2 to 1e7d7c07d

06d4e05

fix(sys): update macro names

6d155ea

* GIT_SHA1_COLLISIONDETECT -> GIT_SHA1_BUILTIN * GIT_WINHTTP -> GIT_HTTPS_WINHTTP * GIT_SECURE_TRANSPORT -> GIT_HTTPS_SECURETRANSPORT * GIT_OPENSSL -> GIT_HTTPS_OPENSSL See <libgit2/libgit2#6994>

feat(sys): add refdb_type field to git_repository_init_options

36ca0e8

This was added in <libgit2/libgit2#7102>

fix(sys): repo init new and old options

e1ac9e9

* Add `GIT_REPOSITORY_INIT_RELATIVE_GITLINK` at libgit2/libgit2@bc737620d * Remove `GIT_REPOSITORY_INIT_NO_DOTGIT_DIR` at libgit2/libgit2@ca2a241e4

fix(sys): remove OFS_DELTA and REF_DELTA values

36d1fcd

* Remove `GIT_OBJECT_OFS_DELTA` * Remove `GIT_OBJECT_REF_DELTA` See libgit2/libgit2@23da3a8f3

feat(sys): add git_remote_oid_type binding

292cfa0

feat: extend init example with --object-format option

7fa9c48

chore(ci): test git2 sha256 support

e6919b7

This also run systest first, so we can dicover bindding mismatch before

weihanglo force-pushed the sha256-git2 branch from 5c04d59 to e6919b7 Compare April 24, 2026 13:05

DanielEScherzer reviewed Apr 27, 2026

View reviewed changes

Conversation

weihanglo commented Jan 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Insta-stable

Behind unstable-sha256

libgit2 1.9 compatibility changes

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lorenzleutgeb commented Apr 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lorenzleutgeb commented Apr 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DanielEScherzer left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

weihanglo commented Jan 12, 2026 •

edited

Loading

Behind `unstable-sha256`

lorenzleutgeb commented Apr 25, 2026 •

edited

Loading

lorenzleutgeb commented Apr 25, 2026 •

edited

Loading

DanielEScherzer left a comment •

edited by rustbot

Loading