fix: gate ErlNifResourceTypeInit members/dyncall on nif_version_2_16 to fix OTP 22/23 heap overflow

[lmth]

Summary

Gates ErlNifResourceTypeInit.members and ErlNifResourceTypeInit.dyncall on #[cfg(feature = "nif_version_2_16")] to fix a heap overflow when running on OTP 22/23.

Root cause

On NIF API versions < 2.16 (OTP 22 and 23), ErlNifResourceTypeInit only has three fields:

// OTP 22/23 -- sizeof = 24 bytes
typedef struct {
    ErlNifResourceDtor* dtor;
    ErlNifResourceStop* stop;
    ErlNifResourceDown* down;
} ErlNifResourceTypeInit;

The fields members and dyncall were added in NIF 2.16 / OTP 24. Since PR #358 (May 2021), Rustler has included both fields in the Rust struct unconditionally, making it 40 bytes regardless of the active NIF version.

When enif_open_resource_type_x is called on OTP 22/23, it does:

sys_memcpy(&ort->new_callbacks, init, sizeof(ErlNifResourceTypeInit));

where OTP's own sizeof is 24 bytes, but init points to Rustler's 40-byte struct, so OTP reads 16 bytes past the end of the new_callbacks field, overflowing into the next heap allocation.

• On OTP debug builds: ASSERT fires immediately on startup.
• On OTP release builds: silent heap corruption with ~35-40% crash rate (timing-dependent heisenbug).

Fix

Gate both fields on #[cfg(feature = "nif_version_2_16")] in rustler/src/sys/types.rs and update the companion struct-literal initialisers in rustler/src/resource/registration.rs accordingly.

The struct is then 24 bytes on NIF < 2.16 and 40 bytes on NIF >= 2.16, matching what OTP expects.

Why this matters

Rustler's own README documents NIF 2.15 (OTP 22) as the default supported version. The bug makes every NIF compiled with default settings silently corrupt heap on OTP 22/23.

[filmor]

Thank you for the fix. OTP's behaviour here is really odd :)

The comment is also a bit off. The behaviour is only broken in OTP<24. From then onwards (erlang/otp#4741) the members are copied individually instead of "blindly" memcpy'ing everything over.

[lmth]

Thank you for the fix. OTP's behaviour here is really odd :)

The comment is also a bit off. The behaviour is only broken in OTP<24. From then onwards (erlang/otp#4741) the members are copied individually instead of "blindly" memcpy'ing everything over.

Well, thank you! That was fast.
(I ran into this when backporting a NIF that was working perfectly on OTP-27, to an older branch of our product which runs on OTP-22. Obviously, when fixing it locally, I must also bring it upstream.)
I really appreciate the rustler project.