erts: Fix alternate signal stack sizing on musl (AT_MINSIGSTKSZ)

[arctarus]

What

Size the JIT's alternate signal stack to max(SIGSTKSZ, AT_MINSIGSTKSZ) instead of the bare
SIGSTKSZ, so the emulator starts on musl/Alpine running on CPUs with large signal frames.

Fixes #11248.

Why

On musl, SIGSTKSZ is a static 8192. On CPUs whose kernel reports AT_MINSIGSTKSZ > 8192
(AVX-512 / AMX), sigaltstack() returns ENOMEM and the BEAM aborts at startup. glibc >= 2.34
made SIGSTKSZ dynamic for exactly this reason; musl did not. Affects musl JIT builds, OTP 27+.

Reproduction (hardware-independent, any musl host)

Force sigaltstack to reject the 8192-byte request (simulating a kernel whose minimum exceeds it):

/* shim.c — gcc -shared -fPIC shim.c -o shim.so -ldl ; LD_PRELOAD=./shim.so erl -noshell -eval 'erlang:halt().' */
#define _GNU_SOURCE
#include <signal.h>
#include <errno.h>
#include <dlfcn.h>
static int (*real)(const stack_t*, stack_t*);
int sigaltstack(const stack_t *ss, stack_t *old){
    if(!real) real = dlsym(RTLD_NEXT, "sigaltstack");
    if(ss && ss->ss_size <= 8192){ errno = ENOMEM; return -1; }
    return real(ss, old);
}

Unpatched: aborts with "Failed to set alternate signal stack". Patched: boots normally.

Notes / open questions for reviewers

• Followed the #if defined(__linux__) guard you suggested; happy to switch to a configure
  check for getauxval (AC_CHECK_FUNCS) instead if you'd prefer that for portability.
• This uses exactly max(SIGSTKSZ, AT_MINSIGSTKSZ). If you'd like headroom above the bare
  kernel minimum (for the handler's own frames), I can add a margin — let me know the policy
  you'd want.

[CLAassistant]

All committers have signed the CLA.

[github-actions]

CT Test Results

    3 files    136 suites   50m 25s ⏱️
1 679 tests 1 622 ✅ 57 💤 0 ❌
2 321 runs  2 246 ✅ 75 💤 0 ❌

Results for commit 3a94587.

♻️ This comment has been updated with latest results.

To speed up review, make sure that you have read Contributing to Erlang/OTP and that all checks pass.

See the TESTING and DEVELOPMENT HowTo guides for details about how to run test locally.

Artifacts

• Complete CT logs (Download Logs)
• HTML Documentation (Download HTML Docs)
• Windows Installer

// Erlang/OTP Github Action Bot

[arctarus]

@garazdawi I applied your suggestions.

Thanks for the review!

[sverker]

Thanks for the contribution. We currently have a bit of administrative ambiguities about AI produced code and licensing. So, I rewrote the fix, based on patch-base-27 for possible patch releasing.

[arctarus]

@sverker no problem at all. Thanks!