Skip to content

feat: Add HardSigmoid, HardSwish, and stabilized Softplus operators#13

Open
AdityaDRathore wants to merge 1 commit intoML4EP:devfrom
AdityaDRathore:feature/hardsigmoid-hardwish-softplus-op
Open

feat: Add HardSigmoid, HardSwish, and stabilized Softplus operators#13
AdityaDRathore wants to merge 1 commit intoML4EP:devfrom
AdityaDRathore:feature/hardsigmoid-hardwish-softplus-op

Conversation

@AdityaDRathore
Copy link

Port of root-project/root#20933, root-project/root#20944, and root-project/root#21092 to the standalone SOFIE repository.

HardSigmoid (ONNX Opset 6+)

  • ROperator_HardSigmoid — Stateful operator with configurable alpha and beta attributes
  • Formula: y = max(0, min(1, alpha * x + beta)) (defaults: α=0.2, β=0.5)
  • Uses std::fmax/std::fmin with hexfloat clamp constants for AVX mapping

HardSwish (ONNX Opset 14)

  • ROperator_HardSwish — Inline code generation (does not instantiate HardSigmoid)
  • Formula: y = x * max(0, min(1, x/6 + 0.5))
  • Split topology with intermediate variable h for debuggability
  • Hexfloat constant 0x1.5555555555555p-3f for exact 1/6 representation

Softplus (Numerical Stability Fix)

  • Fixes overflow vulnerability in std::log(std::exp(x) + 1) for x > 88.72
  • Stabilized formula: y = (x >= 20.0) ? x : std::log1p(std::exp(x))
  • Threshold defined as hexfloat 0x1.4000000000000p+4f for cross-platform exactness
  • Added kSoftplus to EBasicUnaryOperator enum and GetStdLibs()

Testing

Comprehensive GTest suites exist in the ROOT PRs covering:

  • Hexfloat constant verification in generated code
  • Numerical correctness (linear region, clamp boundaries, overflow protection)
  • Type/shape inference, error handling, StdLib dependencies

Tests can be adapted for the standalone SOFIE test infrastructure if needed.

Files changed

File Action
src/SOFIE_core/inc/SOFIE/ROperator_HardSigmoid.hxx New
src/SOFIE_core/inc/SOFIE/ROperator_HardSwish.hxx New
src/SOFIE_core/inc/SOFIE/ROperator_BasicUnary.hxx Modified — add kSoftplus
src/SOFIE_parsers/src/ParseHardSigmoid.cxx New
src/SOFIE_parsers/src/ParseHardSwish.cxx New
src/SOFIE_parsers/src/ParseBasicUnary.cxx Modified — add ParseSoftplus
src/SOFIE_parsers/src/RModelParser_ONNX.cxx Modified — register 3 operators
src/SOFIE_parsers/CMakeLists.txt Modified — add parser sources
src/SOFIE_core/CMakeLists.txt Modified — add operator headers

Port of root-project/root#20933, #20944, and #21092.
- HardSigmoid: stateful operator with alpha/beta attributes (ONNX Opset 6+)
- HardSwish: inline generation with split topology (ONNX Opset 14)
- Softplus: numerical stability fix using log1p and threshold at 20.0f
All use hexfloat constants for bit-exact reproducibility.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant