- [ ] `tanhshrink(x) = x - tanh(x)` **(concavoconvex)** - [ ] `relu6(x) = min(max(0, x), 6)` **(convexoconcave)** - [ ] `mish(x) = x * tanh(softplus(x))` **(mixed convexity)** - [ ] `lisht(x) = x * tanh(x)` - [ ] `hardtanh(x) = max(-1, min(1, x))` **(softmax)**
tanhshrink(x) = x - tanh(x)(concavoconvex)relu6(x) = min(max(0, x), 6)(convexoconcave)mish(x) = x * tanh(softplus(x))(mixed convexity)lisht(x) = x * tanh(x)hardtanh(x) = max(-1, min(1, x))(softmax)