nx_f32_activations.nx
buildroot/runtime/nx_f32_activations.nx
about
nx_f32_activations.nx -- bits-up f32 ML activation functions.
L7 composition brick. Composes L4 mul/add/sub/div + L6 exp.
Provides sigmoid, SiLU/Swish, tanh -- the standard activation
set used in Llama / Mistral / Qwen / GPT-class transformers.
All functions: scalar f32 in, scalar f32 out. Caller broadcasts
across tensor elements. No libm.
Sigmoid: sigma(x) = 1 / (1 + exp(-x))
Endpoints: x -> +inf : 1.0
x -> -inf : +0
Our exp clamps very-negative-arg to 0 -> sigma -> 1,
and very-positive-arg to +inf -> sigma -> 0. Both
extreme cases handled correctly without underflow path.
SiLU / Swish: silu(x) = x * sigma(x)
Used as the gate activation in Llama FFN SwiGLU.
Hendrycks+Gimpel 2016 (GELU); Ramachandran+ 2017 (Swish);
Elfwing+ 2017 (SiLU).
Tanh: tanh(x) = 2 * sigma(2x) - 1
Cheaper than the canonical (exp(x)-exp(-x))/(exp(x)+exp(-x))
form via the sigmoid identity.
dependencies 5 imports · 19 importers
diagram shows first 10 each side; +0 more imports, +9 more importers in the complete lists below.
imports: nx_syscalls.nxnx_tier.nxnx_f32.nxnx_f32_div.nxnx_f32_exp.nx
imported by: nx_f32_activations_test.nxnx_f32_dit_block_linear.nxnx_f32_dit_block_simd.nxnx_f32_dit_block_tiny.nxnx_f32_llama_block.nxnx_f32_llama_block_test.nxnx_f32_llama_block_v4.nxnx_f32_llama_block_v4_test.nxnx_f32_llama_v4b.nxnx_f32_llama_v4p.nxnx_f32_qwen_layer.nxnx_f32_resblock.nxnx_f32_transformer.nxnx_hifigan_gen.nxnx_hifigan_run.nxnx_hifigan_verify.nxnx_moe.nxnx_moe_gate.nxnx_qwen_hybrid_ffn.nx
structs
| none |
consts
| 32 | const NX_F32_ACT_ONE: i64 = 0x3F800000 // 1.0 |
| 33 | const NX_F32_ACT_TWO: i64 = 0x40000000 // 2.0 |
functions
| 37 | func nx_f32_sigmoid(x: i64) -> i64 |
| 48 | func nx_f32_silu(x: i64) -> i64 |
| 62 | func nx_f32_tanh(x: i64) -> i64 |