nx_nofloat_q4k.nx

nx_nofloat_q4k.nx -- LM4 RESIDENT-QUANT DECODE (2026-09-02): the 7B fits because the weights STAY Q4_K. MEASURED (nx_nofloat_llm.nx nf_alloc_layer_wb_i8): the "i8" layer cache stores 2 B/weight (i16 lanes for __i16x16_madd), so Qwen2.5-Coder-7B = 6.5e9 layer params = 13 GB of anonymous memory before the i32 head (2.2 GB), the copied model buffer (4.7 GB via sys_read_file) and the MAXT scratch -- it cannot fit a 16 GB WSL VM and no cache hygiene changes that. This lib keeps every Q4_K projection IN THE FILE MAP (0.5625 B/weight, file-backed, evictable) and dots it with the estate's fused SIMD kernel (nx_q4k_dot_simd_lib, bit-exact vs nx_q4k_dot_row_col), so a Q4_K layer costs 0 anonymous bytes. A tensor of any other quant type (Q4_K_M puts Q6_K on some attn_v/ffn_down rows) falls back PER TENSOR to the incumbent i16 cache (nf_quant_w_i8 + mm_pool_i8) -- counted and announced, never silent. NUMERIC CONTRACT (why the shift arithmetic is what it is): the serve keeps activations in Q16; its i8 path computes dst = (sx*sw*sum(xq*wq)) >> shift with W in Q16 (shift 24 for a Q24 rmsnorm input, 16 for a Q16 input). nx_q4k_dot_simd returns sum over the row of W_Q24 * col_i16 -- the weight carries 8 more fractional bits -- so dst = ((dot >> NQ_PRE_SHIFT) * sx) >> (shift + NQ_W_EXTRA_BITS - NQ_PRE_SHIFT). The activation is packed to i16 under a dynamic per-call scale sx = ceil(max|x| / 32767) (15 significant bits) where the i8 path packs to 7 bits: this route is STRICTLY MORE EXACT on the activation side and lossless on the weight side (the Q4_K bytes ARE the model). The pre-shift is overflow headroom: |dot| reaches ~1e16 on an 18944-wide ffn_down row and sx reaches ~2^19, so the product is taken after >>16. Duplicates deliberately NOT created: attention/softmax/residual come from nf_attn_kv_core (shared with the i8 decoder), the fallback matvec IS mm_pool_i8, the per-row quantiser IS nf_quant_w_i8. license_tier: ORIGINAL No hw writes (Rule 26).