nx_gguf_load_lazy.nx
buildroot/runtime/nx_gguf_load_lazy.nx
about
nx_gguf_load_lazy.nx -- Phase B of conductor arc.
L4.5 brick. Composes nx_placement.nx (Phase A) with nx_gguf_load.nx
to provide a ZERO-RAM-FOOTPRINT tensor handle: load a GGUF tensor
by name and get back an NxPlacedTensor in lazy (MMAP) placement.
The dequantized Q10 i64 storage is NOT allocated until the caller
invokes nx_placed_tensor_get / nx_placed_tensor_materialize.
Honest architectural property (NOT a perf claim):
Each lazy handle is 88 bytes regardless of underlying tensor
size. At LOAD time the substrate holds the model as 88 *
n_tensors bytes of typed handles (e.g. ~25 KB for a Llama-7B
shape) without dequantizing anything. Materialization happens
per-tensor at first compute access; demote releases the
dequantized buffer back to the lazy state.
This is semantically similar to llama.cpp's mmap + per-block
dequant-on-demand pattern. The structural difference: our handle
is a typed primitive a future conductor (Phase C dispatcher / Phase
D arbiter) can route, evict, and tier-promote. llama.cpp's lazy
dequant is in the matmul kernel internals, not exposed.
NOT a claim:
- "lazy = smaller VRAM than CUDA" -- depends on working-set
and dispatcher; lazy alone proves nothing about peak RAM
- "lazy = faster than llama.cpp" -- llama.cpp's mmap path
achieves the same architectural property by a different name
- Any "Nx less RAM" ratio without measurement against a real
workload on real hardware
Composition:
nx_placement.nx -- NxPlacedTensor + lifecycle
nx_gguf.nx -- header + tensor_info walker
nx_gguf_load.nx -- existing per-tensor canonical loader
(composed for materialization path)
genealogy_id: numpy_memmap_pattern + huggingface_safetensors_lazy
lineage_id: substrate_gguf_lazy_loader_v1_conductor_phase_b
dependencies 6 imports · 23 importers
diagram shows first 10 each side; +0 more imports, +13 more importers in the complete lists below.
imports: nx_syscalls.nxnx_tier.nxnx_tensor.nxnx_gguf.nxnx_gguf_load.nxnx_placement.nx
imported by: nx_gguf_load_lazy_test.nxnx_live_fire_gguf.nxnx_live_fire_gguf_test.nxnx_q4k_dot_simd.nxnx_q4k_gemm_mt.nxnx_q4k_linear.nxnx_q4k_linear_hp.nxnx_q4k_real_gemm.nxnx_q4k_speed_bench.nxnx_q5k_dot_simd.nxnx_qwen_blk0_types.nxnx_qwen_hybrid_attn.nxnx_qwen_hybrid_ffn.nxnx_qwen_hybrid_qkv.nxnx_real_gguf_test.nxnx_zimage_dequant_block.nxnx_zimage_gguf_arch.nxnx_zimage_gguf_probe.nxnx_zimage_layout.nxnx_zimage_real_op.nxnx_zimage_real_proj.nxnx_zimage_real_qkv.nxnx_zimage_weights.nx
structs
| none |
consts
| none |
functions
| 71 | func nx_gguf_load_tensor_lazy(buf: *u8, hdr: *NxGgufHeader, |
| 117 | func nx_gguf_lazy_handle_bytes(n_tensors: nx_int) -> i64 called by 1: main |