code wiki / (root) / nx_gguf_load_lazy.nx

nx_gguf_load_lazy.nx

buildroot/runtime/nx_gguf_load_lazy.nx

4698 B119 linesdepth 7pulls 9 transitivereach 23 importersview sourcekind librarytopic gguf
docsdependenciesstructsconstsfunctions

about

nx_gguf_load_lazy.nx -- Phase B of conductor arc. L4.5 brick. Composes nx_placement.nx (Phase A) with nx_gguf_load.nx to provide a ZERO-RAM-FOOTPRINT tensor handle: load a GGUF tensor by name and get back an NxPlacedTensor in lazy (MMAP) placement. The dequantized Q10 i64 storage is NOT allocated until the caller invokes nx_placed_tensor_get / nx_placed_tensor_materialize. Honest architectural property (NOT a perf claim): Each lazy handle is 88 bytes regardless of underlying tensor size. At LOAD time the substrate holds the model as 88 * n_tensors bytes of typed handles (e.g. ~25 KB for a Llama-7B shape) without dequantizing anything. Materialization happens per-tensor at first compute access; demote releases the dequantized buffer back to the lazy state. This is semantically similar to llama.cpp's mmap + per-block dequant-on-demand pattern. The structural difference: our handle is a typed primitive a future conductor (Phase C dispatcher / Phase D arbiter) can route, evict, and tier-promote. llama.cpp's lazy dequant is in the matmul kernel internals, not exposed. NOT a claim: - "lazy = smaller VRAM than CUDA" -- depends on working-set and dispatcher; lazy alone proves nothing about peak RAM - "lazy = faster than llama.cpp" -- llama.cpp's mmap path achieves the same architectural property by a different name - Any "Nx less RAM" ratio without measurement against a real workload on real hardware Composition: nx_placement.nx -- NxPlacedTensor + lifecycle nx_gguf.nx -- header + tensor_info walker nx_gguf_load.nx -- existing per-tensor canonical loader (composed for materialization path) genealogy_id: numpy_memmap_pattern + huggingface_safetensors_lazy lineage_id: substrate_gguf_lazy_loader_v1_conductor_phase_b

dependencies 6 imports · 23 importers

nx_syscalls.nx nx_tier.nx nx_tensor.nx nx_gguf.nx nx_gguf_load.nx nx_placement.nx nx_gguf_load_lazy.nx nx_gguf_load_lazy_test.nx nx_live_fire_gguf.nx nx_live_fire_gguf_test.nx nx_q4k_dot_simd.nx nx_q4k_gemm_mt.nx nx_q4k_linear.nx nx_q4k_linear_hp.nx nx_q4k_real_gemm.nx nx_q4k_speed_bench.nx nx_q5k_dot_simd.nx

diagram shows first 10 each side; +0 more imports, +13 more importers in the complete lists below.

imports: nx_syscalls.nxnx_tier.nxnx_tensor.nxnx_gguf.nxnx_gguf_load.nxnx_placement.nx

imported by: nx_gguf_load_lazy_test.nxnx_live_fire_gguf.nxnx_live_fire_gguf_test.nxnx_q4k_dot_simd.nxnx_q4k_gemm_mt.nxnx_q4k_linear.nxnx_q4k_linear_hp.nxnx_q4k_real_gemm.nxnx_q4k_speed_bench.nxnx_q5k_dot_simd.nxnx_qwen_blk0_types.nxnx_qwen_hybrid_attn.nxnx_qwen_hybrid_ffn.nxnx_qwen_hybrid_qkv.nxnx_real_gguf_test.nxnx_zimage_dequant_block.nxnx_zimage_gguf_arch.nxnx_zimage_gguf_probe.nxnx_zimage_layout.nxnx_zimage_real_op.nxnx_zimage_real_proj.nxnx_zimage_real_qkv.nxnx_zimage_weights.nx

structs

none

consts

none

functions

71func nx_gguf_load_tensor_lazy(buf: *u8, hdr: *NxGgufHeader,
117func nx_gguf_lazy_handle_bytes(n_tensors: nx_int) -> i64
called by 1: main