code wiki / (root) / nx_dispatcher.nx

nx_dispatcher.nx

buildroot/runtime/nx_dispatcher.nx

15139 B385 linesdepth 7pulls 9 transitivereach 2 importersview sourcekind library
docsdependenciesstructsconstsfunctions

about

nx_dispatcher.nx -- hardware-aware op routing. Per CARDINAL [[feedback-conductor-heterogeneous-compute-no-second-class- resources]]: ops dispatch hardware-aware -- matmul on GPU when operands are in VRAM, activations on CPU SIMD when data lives in RAM, no PCIe round-trips for tiny ops. REFUSES the "everything goes to GPU" pattern. CONDUCTOR ARC ROLE: this is PHASE C of the conductor arc documented in nx_placement.nx. Composition stack: A) nx_placement.nx -- NxPlacedTensor with placement tag (MMAP/ OWNED/VRAM/NETWORK). SHIPPED. B) nx_gguf_load_lazy.nx -- lazy GGUF load returning NxPlacedTensor. SHIPPED. C) nx_dispatcher.nx -- per-op hardware routing. THIS BRICK. D) nx_resource_arbiter.nx -- multi-tier budget tracking. THIS SESSION. E) async DMA primitives. Queued. Integration: the bridge function nx_di_req_from_placed_inputs() reads NxPlacedTensor.placement to populate this primitive's inputs_on_gpu/cpu/cold accounting, so callers using the existing nx_placement substrate can compose without forking. V1 ships a sealed op-kind enum + per-op cost model + dispatch decision that picks the cheapest backend given the placement of inputs (uses [[nx_tensor_placement]] semantics: tier index 0=fastest, 7=coldest). Cost model is intentionally simple Q10: BACKEND_COST_PER_BYTE * total_input_bytes + TRANSFER_COST * misplaced_bytes. Refinement (cache effects, batch size, kernel launch overhead) deferred.

dependencies 4 imports · 2 importers

nx_syscalls.nx nx_tier.nx nx_tensor.nx nx_placement.nx nx_dispatcher.nx nx_companion_compose_test.nx nx_dispatcher_test.nx

imports: nx_syscalls.nxnx_tier.nxnx_tensor.nxnx_placement.nx

imported by: nx_companion_compose_test.nxnx_dispatcher_test.nx

structs

73struct NxOpRequest
86struct NxDispatchDecision

consts

35const NX_MAGIC_1000000: i64 = 1000000
36const NX_MAGIC_1024: i64 = 1024
40const NX_DI_OP_MATMUL: nx_int = 0
41const NX_DI_OP_ELEMENTWISE_UNARY: nx_int = 1 // silu / relu / sigmoid
42const NX_DI_OP_ELEMENTWISE_BINARY: nx_int = 2 // add / mul
43const NX_DI_OP_REDUCE_SUM: nx_int = 3
44const NX_DI_OP_RMS_NORM: nx_int = 4
45const NX_DI_OP_SOFTMAX: nx_int = 5
46const NX_DI_OP_ATTENTION: nx_int = 6
47const NX_DI_OP_EMBEDDING_LOOKUP: nx_int = 7
48const NX_DI_OP_COPY: nx_int = 8
49const NX_DI_OP_QUANTIZE: nx_int = 9
50const NX_DI_OP_DEQUANTIZE: nx_int = 10
51const NX_DI_OP_N: nx_int = 11
55const NX_DI_BE_CPU_SCALAR: nx_int = 0
56const NX_DI_BE_CPU_SIMD: nx_int = 1
57const NX_DI_BE_GPU_COMPUTE: nx_int = 2
58const NX_DI_BE_GPU_TENSOR_CORE: nx_int = 3
59const NX_DI_BE_NPU: nx_int = 4 // future: neural processor
60const NX_DI_BE_N: nx_int = 5
64const NX_DI_V_DISPATCHED: nx_int = 0
65const NX_DI_V_REQUIRES_PROMOTION: nx_int = 1 // inputs in wrong tier
66const NX_DI_V_NO_VIABLE_BACKEND: nx_int = 2
67const NX_DI_V_INVALID: nx_int = 3
68const NX_DI_V_NULL: nx_int = 4
69const NX_DI_V_N: nx_int = 5
82const NX_DI_REQ_BYTES: nx_int = 48
93const NX_DI_DEC_BYTES: nx_int = 32

functions

97func nx_di_op_is_valid(o: nx_int) -> nx_int
103func nx_di_be_is_valid(b: nx_int) -> nx_int
109func nx_di_v_is_valid(v: nx_int) -> nx_int
called by 1: main
115func nx_di_op_is_compute_heavy(o: nx_int) -> nx_int
called by 1: main
121func nx_di_op_is_lightweight(o: nx_int) -> nx_int
called by 1: main
128func nx_di_be_is_gpu(b: nx_int) -> nx_int
134func nx_di_be_is_cpu(b: nx_int) -> nx_int
146func _di_base_cost_q10(op: nx_int, backend: nx_int) -> nx_int
219func _di_transfer_penalty_q10(backend: nx_int) -> nx_int
232func nx_di_estimate_cost_q10(req: *NxOpRequest, backend: nx_int) -> nx_int
265func nx_di_dispatch(req: *NxOpRequest, decision: *NxDispatchDecision) -> nx_int
297func nx_di_req_new(op_kind: nx_int,
called by 1: main calls 2: nx_di_op_is_validsys_mmap
315func nx_di_dec_new() -> *NxDispatchDecision
called by 2: mainmain calls 1: sys_mmap
341func _di_bytes_of_placed(p: *NxPlacedTensor) -> nx_size
357func nx_di_req_from_placed_inputs(op_kind: nx_int,