nx_dispatcher.nx
buildroot/runtime/nx_dispatcher.nx
about
nx_dispatcher.nx -- hardware-aware op routing.
Per CARDINAL [[feedback-conductor-heterogeneous-compute-no-second-class-
resources]]: ops dispatch hardware-aware -- matmul on GPU when operands
are in VRAM, activations on CPU SIMD when data lives in RAM, no PCIe
round-trips for tiny ops. REFUSES the "everything goes to GPU" pattern.
CONDUCTOR ARC ROLE: this is PHASE C of the conductor arc documented in
nx_placement.nx. Composition stack:
A) nx_placement.nx -- NxPlacedTensor with placement tag (MMAP/
OWNED/VRAM/NETWORK). SHIPPED.
B) nx_gguf_load_lazy.nx -- lazy GGUF load returning NxPlacedTensor.
SHIPPED.
C) nx_dispatcher.nx -- per-op hardware routing. THIS BRICK.
D) nx_resource_arbiter.nx -- multi-tier budget tracking. THIS SESSION.
E) async DMA primitives. Queued.
Integration: the bridge function nx_di_req_from_placed_inputs() reads
NxPlacedTensor.placement to populate this primitive's
inputs_on_gpu/cpu/cold accounting, so callers using the existing
nx_placement substrate can compose without forking.
V1 ships a sealed op-kind enum + per-op cost model + dispatch decision
that picks the cheapest backend given the placement of inputs (uses
[[nx_tensor_placement]] semantics: tier index 0=fastest, 7=coldest).
Cost model is intentionally simple Q10: BACKEND_COST_PER_BYTE *
total_input_bytes + TRANSFER_COST * misplaced_bytes. Refinement
(cache effects, batch size, kernel launch overhead) deferred.
dependencies 4 imports · 2 importers
imports: nx_syscalls.nxnx_tier.nxnx_tensor.nxnx_placement.nx
imported by: nx_companion_compose_test.nxnx_dispatcher_test.nx
structs
| 73 | struct NxOpRequest |
| 86 | struct NxDispatchDecision |
consts
| 35 | const NX_MAGIC_1000000: i64 = 1000000 |
| 36 | const NX_MAGIC_1024: i64 = 1024 |
| 40 | const NX_DI_OP_MATMUL: nx_int = 0 |
| 41 | const NX_DI_OP_ELEMENTWISE_UNARY: nx_int = 1 // silu / relu / sigmoid |
| 42 | const NX_DI_OP_ELEMENTWISE_BINARY: nx_int = 2 // add / mul |
| 43 | const NX_DI_OP_REDUCE_SUM: nx_int = 3 |
| 44 | const NX_DI_OP_RMS_NORM: nx_int = 4 |
| 45 | const NX_DI_OP_SOFTMAX: nx_int = 5 |
| 46 | const NX_DI_OP_ATTENTION: nx_int = 6 |
| 47 | const NX_DI_OP_EMBEDDING_LOOKUP: nx_int = 7 |
| 48 | const NX_DI_OP_COPY: nx_int = 8 |
| 49 | const NX_DI_OP_QUANTIZE: nx_int = 9 |
| 50 | const NX_DI_OP_DEQUANTIZE: nx_int = 10 |
| 51 | const NX_DI_OP_N: nx_int = 11 |
| 55 | const NX_DI_BE_CPU_SCALAR: nx_int = 0 |
| 56 | const NX_DI_BE_CPU_SIMD: nx_int = 1 |
| 57 | const NX_DI_BE_GPU_COMPUTE: nx_int = 2 |
| 58 | const NX_DI_BE_GPU_TENSOR_CORE: nx_int = 3 |
| 59 | const NX_DI_BE_NPU: nx_int = 4 // future: neural processor |
| 60 | const NX_DI_BE_N: nx_int = 5 |
| 64 | const NX_DI_V_DISPATCHED: nx_int = 0 |
| 65 | const NX_DI_V_REQUIRES_PROMOTION: nx_int = 1 // inputs in wrong tier |
| 66 | const NX_DI_V_NO_VIABLE_BACKEND: nx_int = 2 |
| 67 | const NX_DI_V_INVALID: nx_int = 3 |
| 68 | const NX_DI_V_NULL: nx_int = 4 |
| 69 | const NX_DI_V_N: nx_int = 5 |
| 82 | const NX_DI_REQ_BYTES: nx_int = 48 |
| 93 | const NX_DI_DEC_BYTES: nx_int = 32 |
functions
| 97 | func nx_di_op_is_valid(o: nx_int) -> nx_int |
| 103 | func nx_di_be_is_valid(b: nx_int) -> nx_int |
| 109 | func nx_di_v_is_valid(v: nx_int) -> nx_int called by 1: main |
| 115 | func nx_di_op_is_compute_heavy(o: nx_int) -> nx_int called by 1: main |
| 121 | func nx_di_op_is_lightweight(o: nx_int) -> nx_int called by 1: main |
| 128 | func nx_di_be_is_gpu(b: nx_int) -> nx_int |
| 134 | func nx_di_be_is_cpu(b: nx_int) -> nx_int |
| 146 | func _di_base_cost_q10(op: nx_int, backend: nx_int) -> nx_int called by 1: nx_di_estimate_cost_q10 |
| 219 | func _di_transfer_penalty_q10(backend: nx_int) -> nx_int |
| 232 | func nx_di_estimate_cost_q10(req: *NxOpRequest, backend: nx_int) -> nx_int |
| 265 | func nx_di_dispatch(req: *NxOpRequest, decision: *NxDispatchDecision) -> nx_int |
| 297 | func nx_di_req_new(op_kind: nx_int, |
| 315 | func nx_di_dec_new() -> *NxDispatchDecision |
| 341 | func _di_bytes_of_placed(p: *NxPlacedTensor) -> nx_size called by 1: nx_di_req_from_placed_inputs |
| 357 | func nx_di_req_from_placed_inputs(op_kind: nx_int, |