code wiki / (root) / nx_shard_query.nx

nx_shard_query.nx

buildroot/runtime/nx_shard_query.nx

2744 B46 linesdepth 2pulls 2 transitivereach 1 importersview sourcekind librarytopic shard
docsdependenciesstructsconstsfunctions

about

nx_shard_query.nx -- LIB: sharded SCATTER-GATHER query = the distributed READ PATH that makes the index scale horizontally to petabytes. The index is partitioned into shards (by host/hash -- nx_shard); a query fans out to ALL shards, each returns its LOCAL top-k, and the coordinator MERGES the per-shard top-k into the GLOBAL top-k. EXACT by invariant: a doc in the global top-k must be in its own shard's top-k (a shard can't hide a global winner in its local top-k), so merging (nshards*k) candidates and taking top-k is identical to a single whole-corpus index. Built + gated NOW on a fixture; the network fan-out + parallel shard servers run on the NAS/cluster later. No float. license_tier: ORIGINAL

dependencies 1 imports · 1 importers

nx_syscalls.nx nx_shard_query.nx nx_shard_query_gate.nx

imports: nx_syscalls.nx

imported by: nx_shard_query_gate.nx

structs

none

consts

none

functions

11func sq_insert(tkd: *i64, tks: *i64, k: i64, filled: *i64, d: i64, s: i64) -> i64
called by 1: sq_topk
25func sq_topk(docs: *i64, scores: *i64, n: i64, k: i64, tkd: *i64, tks: *i64) -> i64
33func sq_scatter_gather(sd: *i64, ss: *i64, sn: *i64, nshards: i64, k: i64, tkd: *i64, tks: *i64) -> i64
called by 1: main calls 2: sys_mmapsq_topk