nx_shard_query.nx
buildroot/runtime/nx_shard_query.nx
about
nx_shard_query.nx -- LIB: sharded SCATTER-GATHER query = the distributed READ PATH that makes the index scale
horizontally to petabytes. The index is partitioned into shards (by host/hash -- nx_shard); a query fans out to ALL
shards, each returns its LOCAL top-k, and the coordinator MERGES the per-shard top-k into the GLOBAL top-k. EXACT by
invariant: a doc in the global top-k must be in its own shard's top-k (a shard can't hide a global winner in its
local top-k), so merging (nshards*k) candidates and taking top-k is identical to a single whole-corpus index. Built +
gated NOW on a fixture; the network fan-out + parallel shard servers run on the NAS/cluster later. No float.
license_tier: ORIGINAL
dependencies 1 imports · 1 importers
imports: nx_syscalls.nx
imported by: nx_shard_query_gate.nx
structs
| none |
consts
| none |
functions
| 11 | func sq_insert(tkd: *i64, tks: *i64, k: i64, filled: *i64, d: i64, s: i64) -> i64 called by 1: sq_topk |
| 25 | func sq_topk(docs: *i64, scores: *i64, n: i64, k: i64, tkd: *i64, tks: *i64) -> i64 |
| 33 | func sq_scatter_gather(sd: *i64, ss: *i64, sn: *i64, nshards: i64, k: i64, tkd: *i64, tks: *i64) -> i64 |