nx_token_permutation.nx
buildroot/runtime/nx_token_permutation.nx
about
nx_token_permutation.nx -- order-effect analyzer for prompt token sequences.
Foundation primitive for reasoning about PROMPT ORDER EFFECTS in
image-gen models. Given two ordered token-id sequences that share
the same multi-set (i.e., a permutation of each other), measure:
1. Kendall tau distance # of pairwise inversions; the
canonical permutation-distance metric
2. Position-0 changed did the first token swap? (CLIP and
T5 both weight position-0 heavily)
3. Position-last changed did the trailing token swap?
(some text encoders weight the EOS-
adjacent position differently)
4. New-neighbor pairs pairs of tokens now adjacent that
weren't before (attention is denser
between adjacent tokens; new pairs
means new latent joint-meaning)
5. Same-multi-set check did the two sequences share tokens?
(if not, this analysis doesn't apply
and the verdict is NOT_PERMUTATION)
Output also carries a SEALED-ENUM verdict per the dual-reading
cardinal:
NX_PERMUT_IDENTICAL identical sequences
NX_PERMUT_ADJACENT_SWAP single adjacent transposition
NX_PERMUT_MINOR_REORDER tau >= 0.85
NX_PERMUT_MODERATE_REORDER tau in [0.5, 0.85)
NX_PERMUT_SEVERE_REORDER tau < 0.5
NX_PERMUT_NOT_PERMUTATION different multi-sets
USE CASES:
- "beautiful tall girl" vs "tall beautiful girl" -> ADJACENT_SWAP
position-0 changed -> for CLIP-based models, predict ~35% attention
shift; for T5-based models (Z-Image), predict ~15% shift.
- "beautiful tall girl" vs "girl tall beautiful" -> SEVERE_REORDER
tau ~ 0; position-0 + position-last both changed; new neighbor
pairs (girl-tall, tall-beautiful where before was beautiful-tall).
- "tall girl" vs "tall beautiful girl" -> NOT_PERMUTATION (different
multi-sets; "added a word" case the user asked about).
dependencies 2 imports · 1 importers
imports: nx_syscalls.nxnx_tier.nx
imported by: nx_token_permutation_test.nx
structs
| 75 | struct PermutationReport |
consts
| 64 | const NX_PERMUT_Q: nx_int = 1024 |
| 67 | const NX_PERMUT_IDENTICAL: nx_int = 0 |
| 68 | const NX_PERMUT_ADJACENT_SWAP: nx_int = 1 |
| 69 | const NX_PERMUT_MINOR_REORDER: nx_int = 2 |
| 70 | const NX_PERMUT_MODERATE_REORDER: nx_int = 3 |
| 71 | const NX_PERMUT_SEVERE_REORDER: nx_int = 4 |
| 72 | const NX_PERMUT_NOT_PERMUTATION: nx_int = 5 |
| 73 | const NX_PERMUT_N_VERDICTS: nx_int = 6 |
functions
| 92 | func _permut_same_multi_set(a: *i64, n_a: nx_int, b: *i64, n_b: nx_int) -> nx_int called by 1: nx_token_permutation_compute |
| 127 | func _permut_build_position_map(a: *i64, b: *i64, n: nx_int, pos_buf: *i64) -> nx_int |
| 153 | func _permut_count_inversions(pos_buf: *i64, n: nx_int) -> nx_int called by 1: nx_token_permutation_compute |
| 170 | func _permut_count_new_neighbors(a: *i64, b: *i64, n: nx_int) -> nx_int called by 1: nx_token_permutation_compute |
| 196 | func nx_token_permutation_compute( |
| 270 | func nx_token_permutation_verdict_is_valid(v: nx_int) -> nx_int called by 1: main |