code wiki / _hdl_build / nx_xlate_mt.nx
nx_xlate_mt.nx
buildroot/runtime/_hdl_build/nx_xlate_mt.nx
about
nx_xlate_mt.nx -- sovereign DATA-DRIVEN RU<->EN machine-translation engine (the Belarus<->Texas pair).
Replaces the 16-word hardcoded nx_mt_ruen stub. The lexicon + phrase table + morphology are DATA loaded
from TSV at build time (NOT hardcoded if-else): greedy LONGEST-match over words AND multi-word phrases,
Cyrillic-correct case folding (fixes "Привет"=="привет"), ё->е normalization, and Russian inflectional
suffix-stripping so inflected surface forms map to their lemma. NO float, deterministic.
dir 0 = EN->RU (emit Cyrillic), dir 1 = RU->EN. Unknown tokens pass through verbatim.
This is the R2 data-driven lexicon rung; open-domain neural MT stays the long pole. license_tier: ORIGINAL
dependencies 1 imports · 3 importers
imports: nx_syscalls.nx
imported by: nx_translate_daemon.nxnx_xlate_mt_gate.nxnx_xlate_mt_smoke.nx
structs
| none |
consts
| 9 | const XL_MAGIC_8192: i64 = 8192 |
| 10 | const XL_MAGIC_16384: i64 = 16384 |
| 11 | const XL_MAGIC_2048: i64 = 2048 |
| 13 | const XL_MAXENT: i64 = 8192 |
| 14 | const XL_MAXSUF: i64 = 128 |
| 15 | const XL_NORMCAP: i64 = 524288 // normalized-text arena bytes (holds normalized copies of every lexicon side) |
| 18 | const XLE_NENT: i64 = 0 |
| 19 | const XLE_TYPE: i64 = 1 |
| 20 | const XLE_RUOFF: i64 = 2 |
| 21 | const XLE_RULEN: i64 = 3 |
| 22 | const XLE_ENOFF: i64 = 4 |
| 23 | const XLE_ENLEN: i64 = 5 |
| 24 | const XLE_NRUOFF: i64 = 6 |
| 25 | const XLE_NRULEN: i64 = 7 |
| 26 | const XLE_NENOFF: i64 = 8 |
| 27 | const XLE_NENLEN: i64 = 9 |
| 28 | const XLE_NORM: i64 = 10 |
| 29 | const XLE_LEX: i64 = 11 |
| 30 | const XLE_NSUF: i64 = 12 |
| 31 | const XLE_SUFOFF: i64 = 13 |
| 32 | const XLE_SUFLEN: i64 = 14 |
| 33 | const XLE_SUF: i64 = 15 |
functions
| 35 | func xl_putb(out: *u8, op: *i64, cap: i64, b: i64) -> i64 |
| 41 | func xl_is_tok(c: i64) -> i64 |
| 49 | func xl_norm_tok(src: *u8, s: i64, e: i64, out: *u8, op: *i64, cap: i64) -> i64 |
| 89 | func xl_norm_phrase(src: *u8, s: i64, e: i64, out: *u8, op: *i64, cap: i64) -> i64 |
| 105 | func xl_norm_str(src: *u8, n: i64, nbuf: *u8, ncap: i64, tstart: *i64, maxtok: i64, ntok_out: *i64) -> i64 |
| 129 | func xl_field(buf: *u8, n: i64, ip: *i64, fs: *i64, fe: *i64, term: *i64) -> i64 called by 1: xl_build |
| 145 | func xl_meq(nbuf: *u8, pos: i64, nlen: i64, arena: *u8, aoff: i64, alen: i64) -> i64 |
| 152 | func xl_ntok_of(arena: *u8, aoff: i64, alen: i64) -> i64 called by 1: xl_translate |
| 159 | func xl_tok_end(nlen: i64, tstart: *i64, ntok: i64, t: i64) -> i64 called by 1: xl_translate |
| 164 | func xl_find_word(eng: *i64, dir: i64, nbuf: *u8, pos: i64, len: i64) -> i64 |
| 188 | func xl_morph_lookup(eng: *i64, dir: i64, nbuf: *u8, pos: i64, tlen: i64) -> i64 |
| 224 | func xl_build(eng: *i64, lex: *u8, lexn: i64, morph: *u8, morphn: i64) -> i64 |
| 312 | func xl_translate(eng: *i64, src: *u8, n: i64, dir: i64, dst: *u8, cap: i64) -> i64 |