code wiki / (root) / lex.nx

lex.nx

buildroot/runtime/lex.nx

19362 B590 linesdepth 3pulls 7 transitivereach 32 importersview sourcekind librarytopic lex
docsdependenciesstructsconstsfunctions

about

lex.nx -- NishiLang source tokenizer, ported from lex.c. Accepts a null-terminated source buffer and produces a stream of Tok records. Mirrors lex.c's token layout (kind + line/col + either int_val or text[]). Same keyword set. Single-pass; zero allocations per token beyond the growing token array. Extends at 2x when full. Whitespace + // comments are skipped. Numbers support 0x/0b/0o prefixes with underscore separators. String literals read bytes until closing `"`.

dependencies 3 imports · 32 importers

syscalls.nx types.nx lex_kinds.nx lex.nx _offc_lex_only.nx _offc_nxc_small.nx fuzz_lex_test.nx fuzz_parse_test.nx lex_float_test.nx lex_test.nx main.nx nxc.nx prop_lex_number_test.nx self_parse_test.nx

diagram shows first 10 each side; +0 more imports, +22 more importers in the complete lists below.

imports: syscalls.nxtypes.nxlex_kinds.nx

imported by: _offc_lex_only.nx_offc_nxc_small.nxfuzz_lex_test.nxfuzz_parse_test.nxlex_float_test.nxlex_test.nxmain.nxnxc.nxprop_lex_number_test.nxself_parse_test.nxsmoke_lex2.nxsmoke_lex3.nxsmoke_lex4.nxsmoke_lex5.nxsmoke_lex6.nxsmoke_lex7.nxsmoke_lex_min.nxsmoke_nxc_pipeline.nxsmoke_nxc_pipeline10.nxsmoke_nxc_pipeline11.nxsmoke_nxc_pipeline12.nxsmoke_nxc_pipeline13.nxsmoke_nxc_pipeline14.nxsmoke_nxc_pipeline15.nxsmoke_nxc_pipeline2.nxsmoke_nxc_pipeline3.nxsmoke_nxc_pipeline4.nxsmoke_nxc_pipeline5.nxsmoke_nxc_pipeline6.nxsmoke_nxc_pipeline7.nxsmoke_nxc_pipeline8.nxsmoke_nxc_pipeline9.nx

structs

114struct Lex {

consts

none

functions

54func is_alpha(c: i64) -> i64 {
called by 2: is_alnumlex_source
60func is_digit(c: i64) -> i64 {
called by 4: is_hexdigitis_alnumlex_numberlex_source
64func is_hexdigit(c: i64) -> i64 {
called by 1: lex_number calls 1: is_digit
70func hex_val(c: i64) -> i64 {
called by 1: lex_number
76func is_alnum(c: i64) -> i64 {
called by 1: lex_ident_or_kw calls 2: is_alphais_digit
86func keyword_lookup(t: *u8, len: i64) -> i64 {
called by 1: lex_ident_or_kw
127func push_tok(L: *Lex, t: *Tok) -> i64 {
called by 5: lex_ident_or_kwlex_numberlex_stringemit_punctlex_source
148func advance(L: *Lex) -> i64 {
called by 6: skip_ws_commentslex_ident_or_kwlex_numberlex_stringemit_punctlex_source
161func peek(L: *Lex) -> i64 {
called by 5: skip_ws_commentslex_ident_or_kwlex_numberlex_stringlex_source
165func peek2(L: *Lex) -> i64 {
called by 3: skip_ws_commentslex_numberlex_source
173func skip_ws_comments(L: *Lex) -> i64 {
called by 1: lex_source calls 3: peekadvancepeek2
211func lex_ident_or_kw(L: *Lex) -> i64 {
called by 1: lex_source calls 6: tok_text_ptrpeekis_alnumadvancekeyword_lookuppush_tok
237func lex_number(L: *Lex) -> i64 {
called by 1: lex_source calls 7: peekpeek2advanceis_hexdigithex_valpush_tok+1
391func lex_string(L: *Lex) -> i64 {
called by 1: lex_source calls 4: advancetok_text_ptrpeekpush_tok
460func emit_punct(L: *Lex, kind: i64, len: i64) -> i64 {
called by 1: lex_source calls 2: advancepush_tok
483func lex_source(src: *u8, cap: i64) -> *Tok {
called by 30: mainnxc_compilemainmainmaincompile+24 calls 11: skip_ws_commentspeekis_alphalex_ident_or_kwis_digitlex_number+5