code wiki / _hdl_build / nx_media_import.nx

nx_media_import.nx

buildroot/runtime/_hdl_build/nx_media_import.nx

7123 B128 linesdepth 22pulls 158 transitivereach 1 importersview sourcekind tooltopic media
docsdependenciesstructsconstsfunctions

about

nx_media_import.nx -- GENERAL POLITE media importer: pull images from ANY site (Shein/e-commerce/art/galleries) into the family library, RESPECTING the host's rules. Operator 2026-07-03: "import politely images from places like shein etc to get state of the art." The differentiator vs a crude scraper is POLITENESS BY CONSTRUCTION: 1. fetch the host's /robots.txt and HONOR it (nx_robots_allowed) -- a Disallow => we DO NOT fetch (refuse). 2. send an HONEST User-Agent identifying us + a contact/purpose. 3. pace requests to the host's Crawl-delay (nx_robots_crawl_delay), default a courteous floor. Then it reuses the PROVEN engine: mg_gallery_to_pages (Chrome-JA3 fetch that beats anti-bot CDNs + multi-strategy nx_img_harvest + download-integrity + cbx.json manifest) and lands the result in the caller's PRIVATE reader area (rl_add_personal) so it shows up in the zoom reader as a browsable image collection. Composes; adds only the politeness gate. CLI: nx_media_import <page_url> <slug> <ukey> (forked by the daemon on an "Import" POST). license_tier: ORIGINAL

dependencies 7 imports · 1 importers

nx_syscalls.nx nx_x509_trust_store.nx nx_trust_store_load_from_certdata. nx_https_fetch_follow.nx nx_robots.nx nx_manga_get.nx nx_reader_library.nx nx_media_import.nx nx_media_import_gate.nx

imports: nx_syscalls.nxnx_x509_trust_store.nxnx_trust_store_load_from_certdata.nxnx_https_fetch_follow.nxnx_robots.nxnx_manga_get.nxnx_reader_library.nx

imported by: nx_media_import_gate.nx

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main nx_trust_store_load_from_c sys_mmap sys_read_file sys_openat_rd sys_lseek sys_mmap ↻ sys_read sys_close nx_nss_certdata_parse sys_mmap ↻ _pat_class_cert _pat_value_octal _pat_end _find_newline _starts_with _parse_octal_line sys_mmap ↻ _is_space _decode_octal_escape _is_octal trust_store_alloc sys_mmap ↻ nx_x509_trust_store_load sys_mmap ↻ x509_parse sys_mmap ↻ asn1_cursor_init asn1_expect_tag sys_mmap ↻ asn1_read_tlv_header asn1_read_tag asn1_read_length asn1_read_length ↻ x509_read_tlv sys_mmap ↻ asn1_expect_tag ↻ x509_read_alg_id sys_mmap ↻ asn1_expect_tag ↻

structs

none

consts

19const MI_MAGIC_1000000: i64 = 1000000
20const MI_MAGIC_1024: i64 = 1024
21const MI_MAGIC_9309: i64 = 9309
22const MI_MAGIC_4194304: i64 = 4194304
25const MI_UA: *u8 = "NishiMediaImporter/1.0 (+https://nishifamily.com; personal family library import)"
26const MI_UA_LEN: i64 = 78
27const MI_POLITE_FLOOR_MS: i64 = 1000 // courteous default spacing if the host sets no Crawl-delay
28const MI_ROBOTS_CAP: i64 = 262144

functions

30func mi_w(s: *u8) -> i64 { var n: i64=0; while s[n]!=(0 as u8){n=n+1} sys_write(1,s,n); return 0 }
31func mi_slen(s: *u8) -> i64 { var n: i64=0; while s[n]!=(0 as u8){n=n+1} return n }
35func mi_split(url: *u8, host_out: *u8, path_idx_out: *i64) -> i64
59func mi_pace_ms(ms: i64) -> i64
69func mi_import(page_url: *u8, ukey: *u8, slug: *u8, store: *TrustStore) -> i64
119func main(argc: i64, argv: *i64) -> i64