Entity Gallery

Sovereign entity media pipeline — gather every image a public figure has on the open web, fingerprint it, and answer “where else does this appear, and what else is in that gallery?”

Two organs, both running entirely on our own hardware over our own TLS 1.3 stack: nx_entity_media (gather) and nx_imgsearch (reverse-image, 4-tier). No third-party crawler, no external API, no cloud vision service. This page reports architecture and measurements only — no gathered media is published here; the corpora are private.

Pipeline

Measured — one entity, one seed page

A single image-dense profile page, gathered end to end on the NAS:

101 stored + 10 duplicates = 111 extracted. The partition sums exactly, so nothing was dropped silently.

The corpus itself

Every tile below was decoded by our own decoder — the same code path the fingerprinter uses — then box-average downscaled and composited into a single image. That makes the sheet a decode proof as well as a look at the corpus: a broken decoder yields a hole or noise, never a recognisable thumbnail.

A second corpus of 101 assets exists for another entity and is not shown here: it is adult-industry material, and this is a family domain. It is served from its own directory behind the gallery’s existing authenticated session, never mixed into the public site and never merged into the gallery’s own library index.

Format mix — why this measurement mattered

An earlier build rejected 26 of these as “non-image.” That bucket was never inspected. It turned out to contain no junk at all:

A quarter of the gallery was WebP being thrown away by a sniffer that only recognised JPEG and PNG. Measuring the reject bucket, rather than assuming it was icons and chrome, is what turned a guess into a build target.

Decode coverage — one missing branch

With the corpus in hand, the indexer could read only 12 of the first 64 assets. A per-file probe reported, for every image: is it progressive JPEG, what does the luma decoder return, and what does the RGB decoder return.

Every failure was a progressive JPEG, zero baseline images failed, and all 52 decoded fine through the RGB path. The cause was a single missing branch: the luma decoder called the baseline routine directly, while its RGB sibling — identical up to that one line — branched to the progressive routine. The luma path is the one the indexer actually uses.

Decode coverage went from 187‰ to 1000‰ on that corpus. The robustness ruler was re-run immediately afterwards and came back unchanged, so the change is pure gain rather than a trade.

Reverse-image query

Querying with an image that could not be decoded at all before the fix:

Rank 1 is the exact image. Ranks 2–4 are the similarity tier surfacing genuine look-alikes — the “there is more in this gallery” answer. On a 12-image corpus the same tier returned nothing confident; its silence was corpus size, not a broken ranker. A similarity tier is untestable until the corpus is large enough to have neighbours.

Engine robustness (unchanged by this work)

Measured by an adversary that applies 17 real-world modification classes to a held-out corpus, recall@1 against 256 distractors:

Mirror, rotation, letterbox, and crops to 20% all recover at 1000‰. The one open class is crop-plus-rescale at 843‰.

The advertisement that created the problem

Chasing those 26 undecodable files found something worse than a missing decoder. nx_codec_caps is the single registry that decides which formats we advertise in the HTTP Accept header, on one rule: a format is in the header if and only if a decoder exists. It listed image/webp, justified by a comment claiming “VP8L lossless + VP8 keyframe”.

VP8L is real. VP8 keyframe was not: that module is the boolean entropy decoder and frame-header parser only — no macroblock modes, no coefficient decode, no inverse transform, no reconstruction. A probe over the corpus settled it: 25 of 25 gathered WebP files were lossy VP8, and every one decoded to nothing. Not a single lossless file existed in the wild.

The consequence is the part worth remembering. Because we advertised WebP, content-negotiating origins gave us WebP — a Wikimedia URL ending .jpg came back as a WebP file. The claim we could not honour was manufacturing the very corpus we could not read. Withdrawing it fixed the corpus at its source:

Decoders that existed and nothing called

A GIF decoder and its gate had been in the tree the whole time with zero callers in any image path. Wiring it into both decode paths moved the sheet from 75 to 76 tiles, with the robustness ruler unchanged. That is the fifth capability found built-but-unwired in a single day — the recurring failure here is not missing work, it is unconnected work.

Reporting that was lying

The indexer filtered candidate files by extension before attempting a decode, so unsupported formats were invisible rather than counted, and it printed skipped_undecodable: 0 — which reads as “everything decoded.” Both extensions were admitted deliberately, including the one that still fails, converting an invisible omission into an honest number:

Known gaps — stated, not hidden

• Lossy VP8 WebP still does not decode — 25 files in the sample. Remaining work is the real codec: macroblock modes, coefficient tables, dequantisation, inverse transform, intra prediction, loop filter. The entropy coder and header parser are already built and gated.
• PNG support is partial — no palette, no interlace, no sub-byte bit depth.
• AVIF is entirely undecodable. That slice of the web is invisible rather than badly ranked.
• Seed discovery is structural, not semantic. It keeps outbound links whose URL carries a token of the entity name, which works well for encyclopedic and database pages but will miss a page that never spells the name in its URL. The reverse-image-steered frontier — promote a host once one of its images matches, then harvest its other media — is designed and gate-proven but not yet driven end to end.
• Our PNG encoder writes uncompressed (STORED) zlib, so rendered sheets are far larger than they need to be — a 988×496 sheet exceeded the 1 MiB publish cap and had to be re-rendered smaller. Real deflate in the encoder would remove that ceiling.
• The similarity tier is linear in corpus size. Fine at a hundred images; an approximate-nearest-neighbour index is required before a hundred thousand.
• Recall figures are an upper bound. They are measured against 256 distractors; a production corpus is far larger and ranking gets harder with scale.

Method

Every number here was produced by running the tools and reading their output, not by estimating. Caps are reported rather than applied silently — a run that hits a limit says so. Counts are published as partitions that sum to their total, so a missing item cannot hide in rounding. Where a fix was applied, the benchmark was re-run immediately afterwards to show it did not trade one capability for another.

Nishi — sovereign stack: own TLS 1.3, own HTTP, own image codecs, own search index. No third-party web server, no trackers, no external analytics. Related: reverse-image engine detail.