Web This site Trusted
1 result(s) for “okapi bm25 site terms com pp site github com site lemmy world site:mcp.directory” · 485 ms
Scope: Nishi’s own index (our crawl + Common Crawl) — not the whole live web. Some sites block all automated crawlers (anti-bot walls), so their pages may be absent here even though we never filter lawful content.
Freshness: no result on this page carries a fetch time yet (indexed before the fetch epoch was stored beside the doc)
The five endpoints cover the four jobs an agent actually does — /scrape for a single URL, /crawl for a whole site, /map for sitemap discovery, /extract for structured JSON via an LLM, and /search for query-then-fetch — and the markdown output drops straight into a context window without the chrome, cookie banners
from the top result below
https://mcp.directory/blog/firecrawl-vs-anycrawl-vs-crawlee-vs-playwright-2026
The five endpoints cover the four jobs an agent actually does — /scrape for a single URL, /crawl for a whole site, /map for sitemap discovery, /extract for structured JSON via an LLM, and /search for query-then-fetch — and the markdown output drops straight into a context window without the chrome, cookie banners