llms.txt
A community convention (not a formal web standard) for a plain-Markdown file at a site's /llms.txt root, giving AI agents a curated index of the site's most important pages.
An llms.txt file starts with an H1 site name and a short blockquote description, followed by ## sections listing [Title](url): note links — effectively an LLM-optimized table of contents that's cheaper for an agent to consume than crawling the whole site. It complements, rather than replaces, robots.txt/sitemap.xml (for crawlers) and schema.org JSON-LD (for structured extraction). Crail publishes /llms.txt and a fuller /llms-full.txt export alongside JSON feeds under /api/.
Last verified: