AI Crawler Checker
See which of 22 AI crawlers, from GPTBot to PerplexityBot, your robots.txt allows or blocks.
Build a root /llms.txt that points AI assistants at your best pages, then validate the Markdown before you ship it.
or fill it in by hand
How it works
Enter your site name and a one-line summary of what it is.
Add sections (Docs, Products, Guides…) with links and short notes, or import pages from your sitemap.
Download llms.txt and upload it to your site's root, so it's served at https://yoursite.com/llms.txt.
Fill in sections and links, or edit the Markdown directly and validate it.
Pull page URLs from your sitemap and group them into sections automatically.
Checks the H1, summary, sections and link syntax with line numbers.
Your draft stays in this browser until you clear it.
FAQ
At the root of your domain, so it's available at https://example.com/llms.txt, served as plain text or Markdown.
No one has promised that. Some AI tools and developer platforms read llms.txt, but the major AI search engines haven't said they use it for ranking. It's cheap to add and helps the tools that do read it.
robots.txt tells crawlers what they may access. llms.txt is a curated guide to your most useful content. They work together: llms.txt can't grant access that robots.txt blocks.
A companion file some sites publish with the full text of their key pages in one Markdown file. This tool focuses on llms.txt; for full text, the AI Readiness Checker can extract a page as Markdown.
Missing something?
Describe the job and the file types involved. The most requested tools get built first.