robots.txt is a plain-text file at the root of every website that tells search engine crawlers which pages and sections they are allowed (or not allowed) to access. It's the first thing Googlebot, Bingbot, and other bots look for when they visit a domain.

Common directives include User-agent (which crawler the rules apply to), Allow (explicitly permit a path), Disallow (block a path), and Sitemap (point to your XML sitemap).

Why check it? A misconfigured robots.txt can accidentally block important pages from being indexed — tanking your SEO. It can also leak admin URLs or fail to declare a sitemap. This tool fetches the live robots.txt, parses all directives, and flags common issues so you can fix them fast.