Root Files Pass $1.95 a month or $19.50 a year Free during preview

What it adds

/test

Test a robots.txt.

Paste any robots.txt, pick a crawler and a path. The answer follows RFC 9309, the robots.txt standard: the crawler's own group (or *), longest matching rule, Allow wins a tie. Nothing is fetched and nothing is kept.

01 The robots.txt
02 The visit

GPTBot asks for /menu

Blocked

Deciding rule
Disallow: / (line 11)
Group used
User-agent: gptbot, claudebot, google-extended, applebot-extended, ccbot, meta-externalagent

A Blocked answer means a well-behaved crawler will not fetch that address. It does not stop a person, or a crawler that ignores robots.txt.