Root Files Pass $1.95 a month or $19.50 a year Free during preview

What it adds

/ the root of your website

The small files at the top of your website, written right.

robots.txt tells crawlers what to skip. sitemap.xml lists the pages you want found. security.txt tells people where to report a problem. Type a few answers and Root Files writes all three, checks them against the published rules, and lets you test any robots.txt before it goes live.

Make your root files Test a robots.txt

Free to make, check, test and copy. Root Files Pass $1.95 a month or $19.50 a year for 25 saved sites, big sitemaps, the ZIP and a renewal reminder. Free during preview

www.bakery.example.com/
  • .well-known/
    • security.txtwhere to report a problem
  • assets/
  • index.html
  • robots.txtwhat crawlers should skip
  • sitemap.xmlevery page you want found

robots.txt

# AI-training crawlers, asked to stay out:
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: Google-Extended
User-agent: Applebot-Extended
User-agent: CCBot
User-agent: meta-externalagent
Disallow: /

User-agent: *
Disallow: /cart/
Disallow: /checkout/
Disallow: /search

/three-files

Three files. Three readers. Three ways to get it wrong.

robots.txt

Read by search and AI crawlers

One stray Disallow: / left over from a staging copy and a shop vanishes from search. Root Files writes it from plain choices, lists the AI-training crawlers by the names their operators publish, and warns about the classic mistakes.

sitemap.xml

Read by search engines

Paste your page addresses. Duplicates, typos and addresses on another host are taken out and listed, special characters are escaped the way the sitemap format requires, and a date can ride along with each page.

.well-known/security.txt

Read by people who find a problem

The standard (RFC 9116) way to say where a security report should go. It needs a Contact line and an Expires date, and it goes stale when that date passes. Root Files writes both and can put the renewal in your calendar.

/test

Know what a crawler will do before it does it.

The tester reads a robots.txt the way the current standard (RFC 9309) says crawlers should: it finds the group for that crawler, falls back to *, picks the longest matching rule, lets Allow win a tie, and understands * and $. It shows the rule that decided.

Open the tester

Googlebot asks for /menu

Allowed no rule matches

Googlebot asks for /checkout/pay

Blocked by Disallow: /checkout/

GPTBot asks for /menu

Blocked by Disallow: /

/how

From answers to files in three steps.

  1. 01

    Answer

    Your site's address, the paths crawlers can skip, whether AI-training crawlers are welcome, who takes security reports, and your page list.

  2. 02

    Check

    Every answer is checked against the published formats. Problems are marked "fix", things worth knowing "note".

  3. 03

    Put them up

    Copy or download each file and place it at your site's root. The guide shows where for common hosts.

/pricing

Plans

What you get Free$0 Root Files Pass$1.95/monthor $19.50 a year
Make, check and copy robots.txt, sitemap.xml, security.txtUnlimitedUnlimited
Test any robots.txt against any crawler and pathUnlimitedUnlimited
Download each file on its ownYesYes
Addresses in one sitemap50010,000
Addresses kept in each saved set (the full sitemap is still made)1,0001,000
Site sets kept in your private-link Book125, with duplicate
ZIP of all three files, in the right folders, with a how-toNoYes
Calendar reminder before security.txt expires (.ics)NoYes

Free during preview. Checkout is not open yet, so nothing can be charged: every Pass feature is on for everyone until it opens. These are the prices it will open with. When it opens, payment is through PayPal and the Pass can be cancelled there at any time.

Straight answers