Home › Seo Tools › Robots.txt Generator

Free Robots.txt Generator — Build a Valid robots.txt in Seconds

Add per-bot allow/disallow rules, set a crawl delay, and list your sitemaps — then copy or download a clean, search-engine-ready robots.txt file.

1. Crawl rules (one row per rule)

User agentRule typePath

2. Optional settings

Live preview — your robots.txt 
# Your robots.txt will appear here as you type.
100% private

Everything runs in your browser — your rules are never sent to any server.

No signup

No account, no email, no paywall. Open the page and start building.

Free forever

Generate and download unlimited robots.txt files at no cost.

How to use

  1. 1Add crawl rules

    Type a bot name (or * for all bots), choose Allow or Disallow, and enter the path. Use the + Add rule button for more rows.

  2. 2Set delay and sitemap

    Optionally add a crawl delay in seconds and your sitemap URL so crawlers find it automatically.

  3. 3Review the preview

    The robots.txt preview updates live as you type — check that every rule looks right before copying.

  4. 4Copy or download

    Hit Copy to grab the text, or Download robots.txt to save the file, then upload it to your site’s root folder.

Features

Per-bot rules

Target Googlebot, Bingbot, or any crawler by name — or use * to cover every bot at once.

Allow and Disallow

Block whole sections like /admin/ while still permitting public assets such as /images/.

Unlimited rows

Add as many rule rows as your site needs, grouped automatically by user agent.

Crawl delay

Ask aggressive bots to slow down so they don’t overload a small server.

Sitemap directive

Append a Sitemap line so search engines discover your sitemap without guessing.

Live validation

The tool warns you if you try to export an empty file with no rules or sitemap.

Frequently asked questions

What is a robots.txt file?
A robots.txt file sits in your website’s root folder (for example https://example.com/robots.txt) and tells search engine crawlers which pages or sections they may or may not visit. It’s a public instruction file, not a security measure.
Does robots.txt keep pages out of Google?
Mostly, but not always. Well-behaved crawlers like Googlebot honor Disallow rules. To truly hide a page, use noindex meta tags or password protection instead of relying on robots.txt alone.
What does User-agent: * mean?
The asterisk is a wildcard matching every crawler. Rules under it apply to all bots unless a bot has its own more specific group earlier in the file.
What is crawl-delay for?
It asks crawlers to wait a number of seconds between requests. It’s useful on small or shared hosting, but note that Googlebot ignores crawl-delay — use Google Search Console’s crawl rate settings for Google.
Should I add my sitemap to robots.txt?
Yes — the Sitemap directive helps search engines discover your sitemap without you submitting it manually. Most CMS platforms and sitemap tools rely on this line.
Where do I upload the file after downloading?
Place it at the very root of your domain (https://yourdomain.com/robots.txt). It must be named exactly robots.txt, lowercase, at the domain root — a copy in a subfolder will be ignored.
Is my data private?
Yes. This tool runs entirely in your browser. Your rules, paths, and sitemap URL never leave your device.

Related tools

XML Sitemap Generator

Turn a URL list into a valid sitemap.xml for search engines.

 JSON Formatter & Validator

Format, validate, and inspect JSON with one click.

 Password Generator

Create strong, random passwords for your accounts and servers.

 QR Code Generator

Make free QR codes for links, Wi-Fi, and contact cards.

 Word Counter

Count words, characters, and reading time for your SEO content.
Scroll to Top