Robots.txt Generator

Build a valid robots.txt file in seconds. Control which crawlers can access your site, block private folders, set a crawl delay and add your sitemaps.

Quick Presets

Rules

Sitemaps & Host

Sitemap URLs (one per line)

Must be full absolute URLs.

Host (optional, Yandex only)

Your robots.txt

How to use

Choose a preset or add your own rule groups.
Download the file and name it exactly robots.txt.
Upload it to the root of your domain, e.g. example.com/robots.txt.
Test it in Google Search Console.

Copied to clipboard