🤖

Robots.txt Generator

Control which bots can crawl your site. Block AI scrapers, protect private paths.

Allow all search engine crawlers

Recommended for most public websites

Block specific bots

GPTBot (OpenAI) Blocks ChatGPT training crawler

CCBot (Common Crawl) Blocks AI training data scrapers

Bingbot Blocks Microsoft Bing crawler

Disallow paths

One path per line. Applied to all crawlers unless blocked above.

Sitemap URL

Generating…

🤖

Your robots.txt will appear here

Configure options and click generate