Robots.txt Generator

Control which bots can crawl your site. Block AI scrapers, protect private paths.

Built by Mubashar Iqbal — SEO specialist & web developer with 7+ years experience

Allow all search engine crawlers Recommended for most public websites

Block specific bots

GPTBot (OpenAI) Blocks ChatGPT training crawler

CCBot (Common Crawl) Blocks AI training data scrapers

Bingbot Blocks Microsoft Bing crawler

Disallow paths

One path per line. Applied to all crawlers unless blocked above.

Sitemap URL

Generating…

Configure options and click generate