Robots.txt Generator
Build a robots.txt file that tells search engines and AI crawlers which parts of your site to index. Use a preset or compose custom rules.
Quick presets
User-agent #1
Site directives
User-agent: * Disallow: /admin/ Disallow: /api/ Disallow: /private/ Allow: / Sitemap: https://example.com/sitemap.xml
Agents
1
Rules
4
Size
119B
Place robots.txt at your site root (https://yoursite.com/robots.txt). Empty Disallow: means allow all.