Free tool
Generator for robots.txt
Define the rules per crawler and get a valid robots.txt to drop at the root of your site. Nothing is sent to a server: everything is generated in your browser.
Generated robots.txt
User-agent: * Disallow:
Free tool
Define the rules per crawler and get a valid robots.txt to drop at the root of your site. Nothing is sent to a server: everything is generated in your browser.
User-agent: * Disallow:
This site only uses cookies that are necessary for it to work, including one to remember that you have read this message. We also measure our traffic with a tool we host ourselves, without cookies and without storing your IP address. No advertising, no third-party trackers. Learn more