Robots.txt Generator

Free Online Tools for SEO, Digital Marketing, Domains & more

Robots.txt Generator

Default - All Robots are:  
Sitemap: (leave blank if you don't have) 
Search Robots: Google
  Google Image
  Google Mobile
  MSN Search
  Yahoo MM
  Yahoo Blogs
  DMOZ Checker
  MSN PicSearch
Restricted Directories: The path is relative to root and must contain a trailing slash "/"

Now, Create 'robots.txt' file at your root directory. Copy above text and paste into the text file.

About Robots.txt Generator

Robots.txt is a file that is generally placed in the root folder of a website. The role of this file is to help search engines to give an accurate position for a website on the Search Engine Results Page. Google and other search engines use robots or website crawlers, which go through and review all the content present in a website.

If you want to exclude some pages in a website such as admin pages, you can very well do that. The Robots Exclusion Protocol ensures that robots.txt files exclude pages. This free robot.txt generator tool will easily generate the file for you based on the pages you want to exclude.