support@ultraseotools.com

Robots.txt Generator


Default - All Robots are:  
    
Crawl-Delay:
    
Sitemap: (leave blank if you don't have) 
     
Search Robots: Google
  Google Image
  Google Mobile
  MSN Search
  Yahoo
  Yahoo MM
  Yahoo Blogs
  Ask/Teoma
  GigaBlast
  DMOZ Checker
  Nutch
  Alexa/Wayback
  Baidu
  Naver
  MSN PicSearch
   
Restricted Directories: The path is relative to root and must contain a trailing slash "/"
 
 
 
 
 
 
   



Now, Create 'robots.txt' file at your root directory. Copy above text and paste into the text file.


About Robots.txt Generator

When search engines crawl a site, they initially search for a robots.txt file at the domain root. Search Engines scan robots.txt file for list of directives to determine which files or directories, if any, are blocked from crawling. Robot.txt file can be generated with a Robots.txt Generator. Robots.txt file generated by Robots.txt directs Google and other search engines to find out targeted pages that are to be omitted. It is important to note that robot.txt file generated by Robot.txt Generator works in contrary to sitemap, which is created to encompass files, subjected to include in search.

 

WHY TO USE UTLRASEOTOOLS FREE ROBOTS.TXT GENERATOR

Creating robot.txt file is a difficult task. UltraSEOTools equips its customers with free Robots.txt Generator that will generate robots.txt file for them and absolutely free. You can easily generate and edit your robots.txt file with our free Robots.txt Generator.

 

HOW TO USE UTLRASEOTOOLS FREE ROBOTS.TXT GENERATOR

You can use Robots.txt Generator of UltraSEOTools by following given steps:

  • Click Home button on ultraseotools.com and click on Robots.txt Generator icon that will lead you to facility.
  • Choose Allowed or Refused from dropdown menu that says “Default – All Robots are:” This option will either allow or refuse all robots (software developed and based on artificial intelligence) of search engines.
  • Choose delay time for “Crawl-Delay:” dropdown menu.
  • You can include your website sitemap (list of websites that are to be included in search) in text box that says “Sitemap” or you can leave it blank.
  • You can specifically allow robots or refuse robots or leave them as default for following search engines

          Google

          Google Image

          Google Mobile

          MSN Search

          Yahoo

          Yahoo MM

          Yahoo Blogs

          Ask/Teoma

          GigaBlast

          DMOZ Checker

          Nutch

          Alexa etc

  • Mention restricted directories in text box that says “Restricted Directories”.
  • Press Create Robots.txt button. Ultimately you will get your robots.txt file in blank box. Copy it and past in your HTML file.