SEO Tools Evaluation

My website hosting company recommended I include a “Robots.txt” file in my site. (The SEO tools from my website company is an automated evaluation of my website so nobody to ask.) What should be in this file and how do I create it? Is it just another page that’s not included in the navigation bar?

It’s a plain text file just google it and add the options you need.

Unless you are wanting to “request” Search engines to exclude index a part of the website then you don’t need one

From Google:

Does my website need a robots.txt file?
No. When Googlebot visits a website, we first ask for permission to crawl by attempting to retrieve the robots.txt file. A website without a robots.txt file, robots meta tags or X-Robots-Tag HTTP headers will generally be crawled and indexed normally.


This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.