Home › Forums › Developing Websites › Search Engine Optimization › Onpage SEO › Robots.txt file › Re: Re: Robots.txt file
April 25, 2013 at 11:39 am
#33702
Anonymous
The robots.txt file tells search engines which directories to crawl and which not to. You can use it to block crawlers from looking at your image directory if you don't want your images showing up on Google search.Be careful not to use this to try and block people from directories you want to keep secret. Anyone can view you robots.txt file. Make sure you password protect directories that need to be secured.