When search engine crawlers (robots) look at a website, the first file they will look at is not your index.html or index.php page. It is your robots.txt file.

Posted on Wednesday September 10th, 2008 at 12:11pm in Web Design