Thread: Googles hidden files
View Single Post
(#3)
Old
Heff is Offline
Super Moderator
Heff is on a distinguished road
  
Posts: 482
Join Date: Jun 2006
Location: USA
Send a message via AIM to Heff
Default 08-05-2006, 09:28 PM
Quote:
Originally Posted by YOMOMMAGOTBEEF
Just snooping around and found google's robot file. http://google.com/robots.txt
You are an idiot, everyone has a robots file to tell what the robots should crawl

DaTrueHeff
Reply With Quote