# Robots.txt file Generic for Frontpage.. # Chris Mendla # Updated Dec 5 2002 # updated aug 16 2005 # updated apr 9 2006 - checked to see if we were blocking googlebot # Allow all spiders access to HTTPDOCS and HTTPSDOCS # but block them from the webstat directory User-agent: * Disallow: /httpdocs/webstat/ Disallow: webstat/ Disallow: /webstat/ Disallow: /cgi-bin/ Disallow: /_borders Disallow: /_derived Disallow: /_fpclass Disallow: /_overlay Disallow: /_private Disallow: /_themes Disallow: /_vti_bin Disallow: /_vti_cnf Disallow: /_vti_log Disallow: /_vti_map Disallow: /_vti_pvt Disallow: /_vti_txt Disallow: /revolver Disallow: /flyspeck