#################### # Everett Community College Robots.txt # # IT Web Services # v 1.2 # # Last Updated: 01/22/2010 15:56 # # Block List: # If you are found on the block list, we may have # removed you due to non-compliance with the # robots.txt specification # http://www.robotstxt.org/orig.html # http://www.robotstxt.org/norobots-rfc.txt #################### User-agent: * Disallow: /js/ Disallow: /_inc/ Disallow: /athletics/_inc/ Disallow: /com.cfm # no groups Disallow: /search/ Disallow: /directory/ Disallow: /catalog/ Disallow: /classes/ # no apps Disallow: /emergency/notification/ Disallow: /resources/password/ Sitemap: http://www.everettcc.edu/sitemapxml.cfm #Request-rate: 1/2 # maximum rate is one page every 2 seconds #Crawl-delay: 10 #Visit-time: 0600-0845 # only visit between 06:00 and 08:45 UTC (GMT) #### disallow bot list -- updated: 1/28/2009 # http://en.wikipedia.org/wiki/Robots_exclusion_standard