# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file # See http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=40367 # disallowing crawl of urls with filter qs params, since every possible url like # this was getting crawled: # /people/aaron?inta_source_id=29%2C4%2C8 User-agent: * Disallow: /*?*inta_source_id=* Disallow: /*?*inta_type_id=* Disallow: /*?*buddy_id=* Disallow: /*?*favorites=*