2 Commits

Author	SHA1	Message	Date
Vikhyat Korrapati	e3702ecb30	Improved crawler detection: add Twitterbot, Facebook, curl, Bing, Baidu.	2014-03-16 19:30:20 +05:30
Robin Ward	c4b5455c21	REFACTOR: Rename GooglebotDetection to CrawlerDetection because we ... will likely whitelist more crawlers in the future.	2014-02-20 16:07:02 -05:00