Warning: all recent Wikimedia traffic analysis reports have been generated from old scripts.

The scripts are orphaned, and have not been maintained for at least 6 months. Many bugs are considerably older.
Known Bugzilla bugs: 46190, 46191, 46195, 46201, 46205, 46265, (46267), 46268, 46269, 46271, 46273, 46274, 46275, 46277, 46278, 46279, 46289

 See also: Requests by destination or by origin / Methods / Scripts / User agents / Skins / Crawlers / Op.Sys. / Mobile devices / Browsers / Google / Country data / Traffic trends, and notes about reliability of these data

The following overview of crawler (aka bot) page requests is based on the user agent information that accompanies most server requests. Unfortunately this user agent information follows rather loosely defined guidelines. Also please bear in mind than the most popular crawler names may be somewhat overrepresented. This is the result of so called user agent spoofing (where a requester supplies false credentials, e.g. to bypass web servers filters). GoogleBot seems to be a favorite for spoofing. Therefore requests from an ip address registered by Google (see below) are color coded GoogleBot, others GoogleBot For this report page requests are considered to be issued by a crawler in two cases: 1 The user agent string contains a web address (only crawlers should have that, but there a some false positives, where a browser sends a user agent string with a web address (ill behaved plug-in, main offenders have been eliminated) 2 The user agent string contains the term bot, spider or crawl[er]' In total 97,238,670 page requests (mime type text/html only!) per day are considered crawler requests, out of 631,451,330 external requests, which is 15.4%
Page requests for crawlers that specify a url in the agent string
Count x 1000	Secondary domain (~site) name	URL	Mime type	User agent
	google
		www.google.com/bot.html	text/..	Mozilla/5.0 (compatible; GoogleBot/2.1; url) - -
		www.google.com/bot.html	text/..	Mozilla/5.0 (iPhone; CPU iPhone OS 4_1 like Mac OS X; en-us) AppleWebKit/532.9 KHTML Version/4.0.5 Mobile/8B117 Safari/6531.22.7 (compatible; GoogleBot-Mobile/2.1; url) - -
		www.google.com/bot.html	-	Mozilla/5.0 (iPhone; CPU iPhone OS 4_1 like Mac OS X; en-us) AppleWebKit/532.9 KHTML Version/4.0.5 Mobile/8B117 Safari/6531.22.7 (compatible; GoogleBot-Mobile/2.1; url) - -
		www.google.com/bot.html	-	Mozilla/5.0 (compatible; GoogleBot/2.1; url) - -
		code.google.com/p/crawler4j/	text/..	crawler4j (url) - -
		desktop.google.com/	application/xml	Mozilla/5.0 (compatible; Google Desktop/5.9.1005.12335; url) - -
		www.google.com/bot.html	image/..	Mozilla/5.0 (compatible; GoogleBot/2.1; url) - -
		www.google.com/feedfetcher.html	image/..	Mozilla/5.0 (compatible) FeedFetcher-Google; (url) - -
		desktop.google.com/	image/..	Mozilla/5.0 (compatible; Google Desktop/5.9.1005.12335; url) - -
		www.google.com/feedfetcher.html	-	FeedFetcher-Google; (url) - -
		www.google.com/feedfetcher.html	application/xml	FeedFetcher-Google; (url) - -
		www.google.com/feedfetcher.html	text/..	Mozilla/5.0 (compatible) FeedFetcher-Google; (url) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: wikien3) - -
		desktop.google.com/	text/..	Mozilla/5.0 (compatible; Google Desktop/5.9.1005.12335; url) - -
		www.google.com/feedfetcher.html	text/..	FeedFetcher-Google; (url) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: s~kyouen3) - -
		code.google.com/appengine	application/json	AppEngine-Google; (url; appid: s~redconceptual) - -
		www.google.com/feedfetcher.html	application/json	Mozilla/5.0 (compatible) FeedFetcher-Google; (url) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: ortografia4) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: rarplayer) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: ortopedianew) - -
		code.google.com/appengine	text/..	Acre/dev/53:893 staging.freebase-refinery.appspot.com AppEngine-Google; (url; appid: s~freebase-refinery) - -
		www.google.com/feedfetcher.html	application/xml	Mozilla/5.0 (compatible) FeedFetcher-Google; (url) - -
		code.google.com/appengine	text/..	Mozilla/5.0 (Windows; Windows NT 5.1; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 AppEngine-Google; (url; appid: s~fonetika3) - -
		code.google.com/appengine	text/..	WikiBot/0.1 AppEngine-Google; (url; appid: newikipedia) - -
		docs.google.com	image/..	Mozilla/5.0 (compatible; GoogleDocs; apps-presentations; url) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: wikien4) - -
		docs.google.com	image/..	Mozilla/5.0 (compatible; GoogleDocs; documents; url) - -
		code.google.com/appengine	application/xml	AppEngine-Google; (url; appid: wikipedia-raw) - -
		www.google.com/bot.html	text/..	DoCoMo/2.0 N905i(c100;TB;W24H16) (compatible; GoogleBot-Mobile/2.1; url) - -
		www.google.com/bot.html	text/..	SAMSUNG-SGH-E250/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 UP.Browser/6.2.3.3.c.1.101 (GUI) MMP/2.0 (compatible; GoogleBot-Mobile/2.1; url) - -
		www.google.com/bot.html	text/..	GoogleBot/2.1 (url) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: d24-img) - -
		www.google.com/feedfetcher.html	text/..	Mozilla/5.0 (compatible) FeedFetcher-Google;(url) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: s~sancampoen) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: s~francetiki) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: s~batshiitinsane) - -
		www.google.com/bot.html	-	DoCoMo/2.0 N905i(c100;TB;W24H16) (compatible; GoogleBot-Mobile/2.1; url) - -
		code.google.com/appengine	text/..	Python-urllib/2.5 AppEngine-Google; (url; appid: s~isnt-it) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: boxapp) - -
		code.google.com/appengine	text/..	Wiki.java 0.27 AppEngine-Google; (url; appid: wikipediatools) - -
		www.google.com/coop/cse/cref	text/..	FeedFetcher-Google-CoOp; (url) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: usawebdl) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: s~italiatiki) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: s~kasumiremix) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: s~espanatiki) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: threewiki) - -
		code.google.com/appengine	text/..	www.productontology.org/1.0 (Contact: martin.heppATunibw.de) AppEngine-Google; (url; appid: gr4bing) - -
		www.google.com/bot.html	image/..	Mozilla/5.0 (iPhone; CPU iPhone OS 4_1 like Mac OS X; en-us) AppleWebKit/532.9 KHTML Version/4.0.5 Mobile/8B117 Safari/6531.22.7 (compatible; GoogleBot-Mobile/2.1; url) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: s~japantiki) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: s~app3123ak) - -
		desktop.google.com/	image/..	Mozilla/5.0 (compatible; Google Desktop/5.9.911.3589; url) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: my-api) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: toom16-10) - -
		code.google.com/appengine	image/..	AppEngine-Google; (url; appid: d24-img) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: worldwide-propaganda) - -
		docs.google.com	image/..	Mozilla/5.0 (compatible; GoogleDocs; drawings; url) - -
		code.google.com/appengine	application/json	MWBOT GAE Edition AppEngine-Google; (url; appid: philip-bot) - -
		code.google.com/appengine	image/..	AppEngine-Google; (url; appid: surf6003) - -
		code.google.com/appengine	text/..	Acre/dev/51:889 ubiquity.freebaseapps.com AppEngine-Google; (url; appid: s~freebaseapps) - -
		www.google.com/bot.html	-	SAMSUNG-SGH-E250/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 UP.Browser/6.2.3.3.c.1.101 (GUI) MMP/2.0 (compatible; GoogleBot-Mobile/2.1; url) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: s~ninjallamastuff) - -
		code.google.com/appengine	application/json	Mozilla/5.0 AppEngine-Google; (url; appid: s~redconceptual) - -
		code.google.com/appengine	application/json	User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.5 KHTML Chrome/19.0.1084.56 Safari/536.5 AppEngine-Google; (url; appid: s~in-onda) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: bassgnt) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: khrixy) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: dustbunnytycoonmonitor) - -
		code.google.com/appengine	text/..	oohEmbed.com AppEngine-Google; (url; appid: vipoembed) - -
		www.google.com/coop/cse/cref	text/..	PageFetcher-Google-CoOp;((url) - -
		code.google.com/appengine	text/..	Mozilla/5.0 AppEngine-Google; (url; appid: s~vodio-app) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: s~win8wale) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: s~theunblock) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: s~drizzlprox) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: ridemyhell) - -
		code.google.com/appengine	text/..	mail address AppEngine-Google; (url; appid: s~wiki-sherpa) - -
		desktop.google.com/	-	Mozilla/5.0 (compatible; Google Desktop/5.9.1005.12335; url) - -
		www.google.com/feedfetcher.html	image/..	FeedFetcher-Google; (url) - -
		code.google.com/appengine	text/..	Python-urllib/2.7 AppEngine-Google; (url; appid: s~hr-pulsesubscriber) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: beansacks) - -
		code.google.com/appengine	-	AppEngine-Google; (url; appid: s~ninjallamastuff) - -
		code.google.com/appengine	text/..	AppEngine-Google; (url; appid: s~proxyseekkety) - -
		code.google.com/appengine	image/..	AppEngine-Google; (url; appid: s~batshiitinsane) - -
	bing
		www.bing.com/bingbot.htm	text/..	Mozilla/5.0 (compatible; bingbot/2.0; url) - -
		www.bing.com/bingbot.htm	-	Mozilla/5.0 (compatible; bingbot/2.0; url) - -
		www.bing.com/bingbot.htm	image/..	Mozilla/5.0 (compatible; bingbot/2.0; url) - -
		www.bing.com/bingbot.htm	text/..	Mozilla/5.0 (compatible; bingbot/2.0; url) -
		www.bing.com/bingbot.htm	text/..	Mozilla/5.0 (compatible; bingbot/2.0; url) SitemapProbe - -
		www.bing.com/bingbot.htm	text/..	Mozilla/5.0 (compatible; bingbot/2.0; url) AppEngine-Google; (http://code.google.com/appengine; appid: proxydisk9) - -
		www.bing.com/bingbot.htm	text/..	Mozilla/5.0 (compatible; bingbot/2.0; url) AppEngine-Google; (http://code.google.com/appengine; appid: proxydisk) - -
		www.bing.com/bingbot.htm	text/..	Mozilla/5.0 (compatible; bingbot/2.0; url) AppEngine-Google; (http://code.google.com/appengine; appid: proxypython7) - -
		www.bing.com/bingbot.htm	text/..	Mozilla/5.0 (compatible; bingbot/2.0; url) AppEngine-Google; (http://code.google.com/appengine; appid: yourrevenues) - -
	facebook
		www.facebook.com/externalhit_uatext.php	image/..	facebookexternalhit/1.1 (url) - -
		www.facebook.com/externalhit_uatext.php	text/..	facebookexternalhit/1.1 (url) - -
		www.facebook.com/externalhit_uatext.php	text/..	facebookexternalhit/1.0 (url) - -
		www.facebook.com/externalhit_uatext.php	image/..	facebookexternalhit/1.1 (url) -
		www.facebook.com/externalhit_uatext.php	text/..	facebookexternalhit/1.1 (url) -
		www.facebook.com/externalhit_uatext.php	text/..	facebookexternalhit/1.0 (url) -
		www.facebook.com/externalhit_uatext.php	image/..	facebookexternalhit/1.0 (url) - -
		developers.facebook.com	image/..	facebookplatform/1.0 (url) - -
		www.facebook.com/externalhit_uatext.php	image/..	facebookexternalhit/1.0 (url) -
		developers.facebook.com	image/..	facebookplatform/1.0 (url) -
	google?
		www.google.com/bot.html	text/..	Mozilla/5.0 (compatible; GoogleBot/2.1; url) - -
		www.google.com/bot.html	text/..	Mozilla/5.0 (iPhone; CPU iPhone OS 4_1 like Mac OS X; en-us) AppleWebKit/532.9 KHTML Version/4.0.5 Mobile/8B117 Safari/6531.22.7 (compatible; GoogleBot-Mobile/2.1; url) - -
		www.google.com/bot.html	text/..	GoogleBot/2.1 (url) - -
		www.google.com/bot.html	image/..	Mozilla/5.0 (compatible; GoogleBot/2.1; url) - -
		www.google.com/bot.html	-	Mozilla/5.0 (compatible; GoogleBot/2.1; url) - -
		www.google.com/bot.html	text/..	GoogleBot/2.1 (url) en-us,en-gb,en;q=0.7,*;q=0.3 -
		www.google.com/bot.html	application/json	Mozilla/5.0 (compatible; GoogleBot/2.1; url) - -
		www.google.com/bot.html	-	Mozilla/5.0 (iPhone; CPU iPhone OS 4_1 like Mac OS X; en-us) AppleWebKit/532.9 KHTML Version/4.0.5 Mobile/8B117 Safari/6531.22.7 (compatible; GoogleBot-Mobile/2.1; url) - -
		www.google.com/bot.html	image/..	Mozilla/5.0 (compatible; GoogleBot/2.1; url) en-US,en;q=0.8 -
		www.google.com/bot.html	text/..	Mozilla/5.0 (compatible; GoogleBot/2.1; url) en-US,en;q=0.8 -
		www.google.com/bot.html	text/..	GoogleBot/2.1 (url) en-us,en;q=0.5 -
		www.google.com/bot.html	text/..	DoCoMo/2.0 N905i(c100;TB;W24H16) (compatible; GoogleBot-Mobile/2.1; url) - -
		www.google.com/bot.html	image/..	Mozilla/5.0 (compatible; GoogleBot/2.1; url) en-US,en;q=0.5 -
	baidu
		www.baidu.com/search/spider.html	text/..	Mozilla/5.0 (compatible; Baiduspider/2.0; url) zh-cn,zh-tw -
		www.baidu.com/search/spider.html	text/..	Mozilla/5.0 (compatible; Baiduspider/2.0; url) en-US -
		www.baidu.com/search/spider.html	text/..	Mozilla/5.0 (compatible; Baiduspider/2.0; url) - -
		www.baidu.com/search/spider.html	-	Mozilla/5.0 (compatible; Baiduspider/2.0; url) zh-cn,zh-tw -
		www.baidu.com/search/spider.html	-	Mozilla/5.0 (compatible; Baiduspider/2.0; url) en-US -
		www.baidu.com/search/spider.html	text/..	Mozilla/5.0 (compatible; Baiduspider/2.0; url) ja-JP,ja -
		www.baidu.com/search/spider.htm	image/..	Baiduspider-image(url) - -
		www.baidu.com/search/spider.htm	text/..	Baiduspider-image(url) - -
		www.baidu.com/search/spider.html	text/..	Mozilla/5.0 (Linux;u;Android/2.3.7;zh-cn;) AppleWebKit/533.1 (KHTML,like Gecko) Version/4.0 Mobile Safari/533.1 (compatible; url) - -
		www.baidu.com/search/spider.html	-	Mozilla/5.0 (compatible; Baiduspider/2.0; url) ja-JP,ja -
		www.baidu.com/search/spider.html	application/xml	Mozilla/5.0 (compatible; Baiduspider/2.0; url) zh-cn,zh-tw -
		www.baidu.com/search/spider.html	image/..	Mozilla/5.0 (compatible; Baiduspider/2.0; url) zh-cn,zh-tw -
		www.baidu.com/search/spider.html	application/xml	Mozilla/5.0 (compatible; Baiduspider/2.0; url) en-US -
		www.baidu.com/search/spider.html	application/json	Mozilla/5.0 (compatible; Baiduspider/2.0; url) - -
	yandex
		yandex.com/bots	text/..	Mozilla/5.0 (compatible; YandexBot/3.0; url) ru, uk;q=0.8, be;q=0.8, en;q=0.7, *;q=0.01 -
		yandex.com/bots	text/..	Mozilla/5.0 (compatible; YandexBot/3.0; url) en-us, en;q=0.7, *;q=0.01 -
		yandex.com/bots	text/..	Mozilla/5.0 (compatible; YandexImages/3.0; url) - -
		yandex.com/bots	text/..	Mozilla/5.0 (compatible; YandexBot/3.0; url) - -
		yandex.com/bots	-	Mozilla/5.0 (compatible; YandexBot/3.0; url) ru, uk;q=0.8, be;q=0.8, en;q=0.7, *;q=0.01 -
		yandex.com/bots	text/..	Mozilla/5.0 (compatible; YandexDirect/3.0; url) ru, uk;q=0.8, be;q=0.8, en;q=0.7, *;q=0.01 -
		yandex.com/bots	image/..	Mozilla/5.0 (compatible; YandexImageResizer/2.0; url) - -
		yandex.com/bots	text/..	Mozilla/5.0 (compatible; YandexImages/3.0; url) ru, uk;q=0.8, be;q=0.8, en;q=0.7, *;q=0.01 -
		yandex.com/bots	-	Mozilla/5.0 (compatible; YandexBot/3.0; url) en-us, en;q=0.7, *;q=0.01 -
		yandex.com/bots	image/..	Mozilla/5.0 (compatible; YandexImages/3.0; url) - -
		yandex.com/bots	image/..	Mozilla/5.0 (compatible; YandexImages/3.0; url) ru, uk;q=0.8, be;q=0.8, en;q=0.7, *;q=0.01 -
		yandex.com/bots	text/..	Mozilla/5.0 (compatible; YandexBot/3.0; url) de, en;q=0.7, *;q=0.01 -
		yandex.com/bots	text/..	Mozilla/5.0 (compatible; YandexDirect/3.0; url) en-us, en;q=0.7, *;q=0.01 -
		yandex.com/bots	text/..	Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; url) ru, uk;q=0.8, be;q=0.8, en;q=0.7, *;q=0.01 -
		yandex.com/bots	text/..	Mozilla/5.0 (compatible; YandexImages/3.0; url) en-us, en;q=0.7, *;q=0.01 -
		yandex.com/bots	application/xml	Mozilla/5.0 (compatible; YandexBlogs/0.99; robot; B; url)1 readers - -
		yandex.com/bots	text/..	Mozilla/5.0 (compatible; YandexImageResizer/2.0; url) - -
	msn
		search.msn.com/msnbot.htm	text/..	msnbot-media/1.1 (url) - -
		search.msn.com/msnbot.htm	image/..	msnbot-media/1.1 (url) - -
		search.msn.com/msnbot.htm	text/..	msnbot/2.0b (url) - -
		search.msn.com/msnbot.htm	text/..	msnbot-NewsBlogs/2.0b (url) - -
		search.msn.com/msnbot.htm	image/..	msnbot-NewsBlogs/2.0b (url) - -
		search.msn.com/msnbot.htm	image/..	msnbot/2.0b (url) - -
		search.msn.com/msnbot.htm	text/..	msnbot-Products/1.0 (url) - -
		search.msn.com/msnbot.htm	text/..	msnbot-UDiscovery/2.0b (url) - -
		search.msn.com/msnbot.htm	text/..	msnbot/0.01 (url) - -
	naver
		help.naver.com/robots/	text/..	Yeti/1.0 (NHN Corp.; url) ko,en;q=0.5 -
		help.naver.com/robots/	text/..	Yeti/1.0 (NHN Corp.; url) ko,ja,en;q=0.5 -
		help.naver.com/robots/	text/..	Yeti/1.0 (NHN Corp.; url) ja,en;q=0.5 -
		help.naver.com/robots/	text/..	Yeti/1.0 (NHN Corp.; url) - -
		help.naver.com/robots/	image/..	Yeti/1.0 (NHN Corp.; url) ja,en;q=0.5 -
		help.naver.com/robots/	text/..	Yeti/1.1 (NHN Corp.; url) vi_VN,vi;q=0.8,en-US;q=0.6,en;q=0.4 -
		help.naver.com/robots/	image/..	Yeti/1.0 (NHN Corp.; url) ko,ja,en;q=0.5 -
		help.naver.com/robots/	application/json	Yeti/1.0 (NHN Corp.; url) - -
		help.naver.com/robots/	image/..	Yeti/1.1 (NHN Corp.; url) vi_VN,vi;q=0.8,en-US;q=0.6,en;q=0.4 -
	orange
		wikipedia.orange.fr/	text/..	API/1.0 (url) - -
	yahoo
		help.yahoo.com/help/us/ysearch/slurp	text/..	Mozilla/5.0 (compatible; Yahoo! Slurp; url) en-us,en;q=0.5 -
		help.yahoo.com/help/us/ysearch/slurp	image/..	Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; url) NOT Firefox/3.5 en-us,en;q=0.5 -
		help.yahoo.com/help/us/ysearch/slurp	text/..	Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; url) NOT Firefox/3.5 en-us,en;q=0.5 -
		help.yahoo.com/help/us/ysearch/slurp	text/..	Mozilla/5.0 (compatible; Yahoo! Slurp; url) - -
		help.yahoo.co.jp/help/jp/search/indexing/indexing-15.html	text/..	Y!J-BRJ/YATS crawler (url) - -
		help.yahoo.com/help/us/ysearch/slurp	-	Mozilla/5.0 (compatible; Yahoo! Slurp; url) en-us,en;q=0.5 -
		help.yahoo.co.jp/help/jp/search/indexing/indexing-15.html	text/..	Y!J-BRW/1.0 crawler (url) ja, *;q=0.5 -
		help.yahoo.co.jp/help/jp/search/indexing/indexing-15.html	text/..	Y!J-BRW/1.0 crawler (url) - -
		developer.yahoo.com/yql/provider	text/..	Mozilla/5.0 (compatible; Yahoo Pipes 2.0; url) Gecko/20090729 Firefox/3.5.2 - -
		help.yahoo.com/help/us/ysearch/slurp	text/..	Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; url) en-us,en;q=0.5 -
		help.yahoo.com/help/us/ysearch/slurp	application/json	Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; url) NOT Firefox/3.5 en-us,en;q=0.5 -
		help.yahoo.com/help/us/ysearch/slurp	image/..	Mozilla/5.0 (compatible; Yahoo! Slurp; url) en-us,en;q=0.5 -
		help.yahoo.co.jp/help/jp/search/indexing/indexing-15.html	text/..	Y!J-BRT/1.0 crawler (url) - -
		help.yahoo.com/help/us/ysearch/slurp	application/xml	Mozilla/5.0 (compatible; Yahoo! Slurp;url) - -
		help.yahoo.co.jp/help/jp/search/indexing/indexing-15.html	image/..	Y!J-BRU/VSIDX dead link checker (url) - -
	cibra
		cibra.de/	text/..	CiBra Data Collector (url) - -
	youdao
		www.youdao.com/help/webmaster/spider/	text/..	Mozilla/5.0 (compatible; YoudaoBot/1.0; url; ) - -
		www.youdao.com/help/webmaster/spider/	text/..	Mozilla/5.0 (compatible; YoudaoBot/1.0; url; ) zh-cn;q=1.0, zh-tw;q=0.8, en;q=0.5, *;q=0.1 -
		www.youdao.com/help/webmaster/spider/	-	Mozilla/5.0 (compatible; YoudaoBot/1.0; url; ) - -
	digplanet
		www.digplanet.com/wiki	application/vnd.php.serialized	Digplanet/1.0 (url; ) PHP/5.4 - -
	genieo
		www.genieo.com/webfilter.html	text/..	Mozilla/5.0 (compatible; Genieo/1.0 url) - -
		www.genieo.com/webfilter.html	application/xml	Mozilla/5.0 (compatible; Genieo/1.0 url) - -
		www.genieo.com/webfilter.html	image/..	Mozilla/5.0 (compatible; Genieo/1.0 url) - -
		www.genieo.com/webfilter.html	image/..	Mozilla/5.0 (compatible; Genieo/1.0 url) en,*
	soso
		help.soso.com/webspider.htm	text/..	Sosospider(url) zh-cn,zh-hk,zh-tw,en-us -
		help.soso.com/webspider.htm	application/xml	Sosospider(url) zh-cn,zh-hk,zh-tw,en-us -
		help.soso.com/webspider.htm	text/..	Mozilla/5.0(compatible; Sosospider/2.0; url) zh-cn,zh-hk,zh-tw,en-us -
		help.soso.com/webspider.htm	-	Sosospider(url) zh-cn,zh-hk,zh-tw,en-us -
	zum
		help.zum.com/inquiry	text/..	ZumBot/1.0 (ZUM Search; url) - -
		help.zum.com/inquiry	image/..	ZumBot/1.0 (ZUM Search; url) - -
	sblog
		fulltext.sblog.cz/screenshot/	image/..	Mozilla/5.0 (compatible; Seznam screenshot-generator 2.0; url) cs,cz,sk;q=0.7,*;q=0.5 -
		fulltext.sblog.cz/	text/..	SeznamBot/3.0 (url) cs -
		fulltext.sblog.cz/screenshot/	text/..	Mozilla/5.0 (compatible; Seznam screenshot-generator 2.0; url) cs,cz,sk;q=0.7,*;q=0.5 -
		fulltext.sblog.cz/	-	SeznamBot/3.0 (url) cs -
		fulltext.sblog.cz/screenshot/	text/..	Mozilla/5.0 (compatible; Seznam screenshot-generator 2.0; url) - -
	wordpress
		wildanrenaldi.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
		williamstickevers.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
		josefboberg.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
		williamstickevers.wordpress.com	image/..	WordPress/3.6-alpha-23334; url - -
		ledinobleu.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
		bestofactus.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
		ainhoaaristizabal.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
		urielarte.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
		greatriversofhope.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
		cheltjules.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
		klausgauger.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
		escogitur.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
		cedricgagneux.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
		kalafudra.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
		ramahanumanraksha.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
		imcradiodotnet.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
		armandecastro.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
		foxhugh.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
		euzicasa.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
		whatownsme.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
		kikestark.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
		beyondthelies.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
		02varvara.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
		melusinefee.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
		bharatabharati.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
		gunnyg.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
		energiaslibres.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
		wildanrenaldi.wordpress.com	image/..	WordPress/3.6-alpha-23334; url - -
		onceuponavoice.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
		mcdens13.wordpress.com	text/..	WordPress/3.6-alpha-23334; url - -
	ac
		www.clips.ua.ac.be/pages/pattern	text/..	Pattern/2.3 url - -
		www.celese.sci.waseda.ac.jp	text/..	2QC (url; mail address ) - -
		www.clips.ua.ac.be/pages/pattern	application/json	Pattern/2.5 url - -
	ahrefs
		ahrefs.com/robot/	text/..	Mozilla/5.0 (compatible; AhrefsBot/4.0; url) - -
		ahrefs.com/robot/	text/..	Mozilla/5.0 (compatible; AhrefsBot.FreshPages/0.1; url) - -
		ahrefs.com/	application/xml	AhrefsBot.Feeds v0.1; url - -
	sistrix
		crawler.sistrix.net/	text/..	Mozilla/5.0 (compatible; SISTRIX Crawler; url) - -
		crawler.sistrix.net/	image/..	Mozilla/5.0 (compatible; SISTRIX Crawler; url) - -
	exabot
		www.exabot.com/go/robot	text/..	Mozilla/5.0 (compatible; Exabot/3.0; url) - -
		www.exabot.com/go/robot	text/..	Mozilla/5.0 (compatible; Exabot/3.0 (BiggerBetter); url) - -
		www.exabot.com/go/robot	-	Mozilla/5.0 (compatible; Exabot/3.0; url) - -
	php
		pear.php.net/	application/vnd.php.serialized	PEAR HTTP_Request class ( url ) - -
		pear.php.net/package/http_request2	text/..	HTTP_Request2/0.5.2 (url) PHP/5.2.17 - -
		pear.php.net/	text/..	PEAR HTTP_Request class ( url ) - -
		pear.php.net/	image/..	PEAR HTTP_Request class ( url ) - -
		pear.php.net/package/http_request2	text/..	HTTP_Request2/2.1.1 (url) PHP/5.3.2-1ubuntu4.18 - -
		pear.php.net/package/http_request2	application/xml	HTTP_Request2/2.0.0 (url) PHP/5.3.8 - -
		pear.php.net/package/http_request2	image/..	HTTP_Request2/2.1.1 (url) PHP/5.3.2-1ubuntu4.18 - -
	jike
		shoulu.jike.com/spider.html	text/..	Mozilla/5.0 (compatible; JikeSpider; url) zh-cn;q=0.8, *;q=0.5 -
		shoulu.jike.com/spider.html	text/..	Mozilla/5.0 (compatible; JikeSpider; url) - -
		shoulu.jike.com/spider.html	-	Mozilla/5.0 (compatible; JikeSpider; url) zh-cn;q=0.8, *;q=0.5 -
		shoulu.jike.com/spider.html	application/json	Mozilla/5.0 (compatible; JikeSpider; url) - -
	www.
		www.	text/..	GoogleBot/2.1 ( urlGoogleBot.com/bot.html) - -
		www.	text/..	GoogleBot-Image/1.0 ( urlGoogleBot.com/bot.html) - -
		www.	text/..	GoogleBot/2.1 (urlGoogleBot.com/bot.html) - -
	wikipedia
		en.wikipedia.org/wiki/Wikipedia:Huggle	text/..	Huggle/2.1.20.0 url - -
		en.wikipedia.org/wiki/User:NicoV/Wikipedia_Cleaner/Documentation	text/..	WPCleaner (url) - -
		fr.wikipedia.org/wiki/Utilisateur:OrlodrimBot	text/..	OrlodrimBot/1.0 (url) - -
		en.wikipedia.org/wiki/Wikipedia:Huggle	text/..	Huggle/2.1.19.0 url - -
		en.wikipedia.org/wiki/Web_crawler	text/..	Robobot/1.0 (url) - -
		en.wikipedia.org/wiki/Wikipedia:Huggle	text/..	Huggle/2.1.20 url - -
	toolserver
		wiki.toolserver.org/view/GeoHack	text/..	Geohack (url) - -
		toolserver.org/~dispenser/	text/..	DispensersTools (url) - -
		toolserver.org/~dispenser/	application/json	DispensersTools (url) - -
		toolserver.org/~para/cgi-bin/kmlexport	text/..	url libwww-perl/6.02 - -
		toolserver.org/~guandalug/	application/vnd.php.serialized	Guandalugs PHPWikiBot/1.1 (url;de:User:Guandalug) - -
		toolserver.org/~platonides/catdown/	image/..	catdown High-resolution_TIFF_images_from_the_National_Archives_and_Records_Administration (url) - -
	blekko
		blekko.com/about/blekkobot	text/..	Mozilla/5.0 (compatible; Blekkobot; ScoutJet; url) - -
	majestic12
		www.majestic12.co.uk/bot.php?	text/..	Mozilla/5.0 (compatible; MJ12bot/v1.4.3; url) - -
		www.majestic12.co.uk/bot.php?	text/..	Mozilla/5.0 (compatible; MJ12bot/v1.4.3; url) en -
	gnip
		www.gnip.com/	text/..	UnwindFetchor/1.0 (url) - -
		www.gnip.com/	image/..	UnwindFetchor/1.0 (url) - -
	sogou
		www.sogou.com/docs/help/webmasters.htm#07	text/..	Sogou web spider/4.0(url) zh-cn -
		www.sogou.com/docs/help/webmasters.htm#07	text/..	Sogou web spider/4.0(url) - -
		www.sogou.com/docs/help/webmasters.htm#07	-	Sogou web spider/4.0(url) zh-cn -
		www.sogou.com/docs/help/webmasters.htm#07	application/json	Sogou web spider/4.0(url) - -
		www.sogou.com/docs/help/webmasters.htm#07	text/..	Sogou Pic Spider/3.0(url) zh-cn -
	yacy
		yacy.net/bot.html	text/..	yacybot (webportal-global; amd64 Linux 2.6.23.17-dbserv; java 1.6.0_04; Europe/de) url en-us,en;q=0.5 -
		yacy.net/bot.html	text/..	yacybot (freeworld/global; amd64 Windows 8 6.2; java 1.6.0_41; Europe/de) url en-us,en;q=0.5 -
		yacy.net/bot.html	text/..	yacybot (freeworld/global; amd64 Linux 3.2.0-36-generic; java 1.6.0_27; Europe/en) url en-us,en;q=0.5 -
		yacy.net/bot.html	text/..	yacybot (freeworld/global; amd64 Linux 2.6.32-custom; java 1.6.0_26; Europe/en) url en-us,en;q=0.5 -
		yacy.net/bot.html	text/..	yacybot (webportal/global; i386 Linux 3.8.0-8-generic; java 1.7.0_17; Europe/ru) url en-us,en;q=0.5 -
		yacy.net/bot.html	text/..	yacybot (freeworld/global; amd64 Linux 3.2.0-4-amd64; java 1.6.0_27; Australia/en) url en-us,en;q=0.5 -
		yacy.net/bot.html	text/..	yacybot (freeworld/global; amd64 Linux 3.2.0-39-generic; java 1.6.0_27; Europe/lv) url en-us,en;q=0.5 -
		yacy.net/bot.html	text/..	yacybot (freeworld-global; amd64 Linux 2.6.32-46-server; java 1.6.0_26; Europe/de) url en-us,en;q=0.5 -
		yacy.net/bot.html	text/..	yacybot (freeworld/global; amd64 Linux 3.2.0-39-generic; java 1.6.0_27; Europe/de) url en-us,en;q=0.5 -
		yacy.net/bot.html	text/..	yacybot (freeworld/global; amd64 Linux 3.5.0-26-generic; java 1.6.0_27; America/en) url en-us,en;q=0.5 -
		yacy.net/bot.html	text/..	yacybot (freeworld/global; amd64 Linux 2.6.32-19-pve; java 1.6.0_18; Etc/en) url en-us,en;q=0.5 -
		yacy.net/bot.html	text/..	yacybot (freeworld/global; amd64 Linux 3.1.10-1.16-default; java 1.6.0_27; Europe/de) url en-us,en;q=0.5 -
		yacy.net/bot.html	text/..	yacybot (freeworld/global; x86_64 Mac OS X 10.8.1; java 1.6.0_43; Asia/ru) url en-us,en;q=0.5 -
		yacy.net/bot.html	text/..	yacybot (freeworld/global; i386 Linux 3.2.0-40-generic; java 1.7.0_15; Europe/en) url en-us,en;q=0.5 -
		yacy.net/bot.html	text/..	yacybot (freeworld/global; i386 Linux 3.2.0-40-generic; java 1.7.0_15; Europe/de) url en-us,en;q=0.5 -
		yacy.net/bot.html	text/..	yacybot (freeworld/global; amd64 Linux 3.2.0-39-generic; java 1.6.0_27; Europe/en) url en-us,en;q=0.5 -
	echonest
		the.echonest.com/reader/	application/xml	nestReader/0.3 (discovery; url; reader at echonest.com) en -
		the.echonest.com/reader/	text/..	nestReader/0.3 (discovery; url; reader at echonest.com) en -
	coccoc
		help.coccoc.com/	text/..	coccoc/1.0 (url) en-us;q=0.7,en;q=0.3 -
		help.coccoc.com/	image/..	coccoc/1.0 (url) en-us;q=0.7,en;q=0.3 -
		help.coccoc.com/	text/..	coccoc/1.0 (url) - -
	80legs
		www.80legs.com/webcrawler.html	text/..	Mozilla/5.0 (compatible; 008/0.85; url) Gecko/2008032620 - -
	localhost
		localhost/wordpress	text/..	WordPress/3.5.1; url - -
		localhost/yioop/bot.php	text/..	Mozilla/5.0 (compatible; TESTROBOT; url) - -
		localhost/2	text/..	WordPress/3.5.1; url - -
		localhost/1	text/..	WordPress/3.5.1; url - -
		localhost/dizzyworld	text/..	WordPress/3.5.1; url - -
	traslated
		mymemory.traslated.net/doc/	text/..	Mozilla/5.0 (MyMemory Bot url) - -
	flipboard
		flipboard.com/browserproxy	image/..	Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 (FlipboardProxy/0.0.5; url) en-us,en;q=0.5 -
		flipboard.com/browserproxy	text/..	Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 (FlipboardProxy/1.1; url) en-us,en;q=0.5 -
		flipboard.com/browserproxy	text/..	Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 (FlipboardProxy/0.0.5; url) en-us,en;q=0.5 -
		flipboard.com/browserproxy	image/..	null (FlipboardProxy/1.1; url) - -
	wikidict
		www.wikidict.de	text/..	url - -
	SearchNearMe
		SearchNearMe.com/contact.php	application/vnd.php.serialized	SearchNearMe (url) - -
		SearchNearMe.com/contact.php	text/..	SearchNearMe (url) - -
	goo
		help.goo.ne.jp/contact/	text/..	goo wikipedia (url) - -
		search.goo.ne.jp/option/use/sub4/sub4-1/	-	DoCoMo/2.0 P900i(c100;TB;W24H11) (compatible; ichiro/mobile goo; url) - -
		help.goo.ne.jp/door/crawler.html	text/..	ichiro/3.0 (url) - -
		search.goo.ne.jp/option/use/sub4/sub4-1/	text/..	DoCoMo/2.0 P900i(c100;TB;W24H11) (compatible; ichiro/mobile goo;url) - -
		search.goo.ne.jp/option/use/sub4/sub4-1/	-	DoCoMo/2.0 P900i(c100;TB;W24H11) (compatible; ichiro/mobile goo;url) - -
		goo.gl/7y4SX	image/..	GoogleProducer; (url) - -
		goo.gl/7y4SX	text/..	GoogleProducer; (url) - -
	frontle
		www.frontle.com/	application/vnd.php.serialized	FrontleBot/1.0 (url) - -
	bin-co
		www.bin-co.com/php/scripts/load/	text/..	BinGet/1.00.A (url) - -
		www.bin-co.com/php/scripts/load/	application/vnd.php.serialized	BinGet/1.00.A (url) - -
	mail
		go.mail.ru/help/robots	text/..	Mozilla/5.0 (compatible; Mail.RU_Bot/2.0; url) ru,ua;q=0.7,by;q=0.7,*;q=0.1 -
		go.mail.ru/help/robots	text/..	Mozilla/5.0 (compatible; Mail.RU_Bot/2.0; url) - -
	okian
		www.okian.ro/	text/..	MyBot/1.0 (url) - -
	topsy
		labs.topsy.com/butterfly/	text/..	Mozilla/5.0 (compatible; Butterfly/1.0; url) Gecko/2009032608 Firefox/3.0.8 - -
	daum
		tab.search.daum.net/aboutWebSearch.html	text/..	Mozilla/5.0 (compatible; MSIE or Firefox mutant; not on Windows server; url) Daumoa/3.0 ko-kr,ko;q=0.8,en-us;q=0.5,en;q=0.3 -
		tab.search.daum.net/aboutWebSearch.html	text/..	Mozilla/5.0 (compatible; MSIE or Firefox mutant; not on Windows server; url) Daumoa/3.0 - -
	archive-it
		archive-it.org/files/site-owners.html	image/..	Mozilla/5.0 (compatible; archive.org_bot; Archive-It; url) - -
		archive-it.org/files/site-owners.html	text/..	Mozilla/5.0 (compatible; archive.org_bot; Archive-It; url) - -
	easou
		www.easou.com/search/spider.html	text/..	Mozilla/5.0 (compatible; EasouSpider; url) zh;q=0.9,en;q=0.8 -
		www.easou.com/search/spider.html	application/json	Mozilla/5.0 (compatible; EasouSpider; url) - -
	archive
		www.archive.org/details/archive.org_bot	text/..	Mozilla/5.0 (compatible; archive.org_bot url) - -
		www.archive.org/details/archive.org_bot	image/..	Mozilla/5.0 (compatible; special_archiver/3.1.1 url) - -
		archive.org/details/archive.org_bot	image/..	Mozilla/5.0 (compatible; heritrix/3.1.2-SNAPSHOT-20121013.132750 url) - -
	plos
		alm.plos.org	application/json	PLoS Article Level Metrics - url - -
	leiki
		www.leiki.com	text/..	Leikibot/1.0 (url) - -
	wwwgogetpapers
		wwwgogetpapers.com/	application/json	User-Agent: GoGetPapersBot (url) - -
	bsurprised
		bsurprised.com/	text/..	BSurprised WikiBox 0.1.3 (url) en -
		bsurprised.com/	text/..	BSurprised WikiBox 0.1.3 (url) af -
		bsurprised.com/	text/..	BSurprised WikiBox 0.1.3 (url) hu -
	in
		www.m-culture.in.th	text/..	m-culture.in.th (url) - -
	pagefreezer
		pagefreezer.com/pagefreezer-crawler/	image/..	PageFreezer (pagefreezer crawler; url; mail address ) en-us,en;q=0.5 -
		pagefreezer.com/pagefreezer-crawler/	text/..	PageFreezer (pagefreezer crawler; url; mail address ) en-us,en;q=0.5 -
	elcidharth
		elcidharth.com	text/..	WordPress/3.6-alpha-23334; url - -
	kosmix
		www.kosmix.com/html/kosmos.html	application/xml	Mozilla/5.0(compatible;Kosmos/1.0;url) - -
	wikimpress
		wikimpress.org/	text/..	Mozilla/5.0 (compatible; Linux i686 (x86_64); de-DE; url>Wikimpress) Wikimpress/1.0 - -
		wikimpress.org/	-	Mozilla/5.0 (compatible; Linux i686 (x86_64); de-DE; url>Wikimpress) Wikimpress/1.0 - -
	weblio
		www.weblio.jp/info/crawler.jsp	image/..	Mozilla/5.0 (compatible; Webliobot/0.1; url) - -
		www.weblio.jp/info/crawler.jsp	text/..	Mozilla/5.0 (compatible; Webliobot/0.1; url) - -
		www.weblio.jp/	text/..	Mozilla/5.0 (compatible; WeblioBot; url) - -
		www.weblio.jp/	text/..	Mozilla/5.0 (compatible; WeblioBot; url) ja -
	wikimedia
		commons.wikimedia.org/wiki/User:Thumbnails_Check_Bot	image/..	Thumbnails_Check_Bot/0.1 (url; beta) - -
		commons.wikimedia.org/wiki/User:Thumbnails_Check_Bot	text/..	Thumbnails_Check_Bot/0.1 (url; beta) - -
	zeebox
		www.zeebox.com	text/..	Zeebox (url) en-us,en;q=0.5 -
		www.zeebox.com	application/json	Zeebox (url) en-us,en;q=0.5 -
	embed
		support.embed.ly/	image/..	Mozilla/5.0 (compatible; Embedly/0.2; snap; url) - -
		support.embed.ly/	text/..	Mozilla/5.0 (compatible; Embedly/0.2; url) - -
	paper
		support.paper.li/entries/20023257-what-is-paper-li	text/..	Mozilla/5.0 (compatible; PaperLiBot/2.1; url) - -
	netarkivet
		netarkivet.dk/webcrawler/	text/..	Mozilla/5.0 (compatible; heritrix/1.14.4 url) - -
		netarkivet.dk/webcrawler/	image/..	Mozilla/5.0 (compatible; heritrix/1.14.4 url) - -
	proximic
		www.proximic.com/info/spider.php	text/..	Mozilla/5.0 (compatible; proximic; url) - -
	speaktoit
		www.speaktoit.com	application/json	Speaktoit url - -
	bibalex
		archive.bibalex.org/bot/	text/..	Mozilla/5.0 (compatible; archive.bibalex.org_bot; url) - -
		archive.bibalex.org/bot/	image/..	Mozilla/5.0 (compatible; archive.bibalex.org_bot; url) - -
	emining
		emining.jp/	text/..	emBot-GalaBuzz/Nutch-1.0 (url; mail address ) en-us,en-gb,en;q=0.7,*;q=0.3 -
		emining.jp/	-	emBot-GalaBuzz/Nutch-1.0 (url; mail address ) en-us,en-gb,en;q=0.7,*;q=0.3 -
	apercite
		www.apercite.fr/robot/index.html	image/..	Mozilla/5.0 (compatible; Apercite; url) en-us,en;q=0.5 -
	tineye
		tineye.com/crawler.html	application/json	TinEye/1.1 (url) - -
	thearchangelmichael
		archives.thearchangelmichael.net	text/..	WordPress/3.5.1; url - -
	federatedmedia
		federatedmedia.net	text/..	Mozilla/5.0 (url) Gecko/20061208 Firefox/2.0.0.1 en-us,en;q=0.5 -
	zipcode
		zipcode.us	text/..	Mozilla/5.0 (compatible; YourCoolBot/1.0; url) - -
	cognarius
		cognarius.com	application/json	AppsArlak/1.0 (url) - -
		cognarius.com	text/..	AppsArlak/1.0 (url) - -
	easybib
		content.easybib.com/autocite/	text/..	EasyBib AutoCite (url) - -
		content.easybib.com/autocite/	application/json	EasyBib AutoCite (url) - -
	tweetmeme
		tweetmeme.com/	text/..	Mozilla/5.0 (compatible; TweetmemeBot/3.0; url) en-gb,en;q=0.5 -
	wandex
		wandex.net	text/..	Mozilla/5.0 (compatible; World Wide Web Wanderer (Wandex Bot)/1.4; url) - -
	FeedBurner
		www.FeedBurner.com	text/..	FeedBurner/1.0 (url) - -
	trendiction
		www.trendiction.de/bot	text/..	Mozilla/5.0 (Windows; Windows NT 6.0; en-GB; rv:1.0; trendictionbot0.5.0; trendiction search; url; please let us know of any problems; web at trendiction.com) Gecko/20071127 Firefox/3.0.0.11 en-gb,en;q=0.5 -
		www.trendiction.de/bot	image/..	Mozilla/5.0 (Windows; Windows NT 6.0; en-GB; rv:1.0; trendictionbot0.5.0; trendiction search; url; please let us know of any problems; web at trendiction.com) Gecko/20071127 Firefox/3.0.0.11 en-gb,en;q=0.5 -
	hatena
		a.hatena.ne.jp/help	text/..	Hatena Antenna/0.5 (url) - -
	drupal
		drupal.org/	text/..	Drupal (url) - -
		drupal.org/	text/..	User-Agent: Drupal (url) - -
		drupal.org/	image/..	Drupal (url) - -
	chickyrun
		chickyrun.tk/	text/..	ChickyBot/1.1 (url; mail address ) - -
	toshiba
		www.toshiba.co.jp/rdc/about/crawl_info_en.htm	text/..	TosCrawler/Nutch-1.6 (url; ' mail address dot co dot jp') en-us,en-gb,en;q=0.99,*;q=0.01 -
	rebelmouse
		rebelmouse.com	image/..	RebelMouse/0.1 Mozilla/5.0 (compatible; url) Gecko/20100101 Firefox/7.0.1 en-us,en;q=0.5 -
		rebelmouse.com	text/..	RebelMouse/0.1 Mozilla/5.0 (compatible; url) Gecko/20100101 Firefox/7.0.1 en-us,en;q=0.5 -
	sunaga-lab
		www.sunaga-lab.com/graham-bot	text/..	Grahambot/0.1 (url) - -
	zibalinks
		www.zibalinks.com	text/..	Ziba/Nutch-1.4 (Ziba Links Web Spider; url) en-us,en-gb,en;q=0.7,*;q=0.3 -
	plagiarismcheck
		plagiarismcheck.org	application/json	WikiCrawl 1.0b (url contact-mail: mail address ) - -
	picsearch
		www.picsearch.com/bot.html	text/..	psbot/0.1 (url) - -
	rockpeaks
		www.rockpeaks.com/contact	text/..	RockPeaks/0.1 (url) - -
	xbmc
	spinn3r
		spinn3r.com/robot	text/..	Mozilla/5.0 (X11; Linux x86_64; en-US; rv:1.9.0.19; aggregator:Spinn3r (Spinn3r 3.1); url) Gecko/2010040121 Firefox/3.0.19 - -
		spinn3r.com/robot	application/xml	Mozilla/5.0 (X11; Linux x86_64; en-US; rv:1.9.0.19; aggregator:Spinn3r (Spinn3r 3.1); url) Gecko/2010040121 Firefox/3.0.19 - -
	vermagerd
		www.vermagerd.be/wp	text/..	WordPress/3.4.2; url - -
	monitis
		www.monitis.com	text/..	Mozilla/5.0 (compatible; monitis - premium monitoring service; url) - -
		www.monitis.com	text/..	Mozilla/5.0 (compatible; Monitis - premium monitoring service; url) - -
	github
		github.com/pauldix/typhoeus/tree/master	text/..	Typhoeus - url - -
		wummel.github.com/linkchecker/	text/..	Mozilla/5.0 (compatible; LinkChecker/8.4; url) - -
		github.com/pauldix/feedzirra/tree/master	application/xml	feedzirra url - -
		wiki.github.com/bixo/bixo/bixocrawler	text/..	Mozilla/5.0 (compatible; pub-crawler; url; mail address ) en-us,en-gb,en;q=0.7,*;q=0.3 -
	moviecus
		www.moviecus.com/botcontactinfo.php	application/yaml	moviecus bot (url) - -
	potaru
		potaru.com/robo.html	text/..	Mozilla/5.0 (compatible; Robo/1.0b; url)/Nutch-1.2 en-us,en-gb,en;q=0.7,*;q=0.3 -
	sf
		liferea.sf.net/	text/..	Liferea/1.x.x (Linux; es_ES.UTF-8; url) en-us,en;q=0.5 -
		liferea.sf.net/	text/..	Liferea/0.x.x (Linux; en_US.UTF-8; url) en-us,en;q=0.5 -
		magpierss.sf.net	text/..	MagpieRSS/0.7x (url) en-us,en;q=0.5 -
	treycopplandmusic
		beatsblog.treycopplandmusic.com	text/..	WordPress/3.5.1; url - -
	textdigger
		textdigger.com	text/..	Mozilla/5.0 (url) Gecko/20061208 Firefox/2.0.0.1 en-us,en;q=0.5 -
	pingdom
		www.pingdom.com/	text/..	Pingdom.com_bot_version_1.4_(url) - -
		www.pingdom.com	text/..	Pingdom.com_bot_version_1.4_(url) - -
	rockmelt
		rockmelt.com	text/..	RockmeltEmbedder (url; mail address ) - -
	msai
		www.msai.in/uaprof/micromax/X455.xml	image/..	url en,hi -
		www.msai.in/uaprof/micromax/X1i_Extra.xml	image/..	url en,hi -
		www.msai.in/uaprof/micromax/X101.xml	image/..	url en,hi -
	yoursite
		yoursite.com/botinfo	text/..	Mozilla/5.0 (compatible; YourCoolBot/1.0; url) - -
	alexa
		www.alexa.com/site/help/webmasters	text/..	ia_archiver (url; mail address ) - -
	muso
		www.muso.com	text/..	Mozilla/5.0 (compatible; musobot/1.0; mail address ; url) - -
	zookabot
		zookabot.com	text/..	Zookabot/2.5;url - -
	queryseeker
		queryseeker.com/bot.html	text/..	QuerySeekerSpider ( url ) - -
	js-kit
		js-kit.com/	text/..	JS-Kit URL Resolver, url - -
	sanskritdictionary
		www.sanskritdictionary.com/	application/vnd.php.serialized	User-Agent: SanskritDictionary/0.1 (url) - -
	stackoverflow
		stackoverflow.com/questions/8956331/how-to-get-results-from-the-wikipedia-api-with-php	text/..	Testing for url - -
	go
		kc.nict.go.jp/project1/crawl.html	text/..	ICC-Crawler/2.0 (Mozilla-compatible; ; url) ja -
		kc.nict.go.jp/project1/crawl-ja.html	text/..	ICC-Crawler (Mozilla-compatible; mail address ; url) ja -
	mysite
		www.mysite.com/	text/..	MyBot/1.0 (url) - -
		www.mysite.com/	application/json	MyBot/1.0 (url) - -
	mindano
		www.mindano.com	image/..	Mindano (url) - -
	n-grams
		www.n-grams.org/icorpusbot.html	text/..	iCorpusBot (url) es-es,en-us;q=0.7,en;q=0.3 -
	simplepie
		simplepie.org	text/..	SimplePie/1.2 (Feed Parser; url; Allow like Gecko) Build/20090627192103 - -
		simplepie.org	application/xml	SimplePie/1.2 (Feed Parser; url; Allow like Gecko) Build/20090627192103 - -
	newsgator
		www.newsgator.com	text/..	NewsGatorOnline/2.0 (url; 1 subscribers) en-us,en;q=0.5 -
		www.newsgator.com/	text/..	FeedDemon/2.7 (url; Microsoft Windows XP) en-us,en;q=0.5 -
	zapbot
		www.zapbot.org	text/..	Mozilla/5.0 (compatible; ZapBot/0.2o; url) - -
		www.zapbot.net	text/..	Mozilla/5.0 (compatible; ZapBot/0.2n; url) - -
		www.zapbot.com	text/..	Mozilla/5.0 (compatible; ZapBot/0.2c; url) - -
	pinterest
		pinterest.com/	text/..	Pinterest/0.1 url - -
		pinterest.com/	image/..	Pinterest/0.1 url - -
	site-shot
		www.site-shot.com/	image/..	Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/534.34 KHTML Site-Shot/2.1 (url) Safari/534.34 en-US,* -
	reget
		www.reget.com	text/..	Mozilla/4.0 (compatible; url>ReGet Deluxe 5.1; Windows NT 6.0) - -
	yougorhymes
		www.yougorhymes.com/site/rhyme-bot	text/..	RhymeBot/0.1 (url) - -
	nb
		www.nb.no/vevfangst	image/..	Mozilla/5.0 (compatible; heritrix/1.14.4 url) - -
		www.nb.no/vevfangst	text/..	Mozilla/5.0 (compatible; heritrix/1.14.4 url) - -
	netvibes
		www.netvibes.com	text/..	Netvibes (url) - -
		www.netvibes.com	application/json	Netvibes (url) en-us -
	instapaper
		www.instapaper.com/	text/..	Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/534.50 KHTML Version/5.1 Instapaper/4.0 (url) - -
	backgroundswitcher
		www.backgroundswitcher.com/	text/..	John's Background Switcher 4.4 (url) - -
		www.backgroundswitcher.com/	image/..	John's Background Switcher 4.6 (url) - -
	jetbrains
		www.jetbrains.com/omea_reader/	text/..	JetBrains Omea Reader 2.0 Release Candidate 1 (url) en-us,en;q=0.5 -
		www.jetbrains.com/omea_reader/	text/..	JetBrains Omea Reader 1.0.x (url) en-us,en;q=0.5 -
	vk
		vk.com/dev/Share	text/..	Mozilla/5.0 (compatible; vkShare; url) - -
	w3
		validator.w3.org/services	text/..	W3C_Validator/1.3 url - -
	nvdev
		qa.nvdev.com	text/..	Netvibes (url) - -
		trunk.nvdev.com	text/..	Netvibes (url) - -
	website-datenbank
		www.website-datenbank.de/	text/..	netEstate NE Crawler (url) - -
	weborking
		weborking.com	text/..	Weborking(url) - -
		weborking.com	application/json	Weborking(url) - -
	orcabrowser
		www.orcabrowser.com	text/..	Orca Browser (url) en -
		www.orcabrowser.com	text/..	Orca Browser (url) en-us,en;q=0.5 -
	studiofaca
		www.studiofaca.com/	text/..	Mozila/5.0 (compatible; StudioFACA Search; url) - -
	onemusicapi
		www.onemusicapi.com	text/..	onemusicapi.com/beta url - -
	sonyericsson
		www.sonyericsson.com/UAprof/R800xR301.xml	image/..	Mozilla/5.0 (Linux; Android/2.3.3; en-us; SonyEricssonR800xurl Build/3.0.1.E.1.44) AppleWebKit/533.1 KHTML Version/4.0 Mobile Safari/533.1 en-US -
	QtWeb
	archive-org
		archive-org.com/bot	image/..	Mozilla/5.0 (compatible; archive-org.com/1.1; url) - -
		archive-org.com/bot	text/..	Mozilla/5.0 (compatible; archive-org.com/1.1; url) - -
	creativecommons
		wiki.creativecommons.org/Metadata_Scraper	text/..	CC Metadata Scaper url - -
	jetsli
		jetsli.de/crawler	text/..	Mozilla/5.0 (compatible; Jetslide; url) en-us -
	feedshow
		www.feedshow.com	text/..	FeedshowOnline (url) en-us,en;q=0.5 -
		www.feedshow.com	text/..	Feedshow/x.0 (url; 1 subscriber) en-us,en;q=0.5 -
	scrapy
		scrapy.org	text/..	Scrapy/0.16.4 (url) en -
	rcdtokyo
		www.rcdtokyo.com/pc2m/	text/..	Mozilla/5.0 (compatible; PEAR HTTP_Request class; url) ja -
	mobileproxy
		mobileproxy.mobi	text/..	Mozilla/5.0 (compatible; MobileSurf; url) - -
115354.920000002	total

Page requests for probable crawlers, recognized by keyword
Count x 1000	Agent string
		Mime type (count ≥ 3)
	php wikibot classes - -
		application/vnd.php.serialized
		text/..
	AniBot/0.9 php/curl - -
		application/vnd.php.serialized
	MediaWikiCrawler-Google/2.0 ( mail address ) - -
		text/..
		-
	GoogleBot-Image/1.0 - -
		image/..
		text/..
		-
	Peachy MediaWiki Bot API Version 2.0 (beta) - -
		application/vnd.php.serialized
		text/..
		image/..
	enwiki_removal/r-1 (unknown) Pywikipediabot/2.0 - -
		application/json
		text/..
	LinkParser/2.0 - -
		text/..
	spider - -
		text/..
		application/vnd.php.serialized
		application/json
	pywikipedia-radeh.py/r11231 Pywikipediabot/1.0 - -
		application/json
		application/xml
	DotNetWikiBot/2.101 (Microsoft Windows NT 5.1.2600 Service Pack 3; ) - -
		text/..
		application/xml
	PythonWikipediaBot/1.0 - -
		application/json
		application/xml
		text/..
		-
	GoogleBot-Image/1.0 - -
		text/..
		image/..
		-
		application/xml
		application/opensearchdescription+xml
	RobBot ( mail address ) - -
		application/vnd.php.serialized
	Mozilla/5.0 (compatible; SearchBot) - -
		text/..
	wikidata_create/r-1 (unknown) Pywikipediabot/2.0 - -
		application/json
	pywikipedia-wikidata_wd.py/r11181 (wikipedia.py) Pywikipediabot/1.0 - -
		application/json
		application/xml
	pywikipedia-wikidata9de.py/r11311 Pywikipediabot/1.0 - -
		application/json
		application/xml
	DotNetWikiBot/2.101 (Unix 3.0.0.32; ) - -
		text/..
		application/xml
	pywikipedia-hamsang.py/r11308 Pywikipediabot/1.0 - -
		application/json
	DotNetWikiBot/2.101 (Microsoft Windows NT 6.1.7601 Service Pack 1; ) - -
		text/..
		application/xml
	tigerbot - -
		application/json
		text/..
	Mozilla/5.0 (Windows; Windows NT 5.1; zh-CN; rv:1.8.0.11) Firefox/1.5.0.11; 360Spider zh-CN -
		text/..
		image/..
	pywikipedia-wikidata9de.py/r11228 Pywikipediabot/1.0 - -
		application/json
		application/xml
	ClueBot/1.1 - -
		application/vnd.php.serialized
	wikiwix-bot-3.0 - -
		text/..
	DotNetWikiBot/2.101 (Unix 3.0.0.12; ) - -
		text/..
		application/xml
	pywikipedia-hamsang.py/r11321 Pywikipediabot/1.0 - -
		application/json
	pywikipedia-wikidata9de.py/r11181 (wikipedia.py) Pywikipediabot/1.0 - -
		application/json
		application/xml
	pywikipedia-redirect.py/r11321 Pywikipediabot/1.0 - -
		application/json
	pywikipedia-wikidata_war.py/r11181 (wikipedia.py) Pywikipediabot/1.0 - -
		application/json
		application/xml
	ClueBot/2.0 - -
		application/vnd.php.serialized
	Mozilla/5.0 (Windows; Windows NT 5.1; fr; rv:1.8.1) VoilaBot BETA 1.2 ( mail address ) fr; q=1.0, en; q=0.5, *; q=0.1 -
		text/..
	pywikipedia-wikidata_cleanlabel.py/r11200 Pywikipediabot/1.0 - -
		application/json
		application/xml
	TrueKnowledgeBot bot mail address > - -
		application/xml
		application/vnd.php.serialized
		image/..
	pywikipedia-redirect.py/r11168 Pywikipediabot/1.0 - -
		application/json
		text/..
	Mozilla/5.0 (compatible; Ezooms/1.0; mail address ) - -
		text/..
		application/json
		image/..
	pywikipedia-wikidata_wd.py/r11311 Pywikipediabot/1.0 - -
		application/json
		application/xml
	process_baseball/r-1 (unknown) Pywikipediabot/2.0 - -
		application/json
	DigitalsmithsBot - -
		text/..
	pywikipedia-wikidata_disamb.py/r11200 Pywikipediabot/1.0 - -
		application/json
		application/xml
	pywikipedia-interwiki.py/r11096 Pywikipediabot/1.0 - -
		application/xml
		application/json
	MyCuteBot/0.1 - -
		text/..
		application/json
		application/vnd.php.serialized
	acGen-acGenerator_trunk.py/r11321 Pywikipediabot/1.0 - -
		application/json
	p-welcome-w-bn-core.py/r10872 Pywikipediabot/1.0 - -
		application/json
	pywikipedia-wikidata_wceb.py/r11181 (wikipedia.py) Pywikipediabot/1.0 - -
		application/json
		application/xml
	trunk-featured.py/r11322 Pywikipediabot/1.0 - -
		application/json
		application/xml
	Mozilla 5.0 (Apibot 0.32) - -
		application/vnd.php.serialized
	maj_articles_recents/r11235 Pywikipediabot/2.0 - -
		application/json
	MorbZ-Bot - -
		application/json
	pywikipedia-wikidata9de.py/r11304 Pywikipediabot/1.0 - -
		application/json
	UAA-UAA.py/r-1 (unknown) Pywikipediabot/1.0 - -
		application/json
	pywikipedia-redirect.py/r11308 Pywikipediabot/1.0 - -
		application/json
	pywikipedia-redirect.py/r11301 Pywikipediabot/1.0 - -
		application/json
	MediaWiki::Bot/3.2.6 - -
		application/json
	pywikipedia-commonscat_wikidata.py/r11104 Pywikipediabot/1.0 - -
		application/json
	pywikipedia-featured.py/r11322 Pywikipediabot/1.0 - -
		application/json
		application/xml
	trunk-checkimages.py/r11300 Pywikipediabot/1.0 - -
		application/json
	Semantix Bot 0.1 - -
		text/..
	Wikipediabot-mywikidatabot2.py/r-1 (unknown) Pywikipediabot/1.0 - -
		application/json
		application/xml
	Mozilla/5.0 MaboMwFramework/1.2 (w:de:MerlIwBot) - -
		text/..
	Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.5 (Exabot-Thumbnails) en,* -
		image/..
		text/..
		application/json
	pywikipedia-wikidata9.py/r11311 Pywikipediabot/1.0 - -
		application/json
		application/xml
	pywikipedia-wdph25.py/r11275 Pywikipediabot/1.0 - -
		application/json
	AnomieBOT 1.0 (TagDater; see [[User:AnomieBOT]]) - -
		application/json
	pywikipedia-iw2make.py/r11317 Pywikipediabot/1.0 - -
		application/json
		application/xml
	pywikipedia-wdph22.py/r11275 Pywikipediabot/1.0 - -
		application/json
		application/xml
	DotNetWikiBot/2.101 (Microsoft Windows NT 6.2.9200.0; ) - -
		text/..
	svnpywikipedia-interwiki.py/r11270 Pywikipediabot/1.0 - -
		application/json
		application/xml
	Tawbot (public svn release; plwiki) - -
		text/..
	Wikibot/2.0.2 CFNetwork/609.1.4 Darwin/13.0.0 en-us -
		image/..
		application/json
		text/..
	pywikipedia-welcome.py/r11252 Pywikipediabot/1.0 - -
		application/json
	Python27-pythonw.exe/r11204 Pywikipediabot/1.0 - -
		application/json
	pywikipedia-newiw2.py/r11321 Pywikipediabot/1.0 - -
		application/json
		application/xml
	pywikipedia-wdph23.py/r11275 Pywikipediabot/1.0 - -
		application/json
	DotNetWikiBot/2.99 (Microsoft Windows NT 6.1.7601 Service Pack 1; ) - -
		text/..
		application/xml
	pywikipedia-featured.py/r11307 (wikipedia.py) Pywikipediabot/1.0 - -
		application/json
	mrajedrez-articulos-redirecciones.py/r11321 Pywikipediabot/1.0 - -
		application/json
	pywikipedia-featured.py/r11321 Pywikipediabot/1.0 - -
		application/json
		application/xml
	SurakWare MediaWiki Bot/1.0 - -
		text/..
	pywikipedia-wikidata3cat.py/r11181 (wikipedia.py) Pywikipediabot/1.0 - -
		application/json
		application/xml
	ideasBot 1.0 Series By DGideas - -
		text/..
	DotNetWikiBot/2.100 (Unix 3.2.0.39; ) - -
		text/..
	Peachy MediaWiki Bot API Version 1.0 - -
		application/vnd.php.serialized
	pywikipedia-anagrama.py/r11215 Pywikipediabot/1.0 - -
		application/json
	Mozilla/5.0 (compatible; Nigma.ru/3.0; mail address ) - -
		text/..
	GermCrawler - -
		application/json
	pywikipediabot-wikidata_new_global.py/r11215 Pywikipediabot/1.0 - -
		application/json
	CorenSearchBot/1.7 en libwww-perl/6.04 - -
		text/..
	pywikipedia-dikantenyvaovao.py/r11215 Pywikipediabot/1.0 - -
		application/json
	Wikirage.Com Statistics Bot - -
		text/..
	DotNetWikiBot/2.100 (Microsoft Windows NT 6.2.9200.0; ) - -
		text/..
		application/xml
	mail address mail address – MediaWiki Tcl Bot Framework 0.5 (r1) - -
		application/json
	pywikipedia-featured.py/r11308 Pywikipediabot/1.0 - -
		application/json
		application/xml
	pywikipedia-28main.build89.py/r11131 Pywikipediabot/1.0 - -
		application/json
		application/xml
	wikiscore-MakeScoreTable.py/r11152 Pywikipediabot/1.0 - -
		application/json
	AnomieBOT 1.0 (OrphanReferenceFixer; see [[User:AnomieBOT]]) - -
		application/json
	pywikipedia-vlinders_wikidata.py/r11104 Pywikipediabot/1.0 - -
		application/json
	pywikipedia-iw2make2.py/r11321 Pywikipediabot/1.0 - -
		application/json
		application/xml
	SineBot/1.5.19(User:SineBot) - -
		application/vnd.php.serialized
		text/..
	WikiTrans.net Bot (User:WikiTransBot; Contact: mail address ) - -
		text/..
	dtSearchSpider - -
		text/..
	pywikipedia-wikidata_wceb.py/r11311 Pywikipediabot/1.0 - -
		application/json
		application/xml
	Global-Crawler/Nutch-1.4 (Search platform crawler) en-us,en-gb,en;q=0.7,*;q=0.3 -
		text/..
	pywikipedia-wikidata3cat.py/r11228 Pywikipediabot/1.0 - -
		application/json
	AnomieBOT 1.0 (PERTableUpdater; see [[User:AnomieBOT]]) - -
		application/json
		text/..
	Test Webbot - -
		text/..
	WikiPlaysBot - -
		text/..
	DotNetWikiBot/2.100 (Unix 5.10.0.0; ) - -
		text/..
		application/xml
	Wikidata-wikidata.py/r11202 Pywikipediabot/1.0 - -
		application/json
	pywikipedia-wikidata_war.py/r11311 Pywikipediabot/1.0 - -
		application/json
	HTMLParser/2.0 - -
		text/..
	pywikipedia-coord2.py/r11275 Pywikipediabot/1.0 - -
		application/json
		text/..
	pywikipedia-featured.py/r11324 Pywikipediabot/1.0 - -
		application/json
	pywikipedia-17-progreso1911.py/r11321 Pywikipediabot/1.0 - -
		application/json
		application/xml
	AnomieBOT 1.0 (FlagIconRemover; see [[User:AnomieBOT]]) - -
		application/json
	www.integromedb.org/Crawler - -
		text/..
	pywikipedia-info2.py/r11308 Pywikipediabot/1.0 - -
		application/json
	wikidata-sitelinks.py/r11176 Pywikipediabot/1.0 - -
		application/json
	pywikipedia-featured.py/r11054 Pywikipediabot/1.0 - -
		application/json
		application/xml
	SchoolReviewNetworkWikiBot - -
		application/json
	pywikirish-welcome.py/r11215 Pywikipediabot/1.0 - -
		application/json
	Goalkeeperbot(User:Beetstra)/1.0 - -
		text/..
	pywikipedia-redirect.py/r11324 Pywikipediabot/1.0 - -
		application/json
	AnomieBOT 1.0 (TemplateSubster; see [[User:AnomieBOT]]) - -
		application/json
	pywikipedia-ko_datacheck.py/r11317 Pywikipediabot/1.0 - -
		application/json
		application/xml
	pywikipedia-replace.pyc/r11215 Pywikipediabot/1.0 - -
		application/json
		application/xml
	pywikipedia-interwiki.py/r11265 Pywikipediabot/1.0 - -
		application/json
		application/xml
	pywikipedia-ebham.py/r11321 Pywikipediabot/1.0 - -
		application/json
	WikiBot/0.1 - -
		text/..
	Zing-BottaBot/2.0 - -
		text/..
	trunk-redirect.py/r11322 Pywikipediabot/1.0 - -
		application/json
	SiocWikiBot/1.0 - -
		application/vnd.php.serialized
		text/..
	TVersity Media Robot - -
		text/..
	(u'python-wikitools/1.1.1 (User:Cerabot)',) - -
		application/json
	wikidata_blank_items/r-1 (unknown) Pywikipediabot/2.0 - -
		application/json
	Wiki.java 0.27 r134 (OctraBot 2.3) - -
		text/..
	www.monit24.pl-m24Bot/4.1- - -
		image/..
		text/..
	pywikt-interwiki.py/r11216 (wikipedia.py) Pywikipediabot/1.0 - -
		application/xml
		application/json
	Empedia Bot - -
		text/..
	MediaWiki::Bot/3.005002 - -
		application/json
	FAST Search Web Crawler 14.0.0325.0000 - -
		text/..
		-
	bot: fr-anal - -
		application/json
	usrlab-BotReversor.py/r11321 Pywikipediabot/1.0 - -
		application/json
	pywikipedia-archivebot.py/r11321 Pywikipediabot/1.0 - -
		application/json
	360spider-image - -
		image/..
		text/..
	Bub's wikibot (Wikibot/2013031408; JWBF/1.2; Java/1.7) - -
		text/..
	Twitterbot/1.0 - -
		text/..
	hercule-commonscat.py/r11321 Pywikipediabot/1.0 - -
		application/json
	wikidata_properties/r-1 (unknown) Pywikipediabot/2.0 - -
		application/json
	Wikibot/2.0.2 CFNetwork/609 Darwin/13.0.0 en-us -
		image/..
		application/json
		text/..
	HRoestBot, de-wikipedia using pywikipedia framework - -
		text/..
		application/json
	notifyDisam/r11277 Pywikipediabot/2.0 - -
		application/json
	HosiryuhosiBot IRC-RecentChanges Checker ja -
		text/..
	Phantom.js bot cs-CZ,en,* -
		image/..
		text/..
	pywikipedia-zzlangcat.py/r11265 Pywikipediabot/1.0 - -
		application/json
	Mozilla/5.0 (Windows; Windows NT 5.1; zh-CN; rv:1.8.0.11) Firefox/1.5.0.11; 360Spider - -
		text/..
		application/json
	Webwiki Search Engine Bot - www.webwiki.de - -
		text/..
	OpenSearchServer_Bot - -
		text/..
	commons-imagerecat.py/r11028 Pywikipediabot/1.0 - -
		application/json
	newser-wikinewser2.py/r11321 Pywikipediabot/1.0 - -
		application/json
	pywikipedia-info2.py/r11321 Pywikipediabot/1.0 - -
		application/json
	bot-VM-auto-erl.py/r11215 Pywikipediabot/1.0 - -
		application/json
	pywikipedia-wd_move.py/r11308 Pywikipediabot/1.0 - -
		application/json
	user_scripts-BotReversor.py/r11147 Pywikipediabot/1.0 - -
		application/json
	A .NET Web Crawler - -
		text/..
	Mozilla/5.0 (Windows; Windows NT 5.1; fr; rv:1.8.1) VoilaBot BETA 1.2 ( mail address ) - -
		text/..
	milog_bot/1.0 ( mail address ) - -
		text/..
	ybot-ko_datastat.py/r11308 Pywikipediabot/1.0 - -
		application/json
		application/xml
	pywikipedia-featured.py/r11311 Pywikipediabot/1.0 - -
		application/json
	AnomieBOT 1.0 (BAGBot; see [[User:AnomieBOT]]) - -
		application/json
		text/..
	Epywikipedia-welcome.py/r11181 Pywikipediabot/1.0 - -
		application/json
	trunk-redirect.py/r11308 Pywikipediabot/1.0 - -
		application/json
		application/xml
	pywikipedia-welcome2.py/r11321 Pywikipediabot/1.0 - -
		application/json
	pywikipedia-replace.py/r11307 (wikipedia.py) Pywikipediabot/1.0 - -
		application/xml
		application/json
	pywikipedia-welcome.py/r11027 Pywikipediabot/1.0 - -
		application/json
	JavaCrawler/1.1 - -
		text/..
	wikbotlite/2.0 CFNetwork/609.1.4 Darwin/13.0.0 en-us -
		application/json
		image/..
		text/..
	pywikipedia-17-progreso1911.py/r11308 Pywikipediabot/1.0 - -
		application/json
		application/xml
	~Bot ([[:fr:w:User:TildeBot]] by [[:fr:w:User:Alphos]] mail address ) - -
		text/..
	DotNetWikiBot/2.100 (Unix 3.0.0.12; ) - -
		text/..
	ipipan.waw.pl - nekstbot - -
		text/..
	fa-welcome.py/r11054 Pywikipediabot/1.0 - -
		application/json
	Mozilla/5.0 (SnapPreviewBot) Gecko/20061206 Firefox/1.5.0.9 en-us,en;q=0.5 -
		image/..
		text/..
	pywikipedia-ko_datacheck.py/r11308 Pywikipediabot/1.0 - -
		application/json
	COIBot/1.00 - -
		text/..
	Pywikipedia-anagrama.py/r11215 Pywikipediabot/1.0 - -
		application/json
	admin-ipblocker-auto.py/r11281 Pywikipediabot/1.0 - -
		application/json
	update-task-categories/r11182 (pywikibot/__init__.py) Pywikipediabot/2.0 - -
		application/json
	Pywikipedia-new_interwiki.py/r11255 Pywikipediabot/1.0 - -
		application/json
	AnomieBOT 1.0 (NewArticleAFDTagger; see [[User:AnomieBOT]]) - -
		application/json
	pywikipedia-welcome.py/r11321 Pywikipediabot/1.0 - -
		application/json
	pywikipedia-ebham.py/r11308 Pywikipediabot/1.0 - -
		application/json
	GoogleBot - -
		text/..
	ip-web-crawler.com en-us -
		text/..
	pywikipedia-interwiki.py/r11322 Pywikipediabot/1.0 - -
		application/json
		application/xml
	templateswdl/r11208 Pywikipediabot/2.0 - -
		application/json
	pywikipedia-wikidata_irc_global.py/r11311 Pywikipediabot/1.0 - -
		application/json
	pywikipedia-wikidata_disamb_dp.py/r11200 Pywikipediabot/1.0 - -
		application/json
	pywikipedia-archivebot.py/r11308 Pywikipediabot/1.0 - -
		application/json
	pywikipedia-wikidata9nwd.py/r11181 (wikipedia.py) Pywikipediabot/1.0 - -
		application/xml
		application/json
	Mozilla 5.0 (Apibot 0.30b5) - -
		application/vnd.php.serialized
	Mozilla/5.0 (X11; Linux i686; en-US; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 SnapPreviewBot en-us,en;q=0.5 -
		text/..
	Py-new_interwiki.py/r11278 Pywikipediabot/1.0 - -
		application/json
	DotNetWikiBot, edited by D. Rodionov/2.91 (Microsoft Windows NT 6.0.6002 Service Pack 2; ) - -
		text/..
		application/xml
	pywikipedia-bot_control.py/r504 Pywikipediabot/1.0 - -
		application/json
		application/xml
	EarwigBot/0.2.dev.gitf082fca7 (Python/2.7.3; https://github.com/earwig/earwigbot; mail address ) - -
		application/json
		text/..
	ybot-ko_newpage.py/r11321 Pywikipediabot/1.0 - -
		application/json
	pywikipedia-interwiki.py/r11308 Pywikipediabot/1.0 - -
		application/json
		application/xml
	DeletionBot-deletion.py/r11275 Pywikipediabot/1.0 - -
		application/json
		application/xml
	YBot/0.1 - -
		application/vnd.php.serialized
	ybot-cat_copy.py/r11321 Pywikipediabot/1.0 - -
		application/json
	pywikipedia-category_redirect.py/r11321 Pywikipediabot/1.0 - -
		application/json
		application/xml
	Mozilla/5.001 (windows; NT4.0; en-US; rv:1.0) Gecko/25250101 - mail address - -
		text/..
	pywikipedia-wawiwtable.py/r11307 (wikipedia.py) Pywikipediabot/1.0 - -
		application/json
	python-wikitools/1.2 (User:Mr.Z-bot) - -
		application/json
	Metabot 0.1 - -
		text/..
	CodeGator Crawler v1.0 - -
		text/..
	COIBot/2.0 - -
		text/..
	pywikipedia-interwiki.py/r11231 (wikipedia.py) Pywikipediabot/1.0 - -
		application/xml
		application/json
	Mozilla/5.0 (compatible; FriendFeedBot/0.1; Http://friendfeed.com/about/bot; 416 subscribers; feed-id=3852576738117026533) - -
		application/xml
		-
	pywikipedia-featured.py/r11325 Pywikipediabot/1.0 - -
		application/json
	pywikipedia-zzlangcat.py/r11322 Pywikipediabot/1.0 - -
		application/json
	process_mlb/r-1 (unknown) Pywikipediabot/2.0 - -
		application/json
	AnomieBOT 1.0 (RandomPagePicker; see [[User:AnomieBOT]]) - -
		application/json
	Mozilla/5.0 (compatible; SnapPreviewBot; en-US; rv:1.8.0.9) Gecko/20061206 Firefox/1.5.0.9 en-us,en;q=0.5 -
		text/..
	theWxitBot/0.1 - -
		application/json
		image/..
	Bloom.fm Meta Crawler ( mail address ) - -
		text/..
	Wikibot/2.0.2 CFNetwork/609.1.4 Darwin/13.0.0 en-gb -
		image/..
		application/json
		text/..
	AnomieBOT 1.0 (PUICloser; see [[User:AnomieBOT]]) - -
		application/json
	Mozilla/5.0 (compatible; MyBot/1.0;) - -
		application/json
	Wikibot/2.0.2 CFNetwork/609.1.4 Darwin/13.0.0 de-de -
		image/..
		application/json
		text/..
	pywikipedia-klbot_wikiviajes4.py/r11311 Pywikipediabot/1.0 - -
		application/json
		application/xml
	pywikipediabot-wikidata_irc_global.py/r11215 Pywikipediabot/1.0 - -
		application/json
	bitlybot - -
		text/..
		image/..
	Local Site Parser 1.0 en-us,en;q=0.5 -
		text/..
	newser-wikinewser2.py/r11308 Pywikipediabot/1.0 - -
		application/json
	Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.5 (Exabot-Thumbnails) en;q=0.9,*;q=0.8 -
		text/..
	Wikibot/2.0.2 CFNetwork/548.1.4 Darwin/11.0.0 en-us -
		image/..
		application/json
	DotNetWikiBot/2.96 (Microsoft Windows NT 6.1.7601 Service Pack 1; ) - -
		text/..
		application/xml
	Arkbot/0.1 alpha - -
		application/json
	pywikipedia-iw2data.py/r11308 Pywikipediabot/1.0 - -
		application/json
	TwynCatBot/0.2 (Contact: www.twyn.com) - -
		application/json
	AsgardBot - DotNetWikiBot/2.100 (Microsoft Windows NT 6.1.7601 Service Pack 1; ) - -
		text/..
	('python-wikitools/1.1.1 (User:BernsteinBot)',) - -
		application/json
	Mozilla/5.0 (compatible; Cliqusbot/0.1) en,*;q=0.5 -
		text/..
	UCMore Crawler App en-us,en;q=0.5 -
		text/..
	Peachy MediaWiki Bot API Version 0.1beta - -
		application/vnd.php.serialized
	HTMLParser/1.6 - -
		text/..
	misc-tasks-my_replace.py/r11322 Pywikipediabot/1.0 - -
		application/xml
		application/json
	BuiBui-Bot/1.0 (email: mail address ) - -
		text/..
	Wikibot/2.0.2 CFNetwork/609.1.4 Darwin/13.0.0 zh-cn -
		text/..
		application/json
		image/..
	Zing-BottaBot/1.0 - -
		text/..
	rmShortArtsTag-rmShortArtsTag/r11130 Pywikipediabot/1.0 - -
		application/json
	OmnitureTestAndTargetCrawl/Nutch-1.6 (Nutch/1.6) en-us,en-gb,en;q=0.7,*;q=0.3 -
		text/..
	Mozilla/5.0 (Bgbot 0.5) - -
		text/..
	My Nutch Spider/Nutch-1.6 en-us,en-gb,en;q=0.7,*;q=0.3 -
		text/..
	pywikipedia-welcome2.py/r11308 Pywikipediabot/1.0 - -
		application/json
	UniversalFeedParser/5.1.1 https://code.google.com/p/feedparser/ - -
		text/..
		application/xml
	Pywikipedia-dikantenyvaovao.py/r11215 Pywikipediabot/1.0 - -
		application/json
	translation/r11235 Pywikipediabot/2.0 - -
		application/json
	python-wikitools/1.2 (User:LaraBot) - -
		application/json
	DotNetWikiBot/2.97 (Microsoft Windows NT 6.1.7601 Service Pack 1; ) - -
		text/..
	Python27-pythonw.exe/r11275 Pywikipediabot/1.0 - -
		application/json
	request-daemon/r11182 (pywikibot/__init__.py) Pywikipediabot/2.0 - -
		application/json
	Wikibot/2.0.1 CFNetwork/609 Darwin/13.0.0 en-gb -
		image/..
		application/json
	Geni ircpybot 1.0 - -
		application/json
47595.91	total

IP ranges: known ip ranges for Google are 64.233.[160.0-191.255], 66.249.[64.0-95.255], 66.102.[0.0-15.255], 72.14.[192.0-255.255],
74.125.[0.0-255.255], 209.085.[128.0-255.255], 216.239.[32.0-63.255] and a few minor other subranges

Errata: WMF traffic logging service suffered from server capacity problems in Aug/Sep/Oct 2011.
Absolute traffic counts for October 2011 are approximatly 7% too low.
Data loss only occurred during peak hours. It therefore may have had somewhat different impact for traffic from different parts of the world.
and may have also skewed relative figures like share of traffic per browser or operating system.

From mid September till late November squid log records for mobile traffic were in invalid format.
Data could be repaired for logs from mid October onwards. Older logs were no longer available.

In a an unrelated server outage precisely half of traffic to WMF mobile sites was not counted from Oct 16 - Nov 29 (one of two load-balanced servers did not report traffic).
WMF has since improved server monitoring, so that similar outages should be detected and fixed much faster from now on.

Generated on Mon, Jun 17, 2013 17:32
Author:Erik Zachte (Web site)
Mail: ezachte@### (no spam: ### = wikimedia.org)
All data and images on this page are in the public domain.

Note: page may load slower on Microsoft Internet explorer than on other major browsers