Wikimedia Traffic Analysis Report - Crawler requests

Monthly requests or daily averages, for period: 1 May 2013 - 31 May 2013 (last 12 months)
000 ⇒ k
 
 This analysis is based on a 1:1000 sampled server log (squids)

Warning: all recent Wikimedia traffic analysis reports have been generated from old scripts.

The scripts are orphaned, and have not been maintained for at least 6 months. Many bugs are considerably older.
Known Bugzilla bugs: 46190, 46191, 46195, 46201, 46205, 46265, (46267), 46268, 46269, 46271, 46273, 46274, 46275, 46277, 46278, 46279, 46289

 See also: Requests by destination or by origin / Methods / Scripts / User agents / Skins / Crawlers / Op.Sys. / Mobile devices / Browsers / Google / Country data / Traffic trends, and notes about reliability of these data

The following overview of crawler (aka bot) page requests is based on the user agent information that accompanies most server requests. Unfortunately this user agent information follows rather loosely defined guidelines.
Also please bear in mind than the most popular crawler names may be somewhat overrepresented. This is the result of so called user agent spoofing (where a requester supplies false credentials, e.g. to bypass web servers filters).
GoogleBot seems to be a favorite for spoofing. Therefore requests from an ip address registered by Google (see below) are color coded GoogleBot, others GoogleBot

For this report page requests are considered to be issued by a crawler in two cases:
1 The user agent string contains a web address (only crawlers should have that, but there a some false positives, where a browser sends a user agent string with a web address (ill behaved plug-in, main offenders have been eliminated)
2 The user agent string contains the term bot, spider or crawl[er]'

In total 114,154,480 page requests (mime type text/html only!) per day are considered crawler requests, out of 626,751,160 external requests, which is 18.2%

Page requests for crawlers that specify a url in the agent string
Count
x 1000		Secondary domain
(~site) name		URLMime typeUser agent
google
 www.google.com/bot.htmltext/..Mozilla/5.0 (compatible; GoogleBot/2.1; url) - -
 www.google.com/bot.htmltext/..Mozilla/5.0 (iPhone; CPU iPhone OS 4_1 like Mac OS X; en-us) AppleWebKit/532.9 KHTML Version/4.0.5 Mobile/8B117 Safari/6531.22.7 (compatible; GoogleBot-Mobile/2.1; url) - -
 www.google.com/bot.html-Mozilla/5.0 (compatible; GoogleBot/2.1; url) - -
 www.google.com/bot.html-Mozilla/5.0 (iPhone; CPU iPhone OS 4_1 like Mac OS X; en-us) AppleWebKit/532.9 KHTML Version/4.0.5 Mobile/8B117 Safari/6531.22.7 (compatible; GoogleBot-Mobile/2.1; url) - -
 desktop.google.com/application/xmlMozilla/5.0 (compatible; Google Desktop/5.9.1005.12335; url) - -
 www.google.com/bot.htmlimage/..Mozilla/5.0 (compatible; GoogleBot/2.1; url) - -
 www.google.com/feedfetcher.htmlimage/..Mozilla/5.0 (compatible) FeedFetcher-Google; (url) - -
 desktop.google.com/image/..Mozilla/5.0 (compatible; Google Desktop/5.9.1005.12335; url) - -
 code.google.com/p/crawler4j/text/..crawler4j (url) - -
 www.google.com/feedfetcher.htmlapplication/xmlFeedFetcher-Google; (url) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: ortografia4) - -
 desktop.google.com/text/..Mozilla/5.0 (compatible; Google Desktop/5.9.1005.12335; url) - -
 www.google.com/feedfetcher.html-FeedFetcher-Google; (url) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~world-news-devel) - -
 www.google.com/feedfetcher.htmltext/..Mozilla/5.0 (compatible) FeedFetcher-Google; (url) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: ortopedianew) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: rarplayer) - -
 www.google.com/feedfetcher.htmltext/..FeedFetcher-Google; (url) - -
 code.google.com/appenginetext/..WikiBot/0.1 AppEngine-Google; (url; appid: newikipedia) - -
 docs.google.comimage/..Mozilla/5.0 (compatible; GoogleDocs; apps-presentations; url) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: wikien3) - -
 www.google.com/feedfetcher.htmlapplication/jsonMozilla/5.0 (compatible) FeedFetcher-Google; (url) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: wikien4) - -
 docs.google.comimage/..Mozilla/5.0 (compatible; GoogleDocs; documents; url) - -
 www.google.com/feedfetcher.htmlapplication/xmlMozilla/5.0 (compatible) FeedFetcher-Google; (url) - -
 code.google.com/appengineapplication/xmlAppEngine-Google; (url; appid: wikipedia-raw) - -
 www.google.com/bot.htmltext/..SAMSUNG-SGH-E250/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 UP.Browser/6.2.3.3.c.1.101 (GUI) MMP/2.0 (compatible; GoogleBot-Mobile/2.1; url) - -
 www.google.com/bot.htmltext/..DoCoMo/2.0 N905i(c100;TB;W24H16) (compatible; GoogleBot-Mobile/2.1; url) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~kasumiremix) - -
 desktop.google.com/application/xmlMozilla/5.0 (compatible; Google Desktop/5.9.909.30391; url) - -
 www.google.com/feedfetcher.htmltext/..Mozilla/5.0 (compatible) FeedFetcher-Google;(url) - -
 code.google.com/appenginetext/..Mozilla/5.0 (Windows; Windows NT 5.1; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 AppEngine-Google; (url; appid: s~fonetika3) - -
 code.google.com/appenginetext/..Java/1.6.0_27 AppEngine-Google; (url; appid: s~1989-0723-6022) - -
 code.google.com/appenginetext/..Java/1.6.0_27 AppEngine-Google; (url; appid: s~1989-0723-6031) - -
 code.google.com/appenginetext/..Java/1.6.0_27 AppEngine-Google; (url; appid: s~1989-0723-6028) - -
 code.google.com/appenginetext/..Java/1.6.0_27 AppEngine-Google; (url; appid: s~1989-0723-6030) - -
 code.google.com/appenginetext/..Java/1.6.0_27 AppEngine-Google; (url; appid: s~1989-0723-6024) - -
 code.google.com/appenginetext/..Acre/dev/51:889 ubiquity.freebaseapps.com AppEngine-Google; (url; appid: s~freebaseapps) - -
 code.google.com/appenginetext/..Java/1.6.0_27 AppEngine-Google; (url; appid: s~1989-0723-6026) - -
 www.google.com/bot.htmltext/..GoogleBot/2.1 (url) - -
 code.google.com/appengineapplication/jsonAppEngine-Google; (url; appid: s~redconceptual) - -
 code.google.com/appenginetext/..Java/1.6.0_27 AppEngine-Google; (url; appid: s~1989-0723-6029) - -
 www.google.com/feedfetcher.htmlimage/..Mozilla/5.0 (compatible) FeedFetcher-Google;(url) - -
 code.google.com/appenginetext/..Java/1.6.0_27 AppEngine-Google; (url; appid: s~1989-0723-6023) - -
 code.google.com/appenginetext/..myapp2 AppEngine-Google; (url; appid: s~robinjchengecnu) - -
 www.google.com/bot.htmlimage/..Mozilla/5.0 (iPhone; CPU iPhone OS 4_1 like Mac OS X; en-us) AppleWebKit/532.9 KHTML Version/4.0.5 Mobile/8B117 Safari/6531.22.7 (compatible; GoogleBot-Mobile/2.1; url) - -
 code.google.com/appengineapplication/jsonUser-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.5 KHTML Chrome/19.0.1084.56 Safari/536.5 AppEngine-Google; (url; appid: s~in-onda) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~japantiki) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: d24-img) - -
 www.google.com/bot.html-SAMSUNG-SGH-E250/1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 UP.Browser/6.2.3.3.c.1.101 (GUI) MMP/2.0 (compatible; GoogleBot-Mobile/2.1; url) - -
 www.google.com/coop/cse/creftext/..PageFetcher-Google-CoOp;((url) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~cymanconsole) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: usawebdl) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: boxapp) - -
 code.google.com/appenginetext/..myapp2 AppEngine-Google; (url; appid: s~robinwikispider) - -
 www.google.com/bot.html-DoCoMo/2.0 N905i(c100;TB;W24H16) (compatible; GoogleBot-Mobile/2.1; url) - -
 code.google.com/appenginetext/..myapp2 AppEngine-Google; (url; appid: s~robinspider2) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~espanatiki) - -
 code.google.com/appenginetext/..myapp2 AppEngine-Google; (url; appid: s~chengjirobin) - -
 code.google.com/appenginetext/..Java/1.6.0_27 AppEngine-Google; (url; appid: s~1989-0723-6025) - -
 code.google.com/appenginetext/..myapp2 AppEngine-Google; (url; appid: s~ecnugofree) - -
 code.google.com/appengineimage/..AppEngine-Google; (url; appid: s~kasumiremix) - -
 code.google.com/appenginetext/..myapp2 AppEngine-Google; (url; appid: s~robinspider3) - -
 code.google.com/appenginetext/..Python-urllib/2.5 AppEngine-Google; (url; appid: s~isnt-it) - -
 code.google.com/appenginetext/..myapp2 AppEngine-Google; (url; appid: s~robinspider4) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: abdulfat) - -
 code.google.com/appenginetext/..myapp2 AppEngine-Google; (url; appid: s~phantomecnu) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~francetiki) - -
 code.google.com/appengine-AppEngine-Google; (url; appid: s~cymanconsole) - -
 www.google.com/bot.htmlNONE/wikipedia- Mozilla/5.0 (compatible; GoogleBot/2.1; url) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: wwwwebp0) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~misterhac) - -
 code.google.com/appenginetext/..myapp2 AppEngine-Google; (url; appid: s~ecnuseispider2) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: toom16-10) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~batshiitinsane) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: threewiki) - -
 code.google.com/appengineimage/..AppEngine-Google; (url; appid: d24-img) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: webproxy8-9) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: tusawebproxy4) - -
 code.google.com/appenginetext/..myapp2 AppEngine-Google; (url; appid: s~ecnuseispider) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~app3123ak) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: wwwwebp8) - -
 code.google.com/appenginetext/..www.productontology.org/1.0 (Contact: martin.heppATunibw.de) AppEngine-Google; (url; appid: gr4bing) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: worldwide-propaganda) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~samson-server) - -
 code.google.com/appenginetext/..myapp2 AppEngine-Google; (url; appid: s~ecnuseispider3) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: kikopea-openproxy) - -
 code.google.com/appengineimage/..AppEngine-Google; (url; appid: s~batshiitinsane) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: 100thpriest) - -
 docs.google.comimage/..Mozilla/5.0 (compatible; GoogleDocs; drawings; url) - -
 www.google.com/bot.htmlapplication/pdfMozilla/5.0 (iPhone; CPU iPhone OS 4_1 like Mac OS X; en-us) AppleWebKit/532.9 KHTML Version/4.0.5 Mobile/8B117 Safari/6531.22.7 (compatible; GoogleBot-Mobile/2.1; url) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: wwwwebp2) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~drizzlprox) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~italiatiki) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: 9000tunnels) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~theunblock) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~repcore-prod) - -
 code.google.com/appenginetext/..myapp2 AppEngine-Google; (url; appid: s~ecnuspider8) - -
 desktop.google.com/image/..Mozilla/5.0 (compatible; Google Desktop/5.9.911.3589; url) - -
 code.google.com/appenginetext/..myapp2 AppEngine-Google; (url; appid: s~ecnuspider4) - -
 docs.google.com-Mozilla/5.0 (compatible; GoogleDocs; apps-presentations; url) - -
 www.google.com/feedfetcher.htmltext/..FeedFetcher-Google; (url) ja,en-us;q=0.7,en;q=0.3 -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: webponline9) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: discretepword) - -
 code.google.com/appenginetext/..myapp2 AppEngine-Google; (url; appid: s~ecnuspider5) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: proxyusing121) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: webduplicate) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: my-api) - -
 code.google.com/appenginetext/..Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_2) AppleWebKit/535.7 KHTML Chrome/16.0.912.77 Safari/535.7 AppEngine-Google; (url; appid: s~feedly-social) - -
 www.google.com/feedfetcher.html-Mozilla/5.0 (compatible) FeedFetcher-Google; (url) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: webponline0) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: dex-dwds) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: pakgalaxy) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~alucardtl) - -
 code.google.com/appenginetext/..myapp2 AppEngine-Google; (url; appid: s~ecnuspider6) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: kuryproxy) - -
 code.google.com/appengineapplication/jsonMWBOT GAE Edition AppEngine-Google; (url; appid: philip-bot) - -
 code.google.com/appenginetext/..myapp2 AppEngine-Google; (url; appid: s~ecnuspider7) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: chris-homework-helper) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: hideproxyz) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: kires-roxy) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: simple-tools2) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: easypox) - -
 www.google.com/bot.htmlapplication/pdfMozilla/5.0 (compatible; GoogleBot/2.1; url) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: paradigm-web-proxy) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: simple-tools6) - -
 code.google.com/appenginetext/.. mail address AppEngine-Google; (url; appid: s~wiki-sherpa) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: webproxy8-2) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~kushgenius) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: proxy-devakishor) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~nyinayminproxy2) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: varlopie) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: usawebdl3) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~zagrobelnyprox) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: jptaravellahighschool) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: kaveriselvaraj) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~kyaysarlay) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: atxproxy) - -
 code.google.com/appengineimage/..AppEngine-Google; (url; appid: s~misterhac) - -
 desktop.google.com/application/xmlMozilla/5.0 (compatible; Google Desktop/5.9.911.3589; url) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~vulture-engine) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: 9329269) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: webusadlq0) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: nothing-unusual) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: weps002) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~sl-santabanta) - -
 code.google.com/appengineimage/..AppEngine-Google; (url; appid: s~doatresizer) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~proxyseekkety) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~neidlingermj1) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: ax4413) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: bassgnt) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~gg8mm8qq) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: argim-free) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: arturoproxy) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: ridemyhell) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: ivegotalovelybunch) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~adddon1) - -
 desktop.google.com/-Mozilla/5.0 (compatible; Google Desktop/5.9.1005.12335; url) -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~liquid-helium) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: clh-sw) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: webusadlq3) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: mboharsik) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: webproxy8-5) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: webusadlp8) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: cmd-proxy) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: free-data) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: thinkersearch) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~ahsanovic) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: s~s-alexander) - -
 code.google.com/appenginetext/..AppEngine-Google; (url; appid: webponline5) - -
bing
 www.bing.com/bingbot.htmtext/..Mozilla/5.0 (compatible; bingbot/2.0; url) - -
 www.bing.com/bingbot.htm-Mozilla/5.0 (compatible; bingbot/2.0; url) - -
 www.bing.com/bingbot.htmimage/..Mozilla/5.0 (compatible; bingbot/2.0; url) - -
 www.bing.com/bingbot.htm-Mozilla/5.0 (compatible; bingbot/2.0; url) -
 www.bing.com/bingbot.htmapplication/jsonMozilla/5.0 (compatible; bingbot/2.0; url) - -
 www.bing.com/bingbot.htmtext/..Mozilla/5.0 (compatible; bingbot/2.0; url) -
 www.bing.com/bingbot.htmtext/..Mozilla/5.0 (compatible; bingbot/2.0; url) (via Web-Blaster/2.21 (http://www.assoziations-blaster.de/web-blast.html)) - -
 www.bing.com/bingbot.htmtext/..Mozilla/5.0 (compatible; bingbot/2.0; url) (via Web-Blaster/2.21 (http://www.a-blast.org/web-blast.html)) - -
 www.bing.com/bingbot.htmtext/..Mozilla/5.0 (compatible; bingbot/2.0; url) AppEngine-Google; (http://code.google.com/appengine; appid: wxcity1) - -
 www.bing.com/bingbot.htmtext/..Mozilla/5.0 (compatible; bingbot/2.0; url) AppEngine-Google; (http://code.google.com/appengine; appid: proxydisk) - -
 www.bing.com/bingbot.htmtext/..Mozilla/5.0 (compatible; bingbot/2.0; url) AppEngine-Google; (http://code.google.com/appengine; appid: appcodes) - -
 www.bing.com/bingbot.htmtext/..Mozilla/5.0 (compatible; bingbot/2.0; url) AppEngine-Google; (http://code.google.com/appengine; appid: surfproxy4) - -
 www.bing.com/bingbot.htmtext/..Mozilla/5.0 (compatible; bingbot/2.0; url) AppEngine-Google; (http://code.google.com/appengine; appid: surf603) - -
 www.bing.com/bingbot.htmtext/..Mozilla/5.0 (compatible; bingbot/2.0; url) AppEngine-Google; (http://code.google.com/appengine; appid: surfproxy6) - -
 www.bing.com/bingbot.htmtext/..Mozilla/5.0 (compatible; bingbot/2.0; url) AppEngine-Google; (http://code.google.com/appengine; appid: bereturning) - -
 www.bing.com/bingbot.htmapplication/xmlMozilla/5.0 (compatible; bingbot/2.0; url) - -
 www.bing.com/bingbot.htmtext/..Mozilla/5.0 (compatible; bingbot/2.0; url) AppEngine-Google; (http://code.google.com/appengine; appid: proxydisk8) - -
 www.bing.com/bingbot.htmtext/..Mozilla/5.0 (compatible; bingbot/2.0; url) AppEngine-Google; (http://code.google.com/appengine; appid: cgiproxy6) - -
 www.bing.com/bingbot.htmtext/..Mozilla/5.0 (compatible; bingbot/2.0; url) AppEngine-Google; (http://code.google.com/appengine; appid: proxydisk9) - -
 www.bing.com/bingbot.htmtext/..Mozilla/5.0 (compatible; bingbot/2.0; url) AppEngine-Google; (http://code.google.com/appengine; appid: howtodoita) - -
 www.bing.com/bingbot.htmapplication/oggMozilla/5.0 (compatible; bingbot/2.0; url) - -
 www.bing.com/bingbot.htmtext/..Mozilla/5.0 (compatible; bingbot/2.0; url) AppEngine-Google; (http://code.google.com/appengine; appid: yourrevenues) - -
 www.bing.com/bingbot.htmtext/..Mozilla/5.0 (compatible; bingbot/2.0; url) AppEngine-Google; (http://code.google.com/appengine; appid: proxy6009) - -
 www.bing.com/bingbot.htmtext/..Mozilla/5.0 (compatible; bingbot/2.0; url) AppEngine-Google; (http://code.google.com/appengine; appid: proxyfile8) - -
 www.bing.com/bingbot.htmtext/..Mozilla/5.0 (compatible; bingbot/2.0; url) AppEngine-Google; (http://code.google.com/appengine; appid: surf702) - -
 www.bing.com/bingbot.htmtext/..Mozilla/5.0 (compatible; bingbot/2.0; url) AppEngine-Google; (http://code.google.com/appengine; appid: proxy5070) - -
 www.bing.com/bingbot.htmtext/..Mozilla/5.0 (compatible; bingbot/2.0; url) AppEngine-Google; (http://code.google.com/appengine; appid: response-out) - -
 www.bing.com/bingbot.htmtext/..Mozilla/5.0 (compatible; bingbot/2.0; url) AppEngine-Google; (http://code.google.com/appengine; appid: surf712) - -
 www.bing.com/bingbot.htmtext/..Mozilla/5.0 (compatible; bingbot/2.0; url) AppEngine-Google; (http://code.google.com/appengine; appid: scroll-down) - -
 www.bing.com/bingbot.htmtext/..Mozilla/5.0 (compatible; bingbot/2.0; url) AppEngine-Google; (http://code.google.com/appengine; appid: surfproxy8) - -
 www.bing.com/bingbot.htmtext/..(compatible; bingbot/2.0; url) -
facebook
 www.facebook.com/externalhit_uatext.phpimage/..facebookexternalhit/1.1 (url) -
 www.facebook.com/externalhit_uatext.phptext/..facebookexternalhit/1.1 (url) -
 www.facebook.com/externalhit_uatext.phptext/..facebookexternalhit/1.0 (url) -
 www.facebook.com/externalhit_uatext.phpimage/..facebookexternalhit/1.1 (url) - -
 www.facebook.com/externalhit_uatext.phptext/..facebookexternalhit/1.1 (url) - -
 www.facebook.com/externalhit_uatext.php-facebookexternalhit/1.1 (url) -
 www.facebook.com/externalhit_uatext.phpimage/..facebookexternalhit/1.0 (url) -
 www.facebook.com/externalhit_uatext.phptext/..facebookexternalhit/1.0 (url) - -
 developers.facebook.comimage/..facebookplatform/1.0 (url) -
 www.facebook.com/externalhit_uatext.php-facebookexternalhit/1.1 (url) - -
 www.facebook.com/externalhit_uatext.phpimage/..facebookexternalhit/1.0 (url) - -
 developers.facebook.comimage/..facebookplatform/1.0 (url) - -
 www.facebook.com/externalhit_uatext.php-facebookexternalhit/1.0 (url) -
google?
 www.google.com/bot.htmltext/..Mozilla/5.0 (compatible; GoogleBot/2.1; url) - -
 www.google.com/bot.htmltext/..Mozilla/5.0 (iPhone; CPU iPhone OS 4_1 like Mac OS X; en-us) AppleWebKit/532.9 KHTML Version/4.0.5 Mobile/8B117 Safari/6531.22.7 (compatible; GoogleBot-Mobile/2.1; url) - -
 www.google.com/bot.htmltext/..GoogleBot/2.1 (url) - -
 www.google.com/bot.html-Mozilla/5.0 (compatible; GoogleBot/2.1; url) - -
 www.google.com/bot.htmlimage/..Mozilla/5.0 (compatible; GoogleBot/2.1; url) - -
 www.google.com/bot.html-Mozilla/5.0 (iPhone; CPU iPhone OS 4_1 like Mac OS X; en-us) AppleWebKit/532.9 KHTML Version/4.0.5 Mobile/8B117 Safari/6531.22.7 (compatible; GoogleBot-Mobile/2.1; url) - -
 www.google.com/bot.html-Mozilla/5.0 (compatible; GoogleBot/2.1; url) -
 www.google.com/bot.htmlapplication/jsonMozilla/5.0 (compatible; GoogleBot/2.1; url) - -
 www.google.com/bot.htmltext/..GoogleBot/2.1 (url) en-us,en;q=0.5 -
 www.google.com/bot.htmltext/..Mozilla/5.0 (compatible; GoogleBot/2.1; url) zh-CN,zh;q=0.8 -
 www.google.com/bot.htmltext/..Mozilla/5.0 (compatible; GoogleBot/2.1; url) bg -
 www.google.com/bot.htmltext/..DoCoMo/2.0 N905i(c100;TB;W24H16) (compatible; GoogleBot-Mobile/2.1; url) - -
 www.google.com/bot.htmltext/..(compatible; GoogleBot/2.1; url) -
 www.google.com/bot.html-DoCoMo/2.0 N905i(c100;TB;W24H16) (compatible; GoogleBot-Mobile/2.1; url) - -
 www.google.com/bot.htmlapplication/jsonGoogleBot/2.1 (url) en-us,en;q=0.5 -
 www.google.com/bot.htmlimage/..Mozilla/5.0 (compatible; GoogleBot/2.1; url) en-US,en;q=0.5 -
 www.google.com/bot.htmltext/..Mozilla/5.0 (compatible; GoogleBot/2.1; url) - -
baidu
 www.baidu.com/search/spider.htmltext/..Mozilla/5.0 (compatible; Baiduspider/2.0; url) zh-cn,zh-tw -
 www.baidu.com/search/spider.htmltext/..Mozilla/5.0 (compatible; Baiduspider/2.0; url) en-US -
 www.baidu.com/search/spider.htmltext/..Mozilla/5.0 (compatible; Baiduspider/2.0; url) - -
 www.baidu.com/search/spider.htmltext/..Mozilla/5.0 (Linux;u;Android/2.3.7;zh-cn;) AppleWebKit/533.1 (KHTML,like Gecko) Version/4.0 Mobile Safari/533.1 (compatible; url) - -
 www.baidu.com/search/spider.htmimage/..Baiduspider-image(url) - -
 www.baidu.com/search/spider.htmtext/..Baiduspider-image(url) - -
 www.baidu.com/search/spider.htmltext/..Mozilla/5.0 (compatible; Baiduspider/2.0; url) ja-JP,ja -
 www.baidu.com/search/spider.html-Mozilla/5.0 (compatible; Baiduspider/2.0; url) en-us,en;q=0.5 -
 www.baidu.com/search/spider.html-Mozilla/5.0 (compatible; Baiduspider/2.0; url) en-US -
 www.baidu.com/search/spider.html-Mozilla/5.0 (compatible; Baiduspider/2.0; url) zh-cn,zh-tw -
 www.baidu.com/search/spider.htmltext/..Mozilla/5.0 (compatible; Baiduspider/2.0; url) en-us,en;q=0.5 -
 www.baidu.com/search/spider.htmltext/..(compatible; Baiduspider/2.0; url) - -
 www.baidu.com/search/spider.html-Mozilla/5.0 (compatible; Baiduspider/2.0; url) -
 www.baidu.com/search/spider.htmlapplication/jsonMozilla/5.0 (compatible; Baiduspider/2.0; url) en-us,en;q=0.5 -
 www.baidu.com/search/spider.html-Mozilla/5.0 (compatible; Baiduspider/2.0; url) - -
 www.baidu.com/search/spider.htmltext/..Mozilla/5.0 (compatible; Baiduspider-cpro; url) - -
 www.baidu.com/search/spider.htmlapplication/xmlMozilla/5.0 (compatible; Baiduspider/2.0; url) zh-cn,zh-tw -
 www.baidu.com/search/spider.html-Mozilla/5.0 (Linux;u;Android/2.3.7;zh-cn;) AppleWebKit/533.1 (KHTML,like Gecko) Version/4.0 Mobile Safari/533.1 (compatible; url) en-us,en;q=0.5 -
 www.baidu.com/search/spider.htmlapplication/jsonMozilla/5.0 (compatible; Baiduspider/2.0; url) - -
 www.baidu.com/search/spider.htmltext/..Mozilla/5.0 (Linux;u;Android/2.3.7;zh-cn;) AppleWebKit/533.1 (KHTML,like Gecko) Version/4.0 Mobile Safari/533.1 (compatible; url) en-us,en;q=0.5 -
 www.baidu.com/search/spider.html-Mozilla/5.0 (compatible; Baiduspider/2.0; url) ja-JP,ja -
msn
 search.msn.com/msnbot.htmtext/..msnbot-media/1.1 (url) - -
 search.msn.com/msnbot.htmimage/..msnbot-media/1.1 (url) - -
 search.msn.com/msnbot.htmtext/..msnbot/2.0b (url) - -
 search.msn.com/msnbot.htmtext/..msnbot-NewsBlogs/2.0b (url) - -
 search.msn.com/msnbot.htmimage/..msnbot-NewsBlogs/2.0b (url) - -
 search.msn.com/msnbot.htm-msnbot-media/1.1 (url) - -
 search.msn.com/msnbot.htmtext/..msnbot-Products/1.0 (url) - -
 search.msn.com/msnbot.htmimage/..msnbot/2.0b (url) - -
 search.msn.com/msnbot.htmtext/..msnbot/0.01 (url) - -
 search.msn.com/msnbot.htmtext/..msnbot-UDiscovery/2.0b (url) - -
 search.msn.com/msnbot.htm-msnbot-media/1.1 (url) -
youdao
 www.youdao.com/help/webmaster/spider/text/..Mozilla/5.0 (compatible; YoudaoBot/1.0; url; ) - -
 www.youdao.com/help/webmaster/spider/-Mozilla/5.0 (compatible; YoudaoBot/1.0; url; ) -
 www.youdao.com/help/webmaster/spider/text/..Mozilla/5.0 (compatible; YoudaoBot/1.0; url; ) zh-cn;q=1.0, zh-tw;q=0.8, en;q=0.5, *;q=0.1 -
 www.youdao.com/help/webmaster/spider/-Mozilla/5.0 (compatible; YoudaoBot/1.0; url; ) zh-cn;q=1.0, zh-tw;q=0.8, en;q=0.5, *;q=0.1 -
 toolbar.youdao.com/image/..Youdao Toolbar (url) - -
yandex
 yandex.com/botstext/..Mozilla/5.0 (compatible; YandexBot/3.0; url) ru, uk;q=0.8, be;q=0.8, en;q=0.7, *;q=0.01 -
 yandex.com/botstext/..Mozilla/5.0 (compatible; YandexImages/3.0; url) ru, uk;q=0.8, be;q=0.8, en;q=0.7, *;q=0.01 -
 yandex.com/botstext/..Mozilla/5.0 (compatible; YandexDirect/3.0; url) ru, uk;q=0.8, be;q=0.8, en;q=0.7, *;q=0.01 -
 yandex.com/botstext/..Mozilla/5.0 (compatible; YandexBot/3.0; url) en-us, en;q=0.7, *;q=0.01 -
 yandex.com/bots-Mozilla/5.0 (compatible; YandexBot/3.0; url) ru, uk;q=0.8, be;q=0.8, en;q=0.7, *;q=0.01 -
 yandex.com/botstext/..Mozilla/5.0 (compatible; YandexBot/3.0; url) - -
 yandex.com/botstext/..Mozilla/5.0 (compatible; YandexImages/3.0; url) - -
 yandex.com/botsimage/..Mozilla/5.0 (compatible; YandexImages/3.0; url) ru, uk;q=0.8, be;q=0.8, en;q=0.7, *;q=0.01 -
 yandex.com/botsimage/..Mozilla/5.0 (compatible; YandexImages/3.0; url) - -
 yandex.com/botsimage/..Mozilla/5.0 (compatible; YandexImageResizer/2.0; url) - -
 yandex.com/botstext/..Mozilla/5.0 (compatible; YandexBot/3.0; url) de, en;q=0.7, *;q=0.01 -
 yandex.com/bots-Mozilla/5.0 (compatible; YandexImageResizer/2.0; url) -
 yandex.com/bots-Mozilla/5.0 (compatible; YandexBot/3.0; url) en-us, en;q=0.7, *;q=0.01 -
 yandex.com/botsimage/..Mozilla/5.0 (compatible; YandexImages/3.0; url) en-us, en;q=0.7, *;q=0.01 -
 yandex.com/botstext/..Mozilla/5.0 (compatible; YandexImages/3.0; url) en-us, en;q=0.7, *;q=0.01 -
 yandex.com/bots-Mozilla/5.0 (compatible; YandexImages/3.0; url) -
 yandex.com/botsimage/..Mozilla/5.0 (compatible; YandexAntivirus/2.0; url) ru-RU -
 yandex.com/bots-Mozilla/5.0 (compatible; YandexBot/3.0; url) -
yahoo
 help.yahoo.com/help/us/ysearch/slurpimage/..Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; url) NOT Firefox/3.5 en-us,en;q=0.5 -
 help.yahoo.com/help/us/ysearch/slurptext/..Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; url) NOT Firefox/3.5 en-us,en;q=0.5 -
 help.yahoo.com/help/us/ysearch/slurptext/..Mozilla/5.0 (compatible; Yahoo! Slurp; url) en-us,en;q=0.5 -
 help.yahoo.com/help/us/ysearch/slurptext/..Mozilla/5.0 (compatible; Yahoo! Slurp; url) - -
 help.yahoo.co.jp/help/jp/search/indexing/indexing-15.htmltext/..Y!J-BRJ/YATS crawler (url) - -
 help.yahoo.com/help/us/ysearch/slurp-Mozilla/5.0 (compatible; Yahoo! Slurp; url) en-us,en;q=0.5 -
 developer.yahoo.com/yql/providertext/..Mozilla/5.0 (compatible; Yahoo Pipes 2.0; url) Gecko/20090729 Firefox/3.5.2 - -
 help.yahoo.co.jp/help/jp/search/indexing/indexing-15.htmltext/..Y!J-BRW/1.0 crawler (url) ja, *;q=0.5 -
 help.yahoo.co.jp/help/jp/search/indexing/indexing-15.htmltext/..Y!J-BRW/1.0 crawler (url) - -
 help.yahoo.com/help/us/ysearch/slurpapplication/jsonMozilla/5.0 (compatible; Yahoo! Slurp/3.0; url) NOT Firefox/3.5 en-us,en;q=0.5 -
 help.yahoo.com/help/us/ysearch/slurpimage/..Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; url) en,* -
 help.yahoo.com/help/us/ysearch/slurptext/..Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; url) en,* -
 help.yahoo.com/help/us/ysearch/slurptext/..Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; url) en-us,en;q=0.5 -
naver
 help.naver.com/robots/text/..Yeti/1.0 (NHN Corp.; url) ko,en;q=0.5 -
 help.naver.com/robots/text/..Yeti/1.0 (NHN Corp.; url) ko,ja,en;q=0.5 -
 help.naver.com/robots/text/..Yeti/1.0 (NHN Corp.; url) ja,en;q=0.5 -
 help.naver.com/robots/-Yeti/1.0 (NHN Corp.; url) ko,en;q=0.5 -
 help.naver.com/robots/text/..Yeti/1.0 (NHN Corp.; url) - -
 help.naver.com/robots/text/..Yeti/1.1 (NHN Corp.; url) vi_VN,vi;q=0.8,en-US;q=0.6,en;q=0.4 -
 help.naver.com/robots/image/..Yeti/1.1 (NHN Corp.; url) vi_VN,vi;q=0.8,en-US;q=0.6,en;q=0.4 -
 help.naver.com/robots/image/..Yeti/1.0 (NHN Corp.; url) ja,en;q=0.5 -
 help.naver.com/robots/image/..Yeti/1.0 (NHN Corp.; url) ko,ja,en;q=0.5 -
orange
 wikipedia.orange.fr/text/..API/1.0 (url) - -
 wikipedia.orange.fr/application/xmlAPI/1.0 (url) - -
digplanet
 www.digplanet.com/wikiapplication/vnd.php.serializedDigplanet/1.0 (url; ) PHP/5.4 - -
cibra
 cibra.de/text/..CiBra Data Collector (url) - -
genieo
 www.genieo.com/webfilter.htmltext/..Mozilla/5.0 (compatible; Genieo/1.0 url) - -
 www.genieo.com/webfilter.htmlapplication/xmlMozilla/5.0 (compatible; Genieo/1.0 url) - -
 www.genieo.com/webfilter.htmlimage/..Mozilla/5.0 (compatible; Genieo/1.0 url) - -
 www.genieo.com/webfilter.htmlimage/..Mozilla/5.0 (compatible; Genieo/1.0 url) en,*
soso
 help.soso.com/webspider.htmtext/..Mozilla/5.0 (compatible; Sosospider/2.0; url) zh-cn,zh-hk,zh-tw,en-us -
 help.soso.com/webspider.htm-Mozilla/5.0 (compatible; Sosospider/2.0; url) zh-cn,zh-hk,zh-tw,en-us -
 help.soso.com/webspider.htmtext/..Mozilla/5.0 (compatible; Sosospider/2.0; url) - -
 help.soso.com/webspider.htmtext/..Mozilla/5.0(compatible;Sosospider/2.0;url) zh-cn,zh-hk,zh-tw,en-us -
 help.soso.com/webspider.htmapplication/jsonMozilla/5.0 (compatible; Sosospider/2.0; url) - -
 help.soso.com/webspider.htmtext/..Sosospider(url) zh-cn,zh-hk,zh-tw,en-us -
 help.soso.com/webspider.htmimage/..Mozilla/5.0 (compatible; Sosospider/2.0; url) zh-cn,zh-hk,zh-tw,en-us -
 help.soso.com/webspider.htmtext/..Mozilla/5.0(compatible; Sosospider/2.0; url) zh-cn,zh-hk,zh-tw,en-us -
ahrefs
 ahrefs.com/robot/text/..Mozilla/5.0 (compatible; AhrefsBot/4.0; url) - -
 ahrefs.com/robot/-Mozilla/5.0 (compatible; AhrefsBot/4.0; url) - -
 ahrefs.com/robot/text/..Mozilla/5.0 (compatible; AhrefsBot.FreshPages/0.1; url) - -
 ahrefs.com/robot/-Mozilla/5.0 (compatible; AhrefsBot/4.0; url) -
 ahrefs.com/application/xmlAhrefsBot.Feeds v0.1; url - -
 ahrefs.com/robot/-Mozilla/5.0 (compatible; AhrefsBot.FreshPages/0.1; url) - -
php
 pear.php.net/application/vnd.php.serializedPEAR HTTP_Request class ( url ) - -
 pear.php.net/package/http_request2text/..HTTP_Request2/0.5.2 (url) PHP/5.2.17 - -
 pear.php.net/image/..PEAR HTTP_Request class ( url ) - -
 pear.php.net/text/..PEAR HTTP_Request class ( url ) - -
 pear.php.net/package/http_request2application/xmlHTTP_Request2/2.0.0 (url) PHP/5.3.8 - -
 pear.php.net/package/http_request2text/..HTTP_Request2/2.1.1 (url) PHP/5.3.2-1ubuntu4.18 - -
 pear.php.net/package/http_request2image/..HTTP_Request2/2.1.1 (url) PHP/5.3.2-1ubuntu4.18 - -
exabot
 www.exabot.com/go/robottext/..Mozilla/5.0 (compatible; Exabot/3.0; url) - -
 www.exabot.com/go/robottext/..Mozilla/5.0 (compatible; Exabot/3.0 (BiggerBetter); url) - -
 www.exabot.com/go/robot-Mozilla/5.0 (compatible; Exabot/3.0; url) - -
 www.exabot.com/go/robot-Mozilla/5.0 (compatible; Exabot/3.0 (BiggerBetter); url) - -
sblog
 fulltext.sblog.cz/text/..SeznamBot/3.0 (url) cs -
 fulltext.sblog.cz/screenshot/image/..Mozilla/5.0 (compatible; Seznam screenshot-generator 2.0; url) cs,cz,sk;q=0.7,*;q=0.5 -
 fulltext.sblog.cz/screenshot/text/..Mozilla/5.0 (compatible; Seznam screenshot-generator 2.0; url) cs,cz,sk;q=0.7,*;q=0.5 -
 fulltext.sblog.cz/-SeznamBot/3.0 (url) cs -
 fulltext.sblog.cz/screenshot/text/..Mozilla/5.0 (compatible; Seznam screenshot-generator 2.0; url) - -
 fulltext.sblog.cz/screenshot/-Mozilla/5.0 (compatible; Seznam screenshot-generator 2.0; url) cs,cz,sk;q=0.7,*;q=0.5 -
wordpress
 jaggedykaye.wordpress.comtext/..WordPress/3.6-beta2-24180; url - -
 jaggedykaye.wordpress.comtext/..WordPress/3.6-beta2-24176; url - -
 kingcrimsonprog.wordpress.comimage/..WordPress/3.6-beta3-24306; url - -
 wildanrenaldi.wordpress.comtext/..WordPress/3.6-beta2-24163; url - -
 jaggedykaye.wordpress.comtext/..WordPress/3.6-beta2; url - -
 thesandymonocle.wordpress.comtext/..WordPress/3.6-beta3; url - -
 wildanrenaldi.wordpress.comtext/..WordPress/3.6-beta2; url - -
 raymondpronk.wordpress.comtext/..WordPress/3.6-beta3; url - -
 josefboberg.wordpress.comtext/..WordPress/3.6-beta3; url - -
 jaggedykaye.wordpress.comtext/..WordPress/3.6-beta3; url - -
 jaggedykaye.wordpress.comtext/..WordPress/3.6-beta2-24163; url - -
www.
 www.text/..GoogleBot/2.1 ( urlGoogleBot.com/bot.html) - -
 www.text/..GoogleBot-Image/1.0 ( urlGoogleBot.com/bot.html) - -
 www.text/..GoogleBot/2.1 (urlGoogleBot.com/bot.html) - -
wikipedia
 en.wikipedia.org/wiki/Wikipedia:Huggletext/..Huggle/2.1.20.0 url - -
 en.wikipedia.org/wiki/User:NicoV/Wikipedia_Cleaner/Documentationtext/..WPCleaner (url) - -
 en.wikipedia.org/wiki/User:NicoV/Wikipedia_Cleaner/Documentation-WPCleaner (url) - -
 fr.wikipedia.org/wiki/Utilisateur:OrlodrimBottext/..OrlodrimBot/1.0 (url) - -
 fr.wikipedia.org/wiki/Utilisateur:Salebotapplication/jsonSalebot, see url (uses Perl MediaWiki::API) - -
 en.wikipedia.org/wiki/Wikipedia:Huggletext/..Huggle/2.1.19.0 url - -
 fr.wikipedia.org/wiki/Utilisateur:OrlodrimBot-OrlodrimBot/1.0 (url) -
jike
 shoulu.jike.com/spider.htmltext/..Mozilla/5.0 (compatible; JikeSpider; url) zh-cn;q=0.8, *;q=0.5 -
 shoulu.jike.com/spider.htmltext/..Mozilla/5.0 (compatible; JikeSpider; url) - -
 shoulu.jike.com/spider.html-Mozilla/5.0 (compatible; JikeSpider; url) zh-cn;q=0.8, *;q=0.5 -
 shoulu.jike.com/spider.htmlapplication/jsonMozilla/5.0 (compatible; JikeSpider; url) - -
majestic12
 www.majestic12.co.uk/bot.php?text/..Mozilla/5.0 (compatible; MJ12bot/v1.4.3; url) - -
 www.majestic12.co.uk/bot.php?text/..Mozilla/5.0 (compatible; MJ12bot/v1.4.3; url) en -
 www.majestic12.co.uk/bot.php?-Mozilla/5.0 (compatible; MJ12bot/v1.4.3; url) -
sogou
 www.sogou.com/docs/help/webmasters.htm#07text/..Sogou web spider/4.0(url) zh-cn -
 www.sogou.com/docs/help/webmasters.htm#07-Sogou web spider/4.0(url) zh-cn -
 www.sogou.com/docs/help/webmasters.htm#07text/..Sogou web spider/4.0(url) - -
 www.sogou.com/docs/help/webmasters.htm#07text/..Sogou Pic Spider/3.0(url) zh-cn -
 www.sogou.com/docs/help/webmasters.htm#07application/jsonSogou web spider/4.0(url) - -
 www.sogou.com/docs/help/webmasters.htm#07-Sogou Pic Spider/3.0(url) zh-cn -
coccoc
 help.coccoc.com/text/..Mozilla/5.0 (compatible; coccoc/1.0; url) en-us;q=0.7,en;q=0.3 -
 help.coccoc.com/text/..coccoc/1.0 (url) en-us;q=0.7,en;q=0.3 -
 help.coccoc.com/-Mozilla/5.0 (compatible; coccoc/1.0; url) en-us;q=0.7,en;q=0.3 -
 help.coccoc.com/image/..Mozilla/5.0 (compatible; coccoc/1.0; url) en-us;q=0.7,en;q=0.3 -
 help.coccoc.com/image/..coccoc/1.0 (url) en-us;q=0.7,en;q=0.3 -
 help.coccoc.com/text/..Mozilla/5.0 (compatible; coccoc/1.0; url) - -
blekko
 blekko.com/about/blekkobottext/..Mozilla/5.0 (compatible; Blekkobot; ScoutJet; url) - -
 blekko.com/about/blekkobot-Mozilla/5.0 (compatible; Blekkobot; ScoutJet; url) - -
zum
 help.zum.com/inquirytext/..ZumBot/1.0 (ZUM Search; url) - -
 help.zum.com/inquiryimage/..ZumBot/1.0 (ZUM Search; url) - -
echonest
 the.echonest.com/reader/application/xmlnestReader/0.3 (discovery; url; reader at echonest.com) en -
 the.echonest.com/reader/text/..nestReader/0.3 (discovery; url; reader at echonest.com) en -
toolserver
 wiki.toolserver.org/view/GeoHacktext/..Geohack (url) - -
 wiki.toolserver.org/view/GeoHack-Geohack (url) -
 toolserver.org/~dispenser/-CacheThumbs/1.2 (url) -
 toolserver.org/~dispenser/text/..CacheThumbs/1.2 (url) - -
 toolserver.org/~dispenser/image/..CacheThumbs/1.2 (url) - -
 toolserver.org/~dispenser/text/..DispensersTools (url) - -
 toolserver.org/~para/cgi-bin/kmlexporttext/..url libwww-perl/6.02 - -
 toolserver.org/~dispenser/-DispensersTools (url) -
 toolserver.org/~dispenser/application/x-www-form-urlencodedDispensersTools (url) -
bin-co
 www.bin-co.com/php/scripts/load/text/..BinGet/1.00.A (url) - -
 www.bin-co.com/php/scripts/load/application/vnd.php.serializedBinGet/1.00.A (url) - -
mail
 go.mail.ru/help/robotstext/..Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/2.0; url) ru,ua;q=0.7,by;q=0.7,*;q=0.1 -
 go.mail.ru/help/robotstext/..Mozilla/5.0 (compatible; Mail.RU_Bot/2.0; url) ru,ua;q=0.7,by;q=0.7,*;q=0.1 -
 go.mail.ru/help/robotstext/..Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/2.0; url) - -
 go.mail.ru/help/robotstext/..Mozilla/5.0 (compatible; Mail.RU_Bot/2.0; url) - -
 go.mail.ru/help/robotsimage/..Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/2.0; url) ru,ua;q=0.7,by;q=0.7,*;q=0.1 -
scrapy
 scrapy.orgtext/..Scrapy/0.16.4 (url) en -
 scrapy.orgapplication/xmlScrapy/0.16.4 (url) en -
archive
 www.archive.org/details/archive.org_bottext/..Mozilla/5.0 (compatible; archive.org_bot url) - -
 www.archive.org/details/archive.org_botimage/..Mozilla/5.0 (compatible; special_archiver/3.1.1 url) - -
 www.archive.org/details/archive.org_bottext/..Mozilla/5.0 (compatible; special_archiver/3.1.1 url) - -
 archive.org/details/archive.org_botimage/..Mozilla/5.0 (compatible; heritrix/3.1.2-SNAPSHOT-20121013.132750 url) - -
 www.archive.org/details/archive.org_bot-Mozilla/5.0 (compatible; archive.org_bot url) - -
gnip
 www.gnip.com/text/..UnwindFetchor/1.0 (url) - -
 www.gnip.com/image/..10.196.6.45 UnwindFetchor/1.0 (url) - -
yacy
 yacy.net/bot.htmltext/..yacybot (freeworld-global; amd64 Linux 3.2.0-39-generic; java 1.6.0_27; Europe/en) url en-us,en;q=0.5 -
 yacy.net/bot.htmltext/..yacybot (freeworld/global; amd64 Linux 3.2.0-36-generic; java 1.6.0_27; Europe/ru) url en-us,en;q=0.5 -
 yacy.net/bot.htmltext/..yacybot (freeworld/global; amd64 Linux 3.2.35-minimal; java 1.7.0_10; Europe/en) url en-us,en;q=0.5 -
 yacy.net/bot.htmltext/..yacybot (freeworld/global; amd64 Linux 2.6.26-2-amd64; java 1.6.0_18; Europe/en) url en-us,en;q=0.5 -
 yacy.net/bot.htmltext/..yacybot (freeworld/global; i386 Linux 3.2.0-4-686-pae; java 1.6.0_27; Europe/pl) url en-us,en;q=0.5 -
 yacy.net/bot.htmltext/..yacybot (freeworld/global; amd64 Windows 8 6.2; java 1.6.0_41; Europe/de) url en-us,en;q=0.5 -
 yacy.net/bot.htmltext/..yacybot (freeworld/global; amd64 Linux 2.6.32-custom; java 1.6.0_26; Europe/en) url en-us,en;q=0.5 -
 yacy.net/bot.htmltext/..yacybot (freeworld/global; x86_64 Mac OS X 10.8.3; java 1.6.0_45; Europe/en) url en-us,en;q=0.5 -
 yacy.net/bot.htmltext/..yacybot (allip-any; i386 Linux 2.6.32-042stab078.6; java 1.6.0_27; Europe/en) url en-us,en;q=0.5 -
 yacy.net/bot.htmltext/..yacybot (freeworld/global; i386 Linux 3.2.13-grsec-xxxx-grs-ipv6-32; java 1.7.0_21; Europe/pl) url en-us,en;q=0.5 -
 yacy.net/bot.htmltext/..yacybot (freeworld/global; amd64 Linux 3.2.14-std-ipv6-64; java 1.6.0_18; Europe/fr) url en-us,en;q=0.5 -
 yacy.net/bot.htmltext/..yacybot (freeworld/global; amd64 Linux 2.6.32-47-server; java 1.6.0_26; Europe/de) url en-us,en;q=0.5 -
 yacy.net/bot.htmltext/..yacybot (freeworld-global; amd64 Linux 3.2.13-grsec-xxxx-grs-ipv6-64; java 1.6.0_27; Europe/pl) url en-us,en;q=0.5 -
 yacy.net/bot.htmltext/..yacybot (webportal-global; amd64 Linux 2.6.23.17-dbserv; java 1.6.0_04; Europe/en) url en-us,en;q=0.5 -
 yacy.net/bot.htmltext/..yacybot (freeworld-global; amd64 Linux 2.6.32-46-server; java 1.6.0_26; Europe/de) url en-us,en;q=0.5 -
 yacy.net/bot.htmltext/..yacybot (freeworld-global; amd64 Linux 2.6.32-47-server; java 1.6.0_26; Europe/de) url en-us,en;q=0.5 -
 yacy.net/bot.htmltext/..yacybot (freeworld/global; amd64 Linux 2.6.32-20-pve; java 1.7.0_21; Europe/en) url en-us,en;q=0.5 -
 yacy.net/bot.html-yacybot (freeworld/global; i386 Linux 3.2.0-4-686-pae; java 1.6.0_27; Europe/pl) url en-us,en;q=0.5 -
 yacy.net/bot.htmltext/..yacybot (freeworld/global; amd64 Linux 3.2.0-36-generic; java 1.6.0_27; Europe/en) url en-us,en;q=0.5 -
 yacy.net/bot.htmltext/..yacybot (freeworld/global; amd64 Linux 2.6.32-5-amd64; java 1.6.0_26; Europe/en) url * -
 yacy.net/bot.htmltext/..yacybot (freeworld/global; amd64 Linux 3.2.0-39-generic; java 1.6.0_27; Europe/lv) url en-us,en;q=0.5 -
 yacy.net/bot.htmltext/..yacybot (freeworld/global; amd64 Linux 3.2.0-4-amd64; java 1.6.0_27; Australia/en) url en-us,en;q=0.5 -
 yacy.net/bot.htmltext/..yacybot (freeworld-global; i386 Linux 3.2.13-grsec-xxxx-grs-ipv6-32; java 1.7.0_21; Europe/pl) url en-us,en;q=0.5 -
73
 178.63.73.19-Article Level Metrics - url -
 178.63.73.19application/jsonArticle Level Metrics - url - -
flipboard
 flipboard.com/browserproxyimage/..Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 (FlipboardProxy/0.0.5; url) en-us,en;q=0.5 -
 flipboard.com/browserproxytext/..Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 (FlipboardProxy/1.1; url) en-us,en;q=0.5 -
 flipboard.com/browserproxytext/..Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 (FlipboardProxy/0.0.5; url) en-us,en;q=0.5 -
 flipboard.com/browserproxyimage/..null (FlipboardProxy/1.1; url) - -
 flipboard.com/browserproxy-Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 (FlipboardProxy/0.0.5; url) en-us,en;q=0.5 -
 flipboard.com/browserproxy-Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 (FlipboardProxy/1.1; url) en-us,en;q=0.5 -
zookabot
 zookabot.comtext/..Zookabot/2.5;url - -
 zookabot.comimage/..Zookabot/2.5;url - -
in
 www.m-culture.in.thtext/..m-culture.in.th (url) - -
traslated
 mymemory.traslated.net/doc/text/..Mozilla/5.0 (MyMemory Bot url) - -
searchmetrics
 www.searchmetrics.com/en/searchmetrics-bot/text/..Mozilla/5.0 (compatible; SearchmetricsBot; url) - -
pagefreezer
 pagefreezer.com/pagefreezer-crawler/image/..PageFreezer (pagefreezer crawler; url; mail address ) en-us,en;q=0.5 -
 pagefreezer.com/pagefreezer-crawler/text/..PageFreezer (pagefreezer crawler; url; mail address ) en-us,en;q=0.5 -
daum
 tab.search.daum.net/aboutWebSearch.htmltext/..Mozilla/5.0 (compatible; MSIE or Firefox mutant; not on Windows server; url) Daumoa/3.0 ko-kr,ko;q=0.8,en-us;q=0.5,en;q=0.3 -
 tab.search.daum.net/aboutWebSearch.htmltext/..Mozilla/5.0 (compatible; MSIE or Firefox mutant; not on Windows server; url) Daumoa/3.0 - -
goo
 help.goo.ne.jp/contact/text/..goo wikipedia (url) - -
 help.goo.ne.jp/door/crawler.htmltext/..ichiro/3.0 (url) - -
 search.goo.ne.jp/option/use/sub4/sub4-1/text/..DoCoMo/2.0 P900i(c100;TB;W24H11) (compatible; ichiro/mobile goo;url) - -
 search.goo.ne.jp/option/use/sub4/sub4-1/-DoCoMo/2.0 P900i(c100;TB;W24H11) (compatible; ichiro/mobile goo;url) - -
 goo.gl/7y4SXtext/..GoogleProducer; (url) - -
 search.goo.ne.jp/option/use/sub4/sub4-1/-DoCoMo/2.0 P900i(c100;TB;W24H11) (compatible; ichiro/mobile goo; url) - -
 goo.gl/7y4SXimage/..GoogleProducer; (url) - -
wikidict
 www.wikidict.detext/..url - -
zipcode
 zipcode.ustext/..Mozilla/5.0 (compatible; YourCoolBot/1.0; url) - -
kosmix
 www.kosmix.com/html/kosmos.htmlapplication/xmlMozilla/5.0(compatible;Kosmos/1.0;url) - -
speaktoit
 www.speaktoit.comapplication/jsonSpeaktoit url - -
mediawiki
 www.mediawiki.org/text/..MediaWiki OAI Harvester 0.2 (url) - -
 www.mediawiki.org/-MediaWiki OAI Harvester 0.2 (url) -
80legs
 www.80legs.com/webcrawler.htmltext/..Mozilla/5.0 (compatible; 008/0.85; url) Gecko/2008032620 - -
parsijoo
 www.parsijoo.ir/text/..Mozilla/5.0 (compatible; parsijoo; url; mail address ) - -
 www.parsijoo.ir/image/..Mozilla/5.0 (compatible; parsijoo; url; mail address ) - -
plos
 alm.plos.orgapplication/jsonPLOS Article Level Metrics - url - -
 alm.plos.org-PLOS Article Level Metrics - url -
stackoverflow
 stackoverflow.com/questions/8956331/how-to-get-results-from-the-wikipedia-api-with-phptext/..Testing for url - -
nodejs
 nodejs.orgtext/..NodeJsWikipediaReader/0.1 (url; mail address ) BasedOnNodeJs/0.8.22 - -
okian
 www.okian.ro/text/..MyBot/1.0 (url) - -
discoveryengine
 discoveryengine.com/discoverybot.htmltext/..Mozilla/5.0 (compatible; discoverybot/2.0; url) - -
 discoveryengine.com/discoverybot.html-Mozilla/5.0 (compatible; discoverybot/2.0; url) - -
topsy
 labs.topsy.com/butterfly/text/..Mozilla/5.0 (compatible; Butterfly/1.0; url) Gecko/2009032608 Firefox/3.0.8 - -
 labs.topsy.com/butterfly/-Mozilla/5.0 (compatible; Butterfly/1.0; url) Gecko/2009032608 Firefox/3.0.8 - -
leiki
 www.leiki.comtext/..Leikibot/1.0 (url) - -
paper
 support.paper.li/entries/20023257-what-is-paper-litext/..Mozilla/5.0 (compatible; PaperLiBot/2.1; url) - -
zeebox
 www.zeebox.comtext/..Zeebox (url) en-us,en;q=0.5 -
 www.zeebox.comapplication/jsonZeebox (url) en-us,en;q=0.5 -
FeedBurner
 www.FeedBurner.comtext/..FeedBurner/1.0 (url) - -
emining
 emining.jp/text/..emBot-GalaBuzz/Nutch-1.0 (url; mail address ) en-us,en-gb,en;q=0.7,*;q=0.3 -
wikimpress
 wikimpress.org/text/..Mozilla/5.0 (compatible; Linux i686 (x86_64); de-DE; url>Wikimpress) Wikimpress/1.0 - -
 wikimpress.org/-Mozilla/5.0 (compatible; Linux i686 (x86_64); de-DE; url>Wikimpress) Wikimpress/1.0 - -
bsurprised
 bsurprised.com/text/..BSurprised WikiBox 0.1.3 (url) en -
 bsurprised.com/text/..BSurprised WikiBox 0.1 (url) en -
 bsurprised.com/text/..BSurprised WikiBox 0.1.3 (url) hu -
wwwgogetpapers
 wwwgogetpapers.com/application/jsonUser-Agent: GoGetPapersBot (url) - -
kalooga
 kalooga.com/crawlerimage/..Mozilla/5.0 (compatible; KaloogaBot; url) - -
 kalooga.com/crawlertext/..Mozilla/5.0 (compatible; KaloogaBot; url) - -
federatedmedia
 federatedmedia.nettext/..Mozilla/5.0 (url) Gecko/20061208 Firefox/2.0.0.1 en-us,en;q=0.5 -
weblio
 www.weblio.jp/info/crawler.jspimage/..Mozilla/5.0 (compatible; Webliobot/0.1; url) - -
 www.weblio.jp/text/..Mozilla/5.0 (compatible; WeblioBot; url) - -
 www.weblio.jp/info/crawler.jsptext/..Mozilla/5.0 (compatible; Webliobot/0.1; url) - -
easou
 www.easou.com/search/spider.htmltext/..Mozilla/5.0 (compatible; EasouSpider; url) zh;q=0.9,en;q=0.8 -
 www.easou.com/search/spider.htmlapplication/jsonMozilla/5.0 (compatible; EasouSpider; url) - -
drupal
 drupal.org/text/..Drupal (url) - -
 drupal.org/text/..User-Agent: Drupal (url) - -
easybib
 content.easybib.com/autocite/text/..EasyBib AutoCite (url) - -
 content.easybib.com/autocite/application/jsonEasyBib AutoCite (url) - -
example
 example.com/MyCoolTool/text/..User-Agent: MyCoolTool/1.1 (url; mail address ) - -
 example.com/MyCoolToolPage/application/jsonMyCoolTool (url) - -
 example.com/MyCoolTool/application/jsonMyCoolTool/1.1 (url; mail address ) BasedOnSuperLib/1.4 - -
tweetmeme
 tweetmeme.com/text/..Mozilla/5.0 (compatible; TweetmemeBot/3.0; url) en-gb,en;q=0.5 -
elcidharth
 elcidharth.comtext/..WordPress/3.6-beta2-24180; url - -
 elcidharth.comtext/..WordPress/3.6-beta3; url - -
 elcidharth.comtext/..WordPress/3.6-beta2-24176; url - -
 elcidharth.comtext/..WordPress/3.6-beta2; url - -
 elcidharth.comtext/..WordPress/3.6-beta2-24163; url - -
sanskritdictionary
 www.sanskritdictionary.com/application/vnd.php.serializedUser-Agent: SanskritDictionary/0.1 (url) - -
sourceforge
 j-spider.sourceforge.nettext/..JSpider (url) - -
 linkchecker.sourceforge.net/text/..LinkChecker/7.4 (url) - -
yoursite
 yoursite.com/botinfotext/..Mozilla/5.0 (compatible; YourCoolBot/1.0; url) - -
kyndane
 www.kyndane.com/bot.htmltext/..mozilla/5.0 (compatible; kynbot/1.0; url; IN TESTING) - -
wikimedia
 commons.wikimedia.org/wiki/User:Thumbnails_Check_Botimage/..Thumbnails_Check_Bot/0.1 (url; beta) - -
 tools.wikimedia.de/~daniel/text/..WikiSense (url) - -
 tools.wikimedia.de/~daniel/-WikiSense (url) -
 meta.wikimedia.org/wiki/User:Tietewtext/..Cheebot/0.5.7 (url) - -
hatena
 a.hatena.ne.jp/helptext/..Hatena Antenna/0.5 (url) - -
trendiction
 www.trendiction.de/botimage/..Mozilla/5.0 (Windows; Windows NT 6.0; en-GB; rv:1.0; trendictionbot0.5.0; trendiction search; url; please let us know of any problems; web at trendiction.com) Gecko/20071127 Firefox/3.0.0.11 en-gb,en;q=0.5 -
 www.trendiction.de/bottext/..Mozilla/5.0 (Windows; Windows NT 6.0; en-GB; rv:1.0; trendictionbot0.5.0; trendiction search; url; please let us know of any problems; web at trendiction.com) Gecko/20071127 Firefox/3.0.0.11 en-gb,en;q=0.5 -
embed
 support.embed.ly/image/..Mozilla/5.0 (compatible; Embedly/0.2; snap; url) - -
 support.embed.ly/text/..Mozilla/5.0 (compatible; Embedly/0.2; url) - -
ac
 www.ninjal.ac.jp/corpus_center/ulc/crawl-entext/..Mozilla/5.0 (compatible; heritrix/3.1.1 url) - -
 www.yama.info.waseda.ac.jp/~crawler/info.htmltext/..Mozzila/5.0 (compatible; YamanaLab-Robot/1.0; url) - -
 www.roselab.sci.waseda.ac.jptext/..2QC (url; mail address ) - -
msai
 www.msai.in/uaprof/micromax/X455.xmlimage/..url en,hi -
 www.msai.in/uaprof/micromax/X455.xmltext/..url en,hi -
 www.msai.in/uaprof/micromax/X1i_Extra.xmlimage/..url en,hi -
 www.msai.in/uaprof/micromax/X101.xmlimage/..url en,hi -
 www.msai.in/uaprof/micromax/X101.xmltext/..url en,hi -
 www.msai.in/uaprof/micromax/X1i_Extra.xmltext/..url en,hi -
localhost
 localhost/wordpresstext/..WordPress/3.5.1; url - -
 localhost/1text/..WordPress/3.5.1; url - -
 localhost/2text/..WordPress/3.5.1; url - -
alexa
 www.alexa.com/site/help/webmasterstext/..ia_archiver (url; mail address ) - -
xbmc
bnf
 www.bnf.fr/fr/outils/a.dl_web_capture_robot.htmlimage/..Mozilla/5.0 (compatible; bnf.fr_bot; url) fr, en; q=0.3 -
 www.bnf.fr/fr/outils/a.dl_web_capture_robot.htmltext/..Mozilla/5.0 (compatible; bnf.fr_bot; url) fr, en; q=0.3 -
 www.bnf.fr/fr/outils/a.dl_web_capture_robot.html-Mozilla/5.0 (compatible; bnf.fr_bot; url) fr, en; q=0.3 -
rockpeaks
 www.rockpeaks.com/contacttext/..RockPeaks/0.1 (url) - -
sistrix
 crawler.sistrix.net/text/..Mozilla/5.0 (compatible; SISTRIX Crawler; url) - -
proximic
 www.proximic.com/info/spider.phptext/..Mozilla/5.0 (compatible; proximic; url) - -
spinn3r
 spinn3r.com/robottext/..Mozilla/5.0 (X11; Linux x86_64; en-US; rv:1.9.0.19; aggregator:Spinn3r (Spinn3r 3.1); url) Gecko/2010040121 Firefox/3.0.19 - -
plagiarismcheck
 plagiarismcheck.orgapplication/jsonWikiCrawl 1.0b (url contact-mail: mail address ) - -
vk
 vk.com/dev/Sharetext/..Mozilla/5.0 (compatible; vkShare; url) - -
 vk.com/dev/Share-Mozilla/5.0 (compatible; vkShare; url) -
 vk.com/dev/Shareimage/..Mozilla/5.0 (compatible; vkShare; url) - -
seokicks
 www.seokicks.de/robot.htmltext/..Mozilla/5.0 (compatible; SEOkicks-Robot url) - -
picsearch
 www.picsearch.com/bot.htmltext/..psbot/0.1 (url) - -
 www.picsearch.com/bot.htmlimage/..psbot/0.1 (url) - -
github
 github.com/pauldix/typhoeus/tree/mastertext/..Typhoeus - url - -
 wummel.github.com/linkchecker/text/..Mozilla/5.0 (compatible; LinkChecker/8.4; url) - -
 github.com/pauldix/feedzirra/tree/masterapplication/xmlfeedzirra url - -
textdigger
 textdigger.comtext/..Mozilla/5.0 (url) Gecko/20061208 Firefox/2.0.0.1 en-us,en;q=0.5 -
geolocation
 www.geolocation.wsapplication/jsonGeolocation.ws Wikimedia Commons import. url - -
igrec
 www.igrec.ca/projectstext/..Wikitionary Text Parser 0.2 (url) - -
chickyrun
 chickyrun.tk/text/..ChickyBot/1.1 (url; mail address ) - -
muso
 www.muso.comtext/..Mozilla/5.0 (compatible; musobot/1.0; mail address ; url) - -
netarkivet
 netarkivet.dk/webcrawler/text/..Mozilla/5.0 (compatible; heritrix/1.14.4 url) - -
 netarkivet.dk/webcrawler/image/..Mozilla/5.0 (compatible; heritrix/1.14.4 url) - -
simplepie
 simplepie.orgapplication/xmlSimplePie/1.2 (Feed Parser; url; Allow like Gecko) Build/20090627192103 - -
 simplepie.orgtext/..SimplePie/1.2 (Feed Parser; url; Allow like Gecko) Build/20090627192103 - -
wandex
 wandex.nettext/..Mozilla/5.0 (compatible; World Wide Web Wanderer (Wandex Bot)/1.4; url) - -
monitis
 www.monitis.comtext/..Mozilla/5.0 (compatible; monitis - premium monitoring service; url) - -
 www.monitis.comtext/..Mozilla/5.0 (compatible; Monitis - premium monitoring service; url) - -
searchtechnologies
 www.searchtechnologies.comtext/..Mozilla/5.0 (compatible; heritrix/1.14.3 url) - -
yougorhymes
 www.yougorhymes.com/site/rhyme-bottext/..RhymeBot/0.1 (url) - -
 www.yougorhymes.com/site/rhyme-bot-RhymeBot/0.1 (url) -
superfeedr
 superfeedr.comapplication/xmlSuperfeedr bot/2.0 url - Please get in touch if we are polling too hard. - -
 superfeedr.com-Superfeedr bot/2.0 url - Please get in touch if we are polling too hard. - -
 superfeedr.comtext/..Superfeedr bot/2.0 url - Please get in touch if we are polling too hard. - -
vermagerd
 www.vermagerd.be/wptext/..WordPress/3.4.2; url - -
pingdom
 www.pingdom.com/text/..Pingdom.com_bot_version_1.4_(url) - -
 www.pingdom.comtext/..Pingdom.com_bot_version_1.4_(url) - -
holasoyramon
 www.holasoyramon.com/blogtext/..WordPress/3.5.1; url - -
yunyun
 www.yunyun.com/spider.htmltext/..Mozilla/5.0 (compatible; YYSpider; url) zh-cn;q=0.8, *;q=0.5 -
 www.yunyun.com/SiteInfo.php?r=abouttext/..Mozilla/5.0 (compatible; YRSpider; url) zh-cn;q=0.8, *;q=0.5 -
weborking
 weborking.comtext/..Weborking(url) - -
 weborking.comapplication/jsonWeborking(url) - -
netvibes
 www.netvibes.comtext/..Netvibes (url) - -
backgroundswitcher
 www.backgroundswitcher.com/text/..John's Background Switcher 4.6 (url) - -
 www.backgroundswitcher.com/image/..John's Background Switcher 4.6 (url) - -
 www.backgroundswitcher.com/text/..John's Background Switcher 4.5 (url) - -
 www.backgroundswitcher.com/text/..John's Background Switcher 4.4 (url) - -
sf
 magpierss.sf.nettext/..MagpieRSS/0.7x (url) en-us,en;q=0.5 -
 liferea.sf.net/text/..Liferea/1.x.x (Linux; es_ES.UTF-8; url) en-us,en;q=0.5 -
 liferea.sf.net/text/..Liferea/0.x.x (Linux; en_US.UTF-8; url) en-us,en;q=0.5 -
uni-potsdam
 www.hpi.uni-potsdam.de/meinel/forschung/web_30/blog_intelligence.htmltext/..HPI-BI-Crawler/0.1(url)/Nutch-2.0-dev - -
stad
 stad.comtext/..Mozilla/5.0 (compatible; stadbot/1.0; url) - -
automoveisnaweb
 z3.automoveisnaweb.com.br/wordpresstext/..WordPress/3.5.1; url - -
 z1.automoveisnaweb.com.br/wordpresstext/..WordPress/3.5.1; url - -
 z.automoveisnaweb.com.br/wordpresstext/..WordPress/3.5.1; url - -
pinterest
 pinterest.com/text/..Pinterest/0.1 url - -
 pinterest.com/image/..Pinterest/0.1 url - -
readability
 readability.com/about/text/..Readability/4aa255 - url - -
 readability.com/about/text/..Readability/bdc18f - url - -
gov
 agora.lookinglocal.gov.uk/agora/text/..Agora/1.0 (url; mail address ) - -
 pandora.nla.gov.au/crawl.htmlimage/..Mozilla/5.0 (compatible; archive.org_bot/3.1.2 url) - -
 pandora.nla.gov.au/crawl.htmltext/..Mozilla/5.0 (compatible; archive.org_bot/3.1.2 url) - -
instapaper
 www.instapaper.com/text/..Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/534.50 KHTML Version/5.1 Instapaper/4.0 (url) - -
fueto
 fueto.comapplication/jsonFueto (url) - -
cobadulu
 news.cobadulu.comtext/..WordPress/3.5.1; url - -
js-kit
 js-kit.com/text/..JS-Kit URL Resolver, url - -
homeadvisorcolorado
zapbot
 www.zapbot.orgtext/..Mozilla/5.0 (compatible; ZapBot/0.2o; url) - -
 www.zapbot.comtext/..Mozilla/5.0 (compatible; ZapBot/0.2c; url) - -
 www.zapbot.nettext/..Mozilla/5.0 (compatible; ZapBot/0.2n; url) - -
sonyericsson
 www.sonyericsson.com/UAprof/R800xR301.xmlimage/..Mozilla/5.0 (Linux; Android/2.3.3; en-us; SonyEricssonR800xurl Build/3.0.1.E.1.44) AppleWebKit/533.1 KHTML Version/4.0 Mobile Safari/533.1 en-US -
tineye
 tineye.com/crawler.htmlapplication/jsonTinEye/1.1 (url) - -
seomoz
 www.seomoz.org/dp/rogerbottext/..rogerbot/1.0 (url, mail address ) - -
rockmelt
 rockmelt.comtext/..RockmeltEmbedder (url; mail address ) - -
warebay
 www.warebay.com/bot.htmltext/..Mozilla/5.0 (compatible; WBSearchBot/1.1; url) en-us,en;q=0.5 -
theworldtopbrands
 theworldtopbrands.comtext/..WordPress/3.5.1; url - -
harvard
 cyber.law.harvard.edutext/..mediawords bot (url) - -
sentymetr
 sentymetr.pl/bot.htmlapplication/jsonMozilla/5.0 (compatible; SentymetrBot 1.0; url) - -
 sentymetr.pl/bot.htmltext/..Mozilla/5.0 (compatible; SentymetrBot 1.0; url) - -
watchmouse
turnitin
 www.turnitin.com/robot/crawlerinfo.htmltext/..TurnitinBot/2.1 (url) - -
toshiba
 www.toshiba.co.jp/rdc/about/crawl_info_en.htmtext/..TosCrawler/Nutch-1.6 (url; ' mail address dot co dot jp') en-us,en-gb,en;q=0.99,*;q=0.01 -
bazqux
 crawler.bazqux.comtext/..Mozilla/5.0 (compatible; BazQuxBot/2.4; url) - -
wikiapiary
 wikiapiary.com/wiki/User:Bumble_Beeapplication/jsonBumble Bee/1.0 (WikiApiary; url) - -
 wikiapiary.com/wiki/User:Bumble_Bee-Bumble Bee/1.0 (WikiApiary; url) -
w3
 validator.w3.org/servicestext/..W3C_Validator/1.3 url - -
hvacrepairmedic
 denver.hvacrepairmedic.comtext/..WordPress/3.5.1; url - -
 aurora.hvacrepairmedic.comtext/..WordPress/3.5.1; url - -
publicknowledgeproject
 alm.publicknowledgeproject.orgapplication/jsonArticle Level Metrics - url - -
rebelmouse
 rebelmouse.comimage/..RebelMouse/0.1 Mozilla/5.0 (compatible; url) Gecko/20100101 Firefox/7.0.1 en-us,en;q=0.5 -
 rebelmouse.comtext/..RebelMouse/0.1 Mozilla/5.0 (compatible; url) Gecko/20100101 Firefox/7.0.1 en-us,en;q=0.5 -
photosecrets
 photosecrets.com/emailtext/..PhotoSecrets (url) - -
Anonymouse
 Anonymouse.org/image/..url (Unix) - -
 Anonymouse.org/text/..url (Unix) - -
thecardb
 thecardb.comtext/..User-Agent: theCarDB/1.1 (url; mail address ) - -
seoratio
 www.seoratio.de/zyklop-crawler/text/..Mozilla/5.0 (compatible; zyklop; url) - -
apercite
 www.apercite.fr/robot/index.htmlimage/..Mozilla/5.0 (compatible; Apercite; url) en-us,en;q=0.5 -
 www.apercite.fr/robot/index.html-Mozilla/5.0 (compatible; Apercite; url) en-us,en;q=0.5
avantbrowser
 www.avantbrowser.comtext/..Avant Browser (url) en-us,en;q=0.5 -
 www.avantbrowser.comtext/..Advanced Browser (url) en-us,en;q=0.5 -
142591.949999981total

Page requests for probable crawlers, recognized by keyword
Count
x 1000		Agent string
  Mime type (count ≥ 3)
php wikibot classes - -
 application/vnd.php.serialized
 text/..
 -
AniBot/0.9 php/curl - -
 application/vnd.php.serialized
 -
 text/..
GoogleBot-Image/1.0 - -
 image/..
 text/..
 -
MediaWikiCrawler-Google/2.0 ( mail address ) - -
 text/..
 -
LinkParser/2.0 - -
 text/..
wikiwix-bot-3.0 - -
 text/..
 -
DotNetWikiBot/2.101 (Microsoft Windows NT 6.1.7601 Service Pack 1; ) - -
 text/..
 application/xml
 application/x-www-form-urlencoded
GoogleBot-Image/1.0 - -
 image/..
 text/..
 -
 application/xml
PythonWikipediaBot/1.0 - -
 application/json
 application/xml
 text/..
 -
Mozilla/5.0 (Windows; Windows NT 5.1; zh-CN; rv:1.8.0.11) Firefox/1.5.0.11; 360Spider zh-CN -
 text/..
 -
tigerbot - -
 application/json
 -
 text/..
bot arbre des mots ( mail address ) libcurl7.18.0 MSVC - -
 text/..
Peachy MediaWiki Bot API Version 2.0 (alpha) - -
 application/vnd.php.serialized
 text/..
 -
pywikipedia-radeh.py/r11231 Pywikipediabot/1.0 - -
 application/json
 application/xml
spider - -
 text/..
 application/vnd.php.serialized
 application/json
 -
 image/..
wd_universal_import/r11473 Pywikipediabot/2.0 - -
 application/json
redirect/r11592 Pywikipediabot/2.0 - -
 application/json
redirect/r11569 Pywikipediabot/2.0 - -
 application/json
ClueBot/1.1 - -
 application/vnd.php.serialized
CCBot/2.0 en-us,en-gb,en;q=0.7,*;q=0.3 -
 text/..
 -
Mozilla/5.0 (Windows; Windows NT 5.1; fr; rv:1.8.1) VoilaBot BETA 1.2 ( mail address ) fr; q=1.0, en; q=0.5, *; q=0.1 -
 text/..
 -
 application/ogg
AnomieBOT 1.0 (TagDater; see [[User:AnomieBOT]]) - -
 application/json
pywikipedia-radeh.py/r11231 Pywikipediabot/1.0 -
 -
 application/x-www-form-urlencoded
redirect/r11546 Pywikipediabot/2.0 - -
 application/json
ClueBot/2.0 - -
 application/vnd.php.serialized
YisouSpider - -
 text/..
 -
pywikipedia-redirect.py/r11168 Pywikipediabot/1.0 - -
 application/json
 text/..
pywikipedia-interwiki.py/r11096 Pywikipediabot/1.0 - -
 application/xml
 application/json
 text/..
Mozilla 5.0 (Apibot 0.32) - -
 application/vnd.php.serialized
VIAFbotWD/r11485 Pywikipediabot/2.0 - -
 application/json
PythonWikipediaBot/1.0 -
 -
 application/x-www-form-urlencoded
Mozilla/5.0 (compatible; SearchBot) - -
 text/..
 -
DotNetWikiBot/2.101 (Microsoft Windows NT 6.2.9200.0; ) - -
 text/..
 application/xml
startcmd/r11569 Pywikipediabot/2.0 - -
 application/json
new_interwiki/r11473 Pywikipediabot/2.0 - -
 application/json
Mozilla/5.0 (compatible; Ezooms/1.0; mail address ) - -
 text/..
 application/json
 application/xml
 image/..
 -
 audio/webm
 application/ogg
 application/vnd.php.serialized
Mozilla/5.0 MaboMwFramework/1.2 (w:de:MerlIwBot) - -
 text/..
UAA-UAA.py/r-1 (unknown) Pywikipediabot/1.0 - -
 application/json
pywikipedia3-MakeScoreTable_test.py/r11448 Pywikipediabot/1.0 - -
 application/json
WikiCatResearchBot ( mail address ) - -
 text/..
new2-zzdi1.py/r11452 Pywikipediabot/1.0 - -
 application/json
 application/xml
 -
Wikipediabot-mywikidatabot2.py/r-1 (unknown) Pywikipediabot/1.0 - -
 application/json
 application/xml
redirect/r11587 Pywikipediabot/2.0 - -
 application/json
redirect/r11577 Pywikipediabot/2.0 - -
 application/json
pywikipedia-klbot2_wd_desamb_glob.py/r11364 Pywikipediabot/1.0 - -
 application/json
 application/xml
cis455crawler - -
 text/..
 image/..
 application/ogg
redirect/r11596 Pywikipediabot/2.0 - -
 application/json
MediaWiki::Bot/3.2.6 - -
 application/json
 text/..
redirect/r11564 Pywikipediabot/2.0 - -
 application/json
DotNetWikiBot/2.101 (Unix 3.0.0.32; ) - -
 text/..
 application/xml
svnpywikipedia-interwiki.py/r11270 Pywikipediabot/1.0 - -
 application/json
 application/xml
CorenSearchBot/1.7 en libwww-perl/6.04 - -
 text/..
DotNetWikiBot/2.99 (Microsoft Windows NT 6.1.7601 Service Pack 1; ) - -
 text/..
 application/xml
pywikipedia-fr.wikt.RemiseEnForme.py/r10175 Pywikipediabot/1.0 - -
 application/json
 application/xml
 text/..
pywiki-klbot2_wd_desamb_glob2.py/r11366 Pywikipediabot/1.0 - -
 application/json
 application/xml
Tawbot (public svn release; plwiki) - -
 text/..
pywiki-klbot2_wd_desamb_glob3sett2_count.py/r11366 Pywikipediabot/1.0 - -
 application/json
 application/xml
pywikipedia-dikantenyvaovao.py/r11215 Pywikipediabot/1.0 - -
 application/json
pywikipediabot_old-klbot2_wd_desamb_glob.py/r11364 Pywikipediabot/1.0 - -
 application/json
 application/xml
Wiki.java 0.27 r137 (OctraBot 2.6) - -
 text/..
 -
 application/json
 application/x-www-form-urlencoded
fa-welcome.py/r11054 Pywikipediabot/1.0 - -
 application/json
pywikipedia-welcome.py/r11533 Pywikipediabot/1.0 - -
 application/json
pywikipediabot-klbot2_wd_desamb_glob3comms.py/r11365 (wikipedia.py) Pywikipediabot/1.0 - -
 application/json
 application/xml
AlkalineBOT/1.7 (1.7.1904.0) - -
 text/..
DigitalsmithsBot - -
 text/..
Wikirage.Com Statistics Bot - -
 text/..
pywikipediabot-klbot2_wd_desamb_glob3taxo.py/r11365 (wikipedia.py) Pywikipediabot/1.0 - -
 application/json
 application/xml
pywikipedia-wdph50.py/r11571 Pywikipediabot/1.0 - -
 application/json
pywikipedia-wikidata_connect_ref2.py/r11364 Pywikipediabot/1.0 - -
 application/json
 application/xml
pywikipedia-welcome.py/r11252 Pywikipediabot/1.0 - -
 application/json
MediaWiki::Bot/5.005006 - -
 application/json
 text/..
pywikipedia-wdph38.py/r11442 Pywikipediabot/1.0 - -
 application/json
 application/xml
unknown Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.5 (Exabot-Thumbnails) en,*
 -
Wikibot/2.0.2 CFNetwork/609.1.4 Darwin/13.0.0 en-us -
 image/..
 application/json
 text/..
 -
 application/opensearchdescription+xml
pywikipedia-featured.py/r11380 Pywikipediabot/1.0 - -
 application/json
 application/xml
Peachy MediaWiki Bot API Version 1.0 - -
 application/vnd.php.serialized
GermCrawler - -
 application/json
 application/xml
p-welcome-w-bn-core.py/r10872 Pywikipediabot/1.0 - -
 application/json
pywikipedia-featured.py/r11523 Pywikipediabot/1.0 - -
 application/json
 application/xml
maj_articles_recents/r11356 Pywikipediabot/2.0 - -
 application/json
pywiki-klbot2_wd_desamb_glob3sett2.py/r11366 Pywikipediabot/1.0 - -
 application/json
 application/xml
moje-wymowa.py/r11483 Pywikipediabot/1.0 - -
 application/json
Mozilla/5.0 (compatible; Nigma.ru/3.0; mail address ) - -
 text/..
 -
 application/json
iw2data/r11515 Pywikipediabot/2.0 - -
 application/json
Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.5 (Exabot-Thumbnails) en,* -
 image/..
 text/..
 application/json
 -
Test Webbot - -
 text/..
new2-zzdi1.py/r11452 Pywikipediabot/1.0 -
 -
 application/x-www-form-urlencoded
fa-welcome.py/r11054 Pywikipediabot/1.0 -
 -
 application/x-www-form-urlencoded
pywiki-klbot2_wd_desamb_glob3.py/r11366 Pywikipediabot/1.0 - -
 application/json
 application/xml
python-wikitools/1.2 (User:Irclogbot) - -
 application/json
Sogou web spider/4.0 zh-cn -
 text/..
 -
pyreza-radeh.py/r11593 Pywikipediabot/1.0 - -
 application/json
 application/xml
pywikipedia-substub.py/r11028 Pywikipediabot/1.0 - -
 application/json
DotNetWikiBot/2.100 (Microsoft Windows NT 6.2.9200.0; ) - -
 text/..
 application/xml
pyreza-radeh.py/r11593 Pywikipediabot/1.0 -
 -
 application/x-www-form-urlencoded
Desktop-test.py/r11330 (wikipedia.py) Pywikipediabot/1.0 - -
 application/json
 text/..
maj_articles_recents/r11569 Pywikipediabot/2.0 -
 application/x-www-form-urlencoded
maj_articles_recents/r11569 Pywikipediabot/2.0 - -
 application/json
python-wikitools/1.2 (User:Irclogbot) -
 application/x-www-form-urlencoded
HTMLParser/2.0 - -
 text/..
 -
AnomieBOT 1.0 (OrphanReferenceFixer; see [[User:AnomieBOT]]) - -
 application/json
pywikipediabot-klbot2_wd_desamb_glob3_taxoname2.py/r11365 (wikipedia.py) Pywikipediabot/1.0 - -
 application/json
 application/xml
SineBot/1.5.19(User:SineBot) - -
 application/vnd.php.serialized
 text/..
wdfilm2/r11473 Pywikipediabot/2.0 - -
 application/json
 -
wdfilm1/r11473 Pywikipediabot/2.0 - -
 application/json
DotNetWikiBot/2.100 (Unix 5.10.0.0; ) - -
 text/..
 application/xml
pywikipedia-welcome.py/r11461 Pywikipediabot/1.0 - -
 application/json
www.monit24.pl-m24Bot/4.1- - -
 image/..
 -
 text/..
Simple Page Crawler. Contact: mail address en-us -
 text/..
pywikipedia-redirect.py/r11483 Pywikipediabot/1.0 - -
 application/json
trunk-featured.py/r11508 Pywikipediabot/1.0 - -
 application/json
 application/xml
pywikipediabot-klbot2_wd_desamb_glob3comms_de.py/r11365 (wikipedia.py) Pywikipediabot/1.0 - -
 application/json
 application/xml
Webwiki Search Engine Bot - www.webwiki.de - -
 text/..
Semantix Bot 0.1 - -
 text/..
pywikipedia-featured.py/r11480 Pywikipediabot/1.0 - -
 application/json
 application/xml
pywikipedia-wdph22.py/r11442 Pywikipediabot/1.0 - -
 application/json
 application/xml
pywikipedia-featured.py/r11549 Pywikipediabot/1.0 - -
 application/json
Mozilla/5.0 (Windows; Windows NT 5.1; zh-CN; rv:1.8.0.11) Firefox/1.5.0.11; 360Spider - -
 text/..
 application/json
 application/xml
dtSearchSpider - -
 text/..
pywikipedia-wdph49.py/r11571 Pywikipediabot/1.0 - -
 application/json
 application/xml
WikiBot/0.1 - -
 text/..
 image/..
 application/xml
WPBot 1.0 - -
 text/..
pywikipedia-featured.py/r11593 Pywikipediabot/1.0 - -
 application/json
 application/xml
startcmd/r11523 Pywikipediabot/2.0 - -
 application/json
DotNetWikiBot/2.102 (Microsoft Windows NT 6.1.7601 Service Pack 1; ) - -
 text/..
 application/x-www-form-urlencoded
 application/xml
pywikipedia-28main.build89.py/r11399 Pywikipediabot/1.0 - -
 application/json
 application/xml
moje-porzucone.py/r11483 Pywikipediabot/1.0 - -
 application/json
AnomieBOT 1.0 (FlagIconRemover; see [[User:AnomieBOT]]) - -
 application/json
pywikipedia-flickrripper.py/r11489 Pywikipediabot/1.0 - -
 application/json
pywikipediabot-klbot2_wd_desamb_glob3comms_en.py/r11365 (wikipedia.py) Pywikipediabot/1.0 - -
 application/json
 application/xml
pywikipedia-wikidata_setproperty_NL_frominfobox_gemeentelink.py/r11200 Pywikipediabot/1.0 - -
 application/json
 application/xml
bot: fr-anal - -
 application/json
moje-wymowa.py/r11483 Pywikipediabot/1.0 -
 -
pywiki-klbot2_wd_desamb_glob3sett2es.py/r11366 Pywikipediabot/1.0 - -
 application/json
 application/xml
WikidataBot framework - -
 text/..
 application/json
pywikipedia-basictemplatedisamblist.py/r11480 Pywikipediabot/1.0 - -
 application/json
DotNetWikiBot/2.101 (Microsoft Windows NT 6.1.7600.0; ) - -
 text/..
My Nutch Spider/Nutch-1.6 en-us,en-gb,en;q=0.7,*;q=0.3 -
 text/..
pywikipediabot_old-wikidata_new_global.py/r11364 Pywikipediabot/1.0 - -
 application/json
 application/xml
DotNetWikiBot/2.100 (Unix 3.2.0.43; ) - -
 text/..
pywiki-klbot2_wd_desamb_glob3_fut.py/r11366 Pywikipediabot/1.0 - -
 application/json
 application/xml
AnomieBOT 1.0 (TemplateSubster; see [[User:AnomieBOT]]) - -
 application/json
pywikipedia-zzinter.py/r11593 Pywikipediabot/1.0 - -
 application/json
 application/xml
WikiTrans.net Bot (User:WikiTransBot; Contact: mail address ) - -
 text/..
 application/json
pywikipedia-featured.py/r11534 Pywikipediabot/1.0 - -
 application/json
 application/xml
User-Agent: Archerbot for Hubot (https://github.com/github/hubot-scripts) - -
 text/..
 -
Mozilla/5.0 (compatible; UnisterBot; mail address ) de-DE;q=0.9,de;q=0.8,en;q=0.7,* -
 text/..
 image/..
 application/ogg
pywikipediabot-klbot2_wd_desamb_glob3pers.py/r11365 (wikipedia.py) Pywikipediabot/1.0 - -
 application/json
 application/xml
pywikipediabot-klbot2_wd_desamb_glob3comms_es.py/r11365 (wikipedia.py) Pywikipediabot/1.0 - -
 application/json
trunk-checkimages.py/r11571 Pywikipediabot/1.0 - -
 application/json
Epywikipedia-welcome.py/r11181 Pywikipediabot/1.0 - -
 application/json
wikidata-sitelinks.py/r11176 Pywikipediabot/1.0 - -
 application/json
pywikipedia-wikidata_disamb.py/r11200 Pywikipediabot/1.0 - -
 application/json
 application/xml
pywikipedia-featured.py/r11483 Pywikipediabot/1.0 - -
 application/json
 application/xml
PennCISCrawler - -
 text/..
 -
milog_bot/1.0 ( mail address ) - -
 text/..
pywikipedia-redirect.py/r11593 Pywikipediabot/1.0 - -
 application/json
GoogleBot-Image/1.0 -
 -
 image/..
DotNetWikiBot/2.101 (Unix 3.1.9.0; ) - -
 text/..
 application/xml
pywikipedia-redirect.py/r11593 Pywikipediabot/1.0 -
 application/x-www-form-urlencoded
 -
BotReversor-BotReversor.py/r11483 Pywikipediabot/1.0 - -
 application/json
TicketsBot/0.1 - -
 text/..
Twitterbot/1.0 - -
 text/..
 image/..
 -
 application/pdf
 application/ogg
Mozilla/5.0 (compatible; MyBot/1.0;) - -
 application/json
AnomieBOT 1.0 (PERTableUpdater; see [[User:AnomieBOT]]) - -
 application/json
 text/..
MyCuteBot/0.1 - -
 text/..
 application/json
 application/vnd.php.serialized
User:WP 1.0 bot (operated by User:Theopolisme on enwiki) - -
 text/..
DotNetWikiBot/2.100 (Unix 3.2.0.41; ) - -
 text/..
pywikipedia-featured.py/r11574 Pywikipediabot/1.0 - -
 application/json
 application/xml
pywikipedia-featured.py/r11508 Pywikipediabot/1.0 - -
 application/json
 application/xml
User-Agent: Futuramabot for Hubot (https://github.com/github/hubot-scripts) - -
 text/..
 -
Python27-pythonw.exe/r11461 Pywikipediabot/1.0 - -
 application/json
pywikipedia-basictemplatedisamblist.py/r11480 Pywikipediabot/1.0 -
 -
 application/x-www-form-urlencoded
360spider-image - -
 image/..
 text/..
trunk-featured.py/r11486 Pywikipediabot/1.0 - -
 application/json
 application/xml
pywikipedia-wdph22.py/r11571 Pywikipediabot/1.0 - -
 application/json
 application/xml
ko_missing/r11515 Pywikipediabot/2.0 - -
 application/json
pywikipedia-_move2wikidata_np_ru.py/r11508 Pywikipediabot/1.0 - -
 application/json
 application/xml
FAST Search Web Crawler 14.0.0325.0000 - -
 text/..
 -
 application/ogg
 application/pdf
pywiki-klbot2_wd_desamb_glob3sett.py/r11366 Pywikipediabot/1.0 - -
 application/json
 application/xml
wdsex/r11486 Pywikipediabot/2.0 - -
 application/json
aleph-avbot.py/r11368 Pywikipediabot/1.0 - -
 application/json
 text/..
mail address mail address – MediaWiki Tcl Bot Framework 0.5 (r1) - -
 application/json
pywikipedia-zzinter.py/r11550 Pywikipediabot/1.0 - -
 application/json
 application/xml
pywiki-klbot2_wd_desamb_glob3pais.py/r11366 Pywikipediabot/1.0 - -
 application/json
 application/xml
pywikirish-welcome.py/r11215 Pywikipediabot/1.0 - -
 application/json
iw2data/r11473 Pywikipediabot/2.0 - -
 application/json
pywikipedia-wdph50.py/r11549 Pywikipediabot/1.0 - -
 application/json
pywikipedia-wdph49.py/r11549 Pywikipediabot/1.0 - -
 application/json
pywikipediabot-klbot2_wd_desamb_glob3sett2_count.py/r11365 (wikipedia.py) Pywikipediabot/1.0 - -
 application/json
 application/xml
moje-porzucone.py/r11483 Pywikipediabot/1.0 -
 -
SiocWikiBot/1.0 - -
 application/vnd.php.serialized
 text/..
startcmd/r11505 Pywikipediabot/2.0 - -
 application/json
HosiryuhosiBot IRC-RecentChanges Checker ja -
 text/..
 application/x-www-form-urlencoded
ko_labelcheck/r11473 Pywikipediabot/2.0 - -
 application/json
startcmd/r11526 Pywikipediabot/2.0 - -
 application/json
JavaCrawler/1.1 - -
 text/..
wikidata-sitelinks.py/r11176 Pywikipediabot/1.0 -
 -
 application/x-www-form-urlencoded
pywikipedia-wdph46.py/r11442 Pywikipediabot/1.0 - -
 application/json
 application/xml
Permaculture Super Coop Nutch Spider/Nutch-1.6 en-us,en-gb,en;q=0.7,*;q=0.3 -
 text/..
 application/ogg
 -
 image/..
SurakWare MediaWiki Bot/1.0 - -
 text/..
COIBot/1.00 - -
 text/..
pywikipedia-featured.py/r11543 Pywikipediabot/1.0 - -
 application/json
 application/xml
pywikipedia-wdph49.py/r11442 Pywikipediabot/1.0 - -
 application/json
 application/xml
pywikipedia-redirect.py/r11571 Pywikipediabot/1.0 -
 application/x-www-form-urlencoded
 -
Py-new_interwiki_sk.py/r11564 Pywikipediabot/1.0 - -
 application/json
 application/xml
FTRF: Friendly robot/1.3 - -
 text/..
 -
BotReversor-BotReversor.py/r11483 Pywikipediabot/1.0 -
 -
Mozilla/5.0 (Windows; Windows NT 5.1; fr; rv:1.8.1) VoilaBot BETA 1.2 ( mail address ) - -
 text/..
 -
Noramy 0.1b (knowledge search and index bot) - -
 text/..
 -
 application/ogg
startcmd/r11521 Pywikipediabot/2.0 - -
 application/json
pywikipedia-redirect.py/r11508 Pywikipediabot/1.0 - -
 application/json
Goalkeeperbot(User:Beetstra)/1.0 - -
 text/..
startcmd/r11490 Pywikipediabot/2.0 - -
 application/json
XLinkBot/1.00 - -
 text/..
~Bot ([[:fr:w:User:TildeBot]] by [[:fr:w:User:Alphos]] mail address ) - -
 text/..
DeletionBot-deletion.py/r11525 Pywikipediabot/1.0 - -
 application/json
 application/xml
pywikipedia-redirect.py/r11571 Pywikipediabot/1.0 - -
 application/json
My Nutch Spider/Nutch-1.6 (NutchExploration) en-us,en-gb,en;q=0.7,*;q=0.3 -
 text/..
SchoolReviewNetworkWikiBot - -
 application/json
pywikipedia-wdph42.py/r11442 Pywikipediabot/1.0 - -
 application/json
 application/xml
ShadCrawler - -
 text/..
FTRF: Friendly robot/1.3 -
 -
 text/..
pywikipedia-featured.py/r11543 Pywikipediabot/1.0 -
 -
 application/x-www-form-urlencoded
mineobot_sources/r11473 Pywikipediabot/2.0 - -
 application/json
DotNetWikiBot/2.100 (Unix 3.2.0.40; ) - -
 text/..
pyreza-zzinter.py/r11514 Pywikipediabot/1.0 - -
 application/json
 -
 application/xml
DeletionBot-deletion.py/r11525 Pywikipediabot/1.0 -
 -
 application/x-www-form-urlencoded
pywikipedia-dave_sezioni.py/r11522 Pywikipediabot/1.0 - -
 application/json
 application/xml
trunk-featured.py/r11533 Pywikipediabot/1.0 - -
 application/json
 application/xml
DotNetWikiBot/2.102 (Unix 3.2.0.43; ) - -
 text/..
Python27-pythonw.exe/r11340 Pywikipediabot/1.0 - -
 application/json
AnomieBOT 1.0 (BAGBot; see [[User:AnomieBOT]]) - -
 application/json
 text/..
Mozilla/5.0 (compatible; MyBot/1.0;) -
 application/x-www-form-urlencoded
pywikipedia-wdph49.py/r11442 Pywikipediabot/1.0 -
 -
 application/x-www-form-urlencoded
pywikipediabot-klbot2_wd_desamb_glob3_taxoname.py/r11365 (wikipedia.py) Pywikipediabot/1.0 - -
 application/json
 application/xml
Py-new_interwiki.py/r11521 Pywikipediabot/1.0 - -
 application/json
 application/xml
startcmd/r11496 Pywikipediabot/2.0 - -
 application/json
trunk-featured.py/r11538 Pywikipediabot/1.0 - -
 application/json
 application/xml
pywikipedia-radeh.py/r11593 Pywikipediabot/1.0 - -
 application/json
pywikipediabot-klbot2_wd_desamb_glob3futbol.py/r11365 (wikipedia.py) Pywikipediabot/1.0 - -
 application/json
 application/xml
wdsex/r11485 Pywikipediabot/2.0 - -
 application/json
Bub's wikibot (Wikibot/2011111111; JWBF/1.2; Java/1.7) - -
 text/..
Peachy MediaWiki Bot API Version 1.0 -
 -
BeneBot*/1.0 WikibasePhpLib/0.1 - -
 application/json
pywikipedia-iw2data.py/r11508 Pywikipediabot/1.0 - -
 application/json
 application/xml
User:WP 1.0 bot (operated by User:Theopolisme on enwiki) -
 -
pywikipedia-featured.py/r11574 Pywikipediabot/1.0 -
 -
 application/x-www-form-urlencoded
aleph-avbot.py/r11368 Pywikipediabot/1.0 -
 -
 application/x-www-form-urlencoded
FAST Search Web Crawler 14.0.0325.0000 -
 -
 text/..
php wikibot classes -
 -
(u'python-wikitools/1.1.1 (User:Cerabot)',) - -
 application/json
Addbot Wikimedia Bot - -
 application/vnd.php.serialized
 text/..
trunk-featured.py/r11523 Pywikipediabot/1.0 - -
 application/json
 application/xml
pywikipedia-featured.py/r11571 Pywikipediabot/1.0 - -
 application/json
 application/xml
VWBot - -
 text/..
 application/json
Wikibot/2.1 CFNetwork/609.1.4 Darwin/13.0.0 en-us -
 image/..
 text/..
Empedia Bot - -
 text/..
trunk-checkimages.py/r11425 Pywikipediabot/1.0 - -
 application/json
pywikipedia-basicarticledisamblist.py/r11543 Pywikipediabot/1.0 - -
 application/json
pywikipediabot-klbot2_wd_desamb_glob3film2.py/r11365 (wikipedia.py) Pywikipediabot/1.0 - -
 application/json
HRoestBot, de-wikipedia using pywikipedia framework - -
 text/..
 application/json
pywikipedia-basicarticledisamblist.py/r11480 Pywikipediabot/1.0 - -
 application/json
pywikipedia-fr.wikt.import-sons-commons.py/r10175 Pywikipediabot/1.0 - -
 application/json
 application/xml
TVersity Media Robot - -
 text/..
wdfilm3/r11473 Pywikipediabot/2.0 - -
 application/json
pyreza-zz_make_cat.py/r11533 Pywikipediabot/1.0 - -
 application/json
 -
DotNetWikiBot/2.96 (Microsoft Windows NT 6.1.7601 Service Pack 1; ) - -
 text/..
 application/xml
unknown, 2a00:4b00:13d:200::129e Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.5 (Exabot-Thumbnails) en,* -
 image/..
 text/..
unknown, 2a00:4b00:13d:200::129a Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.5 (Exabot-Thumbnails) en,* -
 image/..
 text/..
Erel Bot - -
 text/..
pyreza-zz_make_cat.py/r11533 Pywikipediabot/1.0 -
 -
Pywikipedia-new_interwiki.py/r11508 Pywikipediabot/1.0 - -
 application/json
 application/xml
pywikipedia-basicarticledisamblist.py/r11543 Pywikipediabot/1.0 -
 -
unknown, 2a00:4b00:13d:200::129f Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.5 (Exabot-Thumbnails) en,* -
 image/..
HopperBot - -
 image/..
 text/..
pywikipedia-featured.py/r11589 Pywikipediabot/1.0 - -
 application/json
 application/xml
logpatrol/r11564 Pywikipediabot/2.0 - -
 application/json
Dorkobot/Nutch-1.6 en-us,en-gb,en;q=0.7,*;q=0.3 -
 text/..
 image/..
pywikipedia-redirect.py/r11514 Pywikipediabot/1.0 - -
 application/json
theWxitBot/0.1 - -
 application/json
 image/..
DotNetWikiBot/2.101 (Microsoft Windows NT 5.1.2600 Service Pack 3; ) - -
 text/..
 application/xml
hatnote_fix/r11592 Pywikipediabot/2.0 - -
 application/json
unknown, 2a00:4b00:13d:200::129c Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.5 (Exabot-Thumbnails) en,* -
 image/..
pywikirish-welcome.py/r11215 Pywikipediabot/1.0 -
 -
 application/x-www-form-urlencoded
MediaWiki::Bot/3.005002 - -
 application/json
pywikipedia-wikidata_wes.py/r11364 Pywikipediabot/1.0 - -
 application/json
 application/xml
unknown, 2a00:4b00:13d:200::129d Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.5 (Exabot-Thumbnails) en,* -
 image/..
 text/..
pywikipedia-redirect.py/r11534 Pywikipediabot/1.0 - -
 application/json
pywikipedia-redirect.py/r11534 Pywikipediabot/1.0 -
 application/x-www-form-urlencoded
 -
claimit/r11537 Pywikipediabot/2.0 - -
 application/json
ReaperBot/2.0.0 (incompatible-notwebbrowser:robot:exclusion-noncompliant) bot> - -
 text/..
pywikipedia-wdph51.py/r11597 Pywikipediabot/1.0 - -
 application/json
FAST Enterprise Crawler 6 used by Bygglo / Svensk byggtjanst - -
 text/..
 -
Just-Crawling 0.1 - -
 text/..
pywikipedia-featured.py/r11480 Pywikipediabot/1.0 -
 -
pywikipedia-commonscat_wikidata.py/r11457 Pywikipediabot/1.0 - -
 application/json
 application/xml
pywikipedia-wikidata_setproperty.py/r11200 Pywikipediabot/1.0 - -
 application/json
 application/xml
enwiki-sandheaderbot.py/r11533 Pywikipediabot/1.0 - -
 -
 application/json
 application/x-www-form-urlencoded
COIBot/2.0 - -
 text/..
DotNetWikiBot/2.100 (Unix 3.2.0.38; ) - -
 text/..
cis455crawler en-US -
 text/..
unknown, 2a00:4b00:13d:200::129b Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.5 (Exabot-Thumbnails) en,* -
 image/..
pybot-VM-auto-erl.py/r11522 Pywikipediabot/1.0 - -
 application/json
wikiparser/1 CFNetwork/596.3.3 Darwin/12.3.0 (x86_64) (MacPro5,1) en-us -
 image/..
 text/..
AnomieBOT 1.0 (DatedCategoryCreator; see [[User:AnomieBOT]]) - -
 application/json
Zing-BottaBot/2.0 - -
 text/..
harvest_template/r11511 Pywikipediabot/2.0 - -
 application/json
wdfilm4/r11473 Pywikipediabot/2.0 - -
 application/json
WMU-bot - -
 text/..
 -
pywikipedia-wdph46.py/r11442 Pywikipediabot/1.0 -
 -
 application/x-www-form-urlencoded
Pywikipedia-new_interwiki_sk.py/r11557 Pywikipediabot/1.0 - -
 application/json
trunk-featured.py/r11526 Pywikipediabot/1.0 - -
 application/json
 application/xml
pywikipedia-welcome.py/r11027 Pywikipediabot/1.0 - -
 application/json
pywikipedia-wdph47.py/r11442 Pywikipediabot/1.0 - -
 application/json
trunk-featured.py/r11494 Pywikipediabot/1.0 - -
 application/json
 application/xml
pyreza-Radeh.py/r11514 Pywikipediabot/1.0 - -
 application/json
 application/xml
pywikipedia-welcome-custom.py/r11354 Pywikipediabot/1.0 - -
 application/json
SLQbot2-SLQ_write_new_xml.py/r11368 Pywikipediabot/1.0 - -
 application/json
pyreza-zz_make_cat.py/r11593 Pywikipediabot/1.0 -
 -
pyreza-zz_make_cat.py/r11593 Pywikipediabot/1.0 - -
 application/json
~Bot ([[:fr:w:User:TildeBot]] by [[:fr:w:User:Alphos]] mail address ) -
 -
cis455Crawler en-US -
 text/..
pywikipedia-wdtashih2.py/r11442 Pywikipediabot/1.0 - -
 application/json
 application/xml
pywikipedia-redirect.py/r11507 Pywikipediabot/1.0 - -
 application/json
HTMLParser/1.6 - -
 text/..
 -
pywikipedia-redirect.py/r11533 Pywikipediabot/1.0 - -
 application/json
pywikipedia-wikidata_setproperty_FR_frominfobox.py/r11200 Pywikipediabot/1.0 - -
 application/json
 application/xml
TwynCatBot/0.2 (Contact: www.twyn.com) - -
 application/json
pyreza-zzinteren.py/r11514 Pywikipediabot/1.0 - -
 application/json
Screaming Frog SEO Spider/2.11 - -
 text/..
 image/..
 -
DotNetWikiBot, edited by D. Rodionov/2.91 (Microsoft Windows NT 6.0.6002 Service Pack 2; ) - -
 text/..
 application/xml
DotNetWikiBot/2.100 (Unix 3.0.0.12; ) - -
 text/..
 application/xml
SLQbot2-SLQ_subject_mapper.py/r11368 Pywikipediabot/1.0 - -
 application/json
pywikipedia-redirect.py/r11599 Pywikipediabot/1.0 - -
 application/json
trunk-featured.py/r11487 Pywikipediabot/1.0 - -
 application/json
 application/xml
Soundkiosk Relation-Crawler (Version 1.0; soundkiosk.de) - -
 application/xml
 text/..
pywikipedia-redirect.py/r11385 Pywikipediabot/1.0 - -
 application/json
 text/..
AnomieBOT 1.0 (RandomPagePicker; see [[User:AnomieBOT]]) - -
 application/json
taxon_name/r11473 Pywikipediabot/2.0 - -
 application/json
HersfoldIRCBot version 1.7 - -
 text/..
 -
 application/x-www-form-urlencoded
pynik-checkimages.py/r11508 Pywikipediabot/1.0 - -
 application/json
startcmd/r11564 Pywikipediabot/2.0 - -
 application/json
SkimBot/1.0 (www.skimlinks.com mail address ) en-us,en;q=0.5 -
 text/..
 -
 image/..
pywikipedia-kleida-ell.py/r11543 Pywikipediabot/1.0 - -
 application/json
 application/xml
wiki (artditel spider) en -
 text/..
pyreza-zzinter.py/r11514 Pywikipediabot/1.0 -
 -
 application/x-www-form-urlencoded
Wikipediabot-wikidata_gndcheck.py/r-1 (unknown) Pywikipediabot/1.0 - -
 application/json
python-wikitools/1.2 (User:Mr.Z-bot) - -
 application/json
pywikipedia-featured.py/r11384 Pywikipediabot/1.0 - -
 application/json
 application/xml
pywikipedia-redirect.py/r11599 Pywikipediabot/1.0 -
 application/x-www-form-urlencoded
 -
pywikipedia-17-progreso1911.py/r11593 Pywikipediabot/1.0 - -
 application/json
 application/xml
GoogleBot - -
 text/..
 -
pywikipedia-wikidata_cleanlabel.py/r11200 Pywikipediabot/1.0 - -
 application/json
 application/xml
crawler/1.0 en -
 text/..
pywikipedia-redirect.py/r11533 Pywikipediabot/1.0 -
 -
 application/x-www-form-urlencoded
maj_articles_recents/r11356 Pywikipediabot/2.0 -
 application/x-www-form-urlencoded
merge/r11473 Pywikipediabot/2.0 - -
 application/json
trunk-featured.py/r11488 Pywikipediabot/1.0 - -
 application/json
 application/xml
Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.5 (Exabot-Thumbnails) en;q=0.9,*;q=0.8 -
 text/..
 image/..
pywikipedia-featured.py/r11365 Pywikipediabot/1.0 - -
 application/json
Peachy MediaWiki Bot API Version 0.1beta - -
 application/vnd.php.serialized
EarwigBot/0.2.dev.gitca7cbceb (Python/2.7.3; https://github.com/earwig/earwigbot; mail address ) - -
 application/json
Pywikipedia-new_interwiki.py/r11485 Pywikipediabot/1.0 - -
 application/json
Wikibot/2.0.2 CFNetwork/609 Darwin/13.0.0 en-us -
 image/..
 application/json
 text/..
pywikipedia-17-progreso1911.py/r11593 Pywikipediabot/1.0 -
 -
 application/x-www-form-urlencoded
pywikipedia-_move2wikidata_np_ua.py/r11508 Pywikipediabot/1.0 - -
 application/json
AdMedia bot - -
 text/..
 -
maj_articles_recents/r11539 Pywikipediabot/2.0 -
 application/x-www-form-urlencoded
weave_rest_bot, contact mail address - -
 text/..
pynik-checkimages.py/r11483 Pywikipediabot/1.0 - -
 application/json
Py-new_interwiki_sk.py/r11593 Pywikipediabot/1.0 - -
 application/json
 application/xml
pywiki.test-_parserfunctioncount.py/r11467 Pywikipediabot/1.0 - -
 application/json
pywikilab-llista_mils_2013.py/r11483 Pywikipediabot/1.0 - -
 application/json
pywikipedia-basicarticledisamblist.py/r11480 Pywikipediabot/1.0 -
 -
CodeGator Crawler v1.0 - -
 text/..
 -
AnomieBOT 1.0 (AFDMergeFromCleaner; see [[User:AnomieBOT]]) - -
 application/json
bncf/r11473 Pywikipediabot/2.0 - -
 application/json
pywikipedia-redirect.py/r11578 Pywikipediabot/1.0 -
 application/x-www-form-urlencoded
 -
maj_articles_recents/r11539 Pywikipediabot/2.0 - -
 application/json
UCMore Crawler App en-us,en;q=0.5 -
 text/..
pywikipedia-wdph45.py/r11442 Pywikipediabot/1.0 - -
 application/json
pywikipedia-wdph23.py/r11442 Pywikipediabot/1.0 - -
 application/json
 application/xml
pywikipedia-wikidata_wnl.py/r11364 Pywikipediabot/1.0 - -
 application/json
 application/xml
py-dataMakeScoreTable.py/r11470 Pywikipediabot/1.0 - -
 application/json
AnomieBOT 1.0 (DeletionSortingCleaner; see [[User:AnomieBOT]]) - -
 application/json
HRoestBot, de-wikipedia using pywikipedia framework -
 application/x-www-form-urlencoded
 -
Mozilla/5.0 (compatible; SnapPreviewBot; en-US; rv:1.8.0.9) Gecko/20061206 Firefox/1.5.0.9 en-us,en;q=0.5 -
 text/..
trunk-checkimages.py/r11506 Pywikipediabot/1.0 - -
 application/json
pyreza-zzinteren.py/r11514 Pywikipediabot/1.0 -
 -
pywikipedia-redirect.py/r11578 Pywikipediabot/1.0 - -
 application/json
pywikt-interwiki.py/r11382 (wikipedia.py) Pywikipediabot/1.0 - -
 application/xml
 application/json
ko_list/r11473 Pywikipediabot/2.0 - -
 application/json
htemp/r11521 Pywikipediabot/2.0 - -
 application/json
python-wikitools/1.2 (User:LaraBot) - -
 application/json
claimit/r11596 Pywikipediabot/2.0 - -
 application/json
pywikilab-llista_mils_2013.py/r11483 Pywikipediabot/1.0 -
 -
pywikipedia-wdph22.py/r11549 Pywikipediabot/1.0 - -
 application/json
MediaWiki::Bot 3.1.5 - -
 application/json
artists/r11276 (pywikibot/__init__.py) Pywikipediabot/2.0 - -
 application/json
trunk-featured.py/r11532 Pywikipediabot/1.0 - -
 application/json
('python-wikitools/1.1.1 (User:BernsteinBot)',) - -
 application/json
wikiscore-MakeScoreTable.py/r11152 Pywikipediabot/1.0 - -
 application/json
artists/r11515 Pywikipediabot/2.0 - -
 application/json
Py-new_interwiki.py/r11508 Pywikipediabot/1.0 - -
 application/json
Wikibot/2.0.2 CFNetwork/609.1.4 Darwin/13.0.0 en-gb -
 image/..
 application/json
 text/..
pywikipedia-redirect.py/r11586 Pywikipediabot/1.0 -
 application/x-www-form-urlencoded
 -
trunk-checkimages.py/r11488 Pywikipediabot/1.0 - -
 application/json
pywikipedia-17-progreso1911.py/r11508 Pywikipediabot/1.0 - -
 application/json
 application/xml
CheMoBot/1.00 - -
 text/..
bitlybot - -
 text/..
 image/..
DotNetWikiBot/2.101 (Unix 3.0.0.12; ) - -
 text/..
 application/xml
Mozilla/5.0 (X11; Linux i686; en-US; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 SnapPreviewBot en-us,en;q=0.5 -
 text/..
Local Site Parser 1.0 en-us,en;q=0.5 -
 text/..
PhronesisSpider - -
 text/..
pywikipedia-28main.build89.py/r11131 Pywikipediabot/1.0 - -
 application/json
 application/xml
pywikipedia-17-progreso1911.py/r11483 Pywikipediabot/1.0 - -
 application/json
 application/xml
Phantom.js bot cs-CZ,en,* -
 image/..
 text/..
pywikipedia-redirect.py/r11561 Pywikipediabot/1.0 -
 application/x-www-form-urlencoded
 -
wikbotlite/2.0 CFNetwork/609.1.4 Darwin/13.0.0 en-us -
 image/..
 application/json
 text/..
Pywikipedia-interwiki.py/r11549 Pywikipediabot/1.0 - -
 application/json
 application/xml
pywiki-klbot2_wd_desamb_glob3film.py/r11366 Pywikipediabot/1.0 - -
 application/json
pywikipedia-redirect.py/r11530 Pywikipediabot/1.0 - -
 application/json
Pywikipedia-new_interwiki_sk.py/r11579 Pywikipediabot/1.0 - -
 application/json
pywikipedia-_move2wikidata_movie.py/r11508 Pywikipediabot/1.0 - -
 application/json
commons-substituter.py/r11436 Pywikipediabot/1.0 - -
 application/json
update-regular-users/r11276 (pywikibot/__init__.py) Pywikipediabot/2.0 - -
 application/json
pywikipedia-redirect.py/r11561 Pywikipediabot/1.0 - -
 application/json
bot-VM-auto-erl.py/r11365 Pywikipediabot/1.0 - -
 application/json
DotNetWikiBot/2.96 (Microsoft Windows NT 6.2.9200.0; ) - -
 text/..
hercule-commonscat.py/r11593 Pywikipediabot/1.0 - -
 application/json
Mozilla/5.0 (Bgbot 0.5) - -
 text/..
Py-new_interwiki.py/r11564 Pywikipediabot/1.0 - -
 application/json
pywikipedia-redirect.py/r11549 Pywikipediabot/1.0 -
 application/x-www-form-urlencoded
 -
commons-imagerecat.py/r11028 Pywikipediabot/1.0 - -
 application/json
 -
pywikt-interwiki.py/r11574 Pywikipediabot/1.0 -
 application/x-www-form-urlencoded
 -
Wikidata-wikidata.py/r11202 Pywikipediabot/1.0 - -
 application/json
Mozilla/5.0 (Windows; Windows NT 5.1; zh-CN; rv:1.8.0.11) Firefox/1.5.0.11; 360Spider zh-CN -
 text/..
 image/..
 -
Feedstripes Bot v1.0 - -
 text/..
 application/xml
Py-new_interwiki.py/r11487 Pywikipediabot/1.0 - -
 application/json
Mozilla/5.0 (SnapPreviewBot) Gecko/20061206 Firefox/1.5.0.9 en-us,en;q=0.5 -
 image/..
 text/..
startcmd/r11276 (pywikibot/__init__.py) Pywikipediabot/2.0 - -
 application/json
EarwigBot/0.2.dev.gitca7cbceb (Python/2.7.3; https://github.com/earwig/earwigbot; mail address ) -
 application/x-www-form-urlencoded
HersfoldIRCBot version 1.6.2 - -
 text/..
 -
trunk-checkimages.py/r11508 Pywikipediabot/1.0 - -
 application/json
pywikipedia-redirect.py/r11385 Pywikipediabot/1.0 -
 -
 application/x-www-form-urlencoded
pwb-templatecount.py/r11546 Pywikipediabot/1.0 - -
 application/json
WorldOfMusicBetaBot/1.0 - -
 text/..
33776.89total

IP ranges: known ip ranges for Google are 64.233.[160.0-191.255], 66.249.[64.0-95.255], 66.102.[0.0-15.255], 72.14.[192.0-255.255],
74.125.[0.0-255.255], 209.085.[128.0-255.255], 216.239.[32.0-63.255] and a few minor other subranges

Errata: WMF traffic logging service suffered from server capacity problems in Aug/Sep/Oct 2011.
Absolute traffic counts for October 2011 are approximatly 7% too low.
Data loss only occurred during peak hours. It therefore may have had somewhat different impact for traffic from different parts of the world.
and may have also skewed relative figures like share of traffic per browser or operating system.

From mid September till late November squid log records for mobile traffic were in invalid format.
Data could be repaired for logs from mid October onwards. Older logs were no longer available.

In a an unrelated server outage precisely half of traffic to WMF mobile sites was not counted from Oct 16 - Nov 29 (one of two load-balanced servers did not report traffic).
WMF has since improved server monitoring, so that similar outages should be detected and fixed much faster from now on.

Generated on Sun, Jul 7, 2013 13:33
Author:Erik Zachte (Web site)
Mail: ezachte@### (no spam: ### = wikimedia.org)
All data and images on this page are in the public domain.

Note: page may load slower on Microsoft Internet explorer than on other major browsers