This report shows all requests to Wikimedia servers where a Google server of service was involved in any way, be it the GoogleBot crawler or FeedFetcher collector scripts that run on Google servers, or a user that follows a link from a Google Web or Google Desktop search results page, or from Google Maps or Google Earth etcetera. Technically speaking three fields in the squid log records are checked for this: client ip address, referer header and user agent header. A request can originate from an ip address which has been registered by Google and/or it can carry a referer tag that tells us a user clicked a link on a Google results page and/or it can carry an agent string that mentions a Google application which can reasonably be assumed to be genuinely Google's. See bottom of page for further details.
In total Google was somehow involved in 48.1% of daily external page* requests
Google referred to our sites, through its services including search, maps, and Google Earth, 93,880,000 page views per day, representing 40.0% of our external page requests.
Including all of its different search crawlers and services hosted on its servers, Google itself requested another 19,198,000 page pages per day, representing 8.2% of our external page requests.
* = mime type text/html only
|
In order of request volume |
Requests originating from a Google ip address |
Service | Total | Pages | Images | Other |
GoogleBot | 19,527 | 16,833 | 520 | 2,175 |
Wireless | 2,825 | 1,726 | 1,085 | 14 |
Translate | 387 | 387 | 1 | 1 |
FeedFetcher | 350 | 126 | 1 | 223 |
Other | 191 | 106 | 33 | 53 |
Web search | 27 | 19 | 7 | 1 |
Desktop | 2 | 1 | | 2 |
KeywordTool | 1 | 1 | | |
Image search | 1 | 1 | 1 | |
Toolbar | 1 | 1 | | |
Maps | 1 | 1 | 1 | |
Earth | 1 | | 1 | |
Mail | 1 | | 1 | |
Total | 23,310 | 19,198 | 1,646 | 2,468 |
|
Requests originating from elsewhere |
Service | Total | Pages | Images | Other |
Web search | 91,009 | 89,407 | 1,124 | 478 |
Other | 10,221 | 222 | 4,526 | 5,472 |
Image search | 3,118 | 1,797 | 1,274 | 48 |
Desktop | 2,242 | 58 | 1 | 2,183 |
GoogleBot? | 1,992 | 1,922 | 5 | 64 |
Toolbar | 973 | 226 | 77 | 670 |
Earth | 819 | 158 | 654 | 8 |
Maps | 191 | 53 | 135 | 3 |
Mail | 63 | 12 | 44 | 6 |
Translate | 29 | 15 | 8 | 6 |
Wireless | 11 | 10 | 1 | |
FeedFetcher | 1 | 1 | | 1 |
Total | 110,668 | 93,880 | 7,847 | 8,938 |
|
Top level domains |
.com | 31,030 | 29,686 | 1,106 | 238 |
.de | 7,369 | 7,168 | 178 | 23 |
.jp | 4,836 | 4,764 | 55 | 16 |
.uk | 4,787 | 4,637 | 134 | 15 |
.br | 3,681 | 3,601 | 64 | 17 |
.fr | 3,646 | 3,527 | 100 | 19 |
.mx | 3,168 | 3,101 | 53 | 14 |
.it | 3,023 | 2,950 | 63 | 10 |
.ca | 2,766 | 2,674 | 82 | 10 |
.pl | 2,540 | 2,480 | 50 | 10 |
.es | 2,285 | 2,227 | 48 | 10 |
.au | 1,951 | 1,876 | 59 | 16 |
.co | 1,740 | 1,648 | 23 | 69 |
.in | 1,734 | 1,670 | 52 | 11 |
.ar | 1,304 | 1,272 | 24 | 8 |
.nl | 1,064 | 1,030 | 30 | 3 |
.cl | 954 | 927 | 20 | 6 |
.tr | 866 | 833 | 30 | 3 |
.pe | 830 | 810 | 16 | 4 |
.se | 808 | 781 | 25 | 3 |
.ph | 783 | 761 | 13 | 9 |
.fi | 706 | 694 | 11 | 2 |
.be | 671 | 650 | 19 | 2 |
.ch | 653 | 632 | 19 | 2 |
.th | 642 | 623 | 15 | 5 |
.at | 612 | 588 | 22 | 3 |
.ru | 598 | 578 | 17 | 3 |
.id | 492 | 467 | 20 | 5 |
.ve | 429 | 419 | 9 | 2 |
.hu | 410 | 386 | 18 | 6 |
.my | 407 | 394 | 10 | 3 |
.pt | 397 | 385 | 11 | 2 |
.il | 374 | 365 | 7 | 2 |
.ro | 373 | 355 | 16 | 2 |
.sg | 367 | 360 | 6 | 2 |
.nz | 345 | 333 | 10 | 2 |
.dk | 314 | 303 | 11 | 1 |
.vn | 297 | 290 | 6 | 1 |
.ie | 288 | 282 | 6 | 1 |
.no | 282 | 271 | 10 | 1 |
.cn | 268 | 219 | 46 | 3 |
.gr | 257 | 247 | 9 | 1 |
.bg | 257 | 249 | 7 | 1 |
.za | 245 | 237 | 7 | 1 |
.ua | 215 | 209 | 5 | 1 |
.gt | 188 | 183 | 4 | 1 |
.cz | 186 | 176 | 10 | 1 |
.hr | 184 | 178 | 6 | 1 |
.do | 183 | 179 | 3 | 1 |
.tw | 180 | 171 | 8 | 1 |
.ec | 173 | 170 | 3 | 1 |
.hk | 169 | 163 | 4 | 1 |
.pk | 146 | 141 | 5 | 1 |
.sv | 144 | 141 | 2 | 1 |
.sk | 133 | 124 | 8 | 1 |
.cr | 132 | 129 | 2 | 1 |
.lt | 129 | 125 | 4 | 1 |
.uy | 127 | 125 | 2 | 1 |
.pr | 116 | 113 | 3 | 1 |
.bo | 114 | 111 | 2 | 1 |
.si | 114 | 111 | 3 | 1 |
.eg | 111 | 105 | 5 | 1 |
.sa | 92 | 87 | 4 | 1 |
.kr | 85 | 68 | 17 | 1 |
.ae | 84 | 82 | 2 | 1 |
.ma | 76 | 73 | 3 | 1 |
.ee | 68 | 66 | 2 | 1 |
.lv | 61 | 59 | 2 | 1 |
.py | 59 | 58 | 1 | 1 |
.hn | 53 | 51 | 1 | 1 |
.ba | 53 | 51 | 2 | 1 |
.lk | 41 | 39 | 2 | 1 |
.ni | 33 | 32 | 1 | 1 |
.bd | 30 | 29 | 1 | 1 |
.ke | 29 | 28 | 1 | 1 |
.jo | 28 | 26 | 1 | 1 |
.ng | 27 | 26 | 1 | 1 |
.tt | 27 | 26 | 1 | 1 |
.rs | 25 | 24 | 1 | 1 |
.kw | 25 | 24 | 1 | 1 |
.jm | 24 | 23 | 1 | 1 |
.is | 22 | 21 | 1 | 1 |
.lu | 22 | 21 | 1 | 1 |
.qa | 22 | 22 | 1 | 1 |
.mt | 21 | 21 | 1 | 1 |
.bh | 18 | 17 | 1 | 1 |
.by | 18 | 18 | 1 | 1 |
.mu | 17 | 17 | 1 | 1 |
.md | 17 | 16 | 1 | 1 |
.kz | 16 | 15 | 1 | 1 |
.pa | 14 | 13 | 1 | 1 |
.gh | 13 | 12 | 1 | 1 |
.lb | 13 | 13 | 1 | 1 |
.np | 11 | 11 | 1 | 1 |
.ge | 10 | 9 | 1 | 1 |
.bn | 9 | 8 | 1 | 1 |
.mv | 8 | 7 | 1 | 1 |
.om | 8 | 8 | 1 | |
.az | 8 | 8 | 1 | 1 |
.cu | 8 | 8 | 1 | 1 |
.ly | 7 | 7 | 1 | 1 |
.bs | 7 | 7 | 1 | 1 |
.ci | 6 | 6 | 1 | 1 |
.ps | 6 | 5 | 1 | |
.fj | 6 | 6 | 1 | 1 |
.sn | 6 | 6 | 1 | 1 |
.et | 6 | 6 | 1 | 1 |
.mz | 5 | 5 | 1 | |
.na | 4 | 4 | 1 | |
.tz | 4 | 4 | 1 | |
.am | 4 | 4 | 1 | 1 |
.dz | 4 | 4 | 1 | |
.mn | 4 | 4 | 1 | 1 |
.bw | 4 | 4 | 1 | 1 |
.cat | 4 | 4 | 1 | 1 |
.net | 4 | 2 | 2 | 1 |
.org | 4 | 1 | 3 | 1 |
.zw | 4 | 4 | 1 | |
.gp | 4 | 4 | 1 | |
.rw | 3 | 3 | | 1 |
.mg | 3 | 3 | 1 | 1 |
.ag | 3 | 3 | 1 | |
.bz | 3 | 3 | | |
.zm | 3 | 3 | | |
.dm | 2 | 2 | 1 | |
.gi | 2 | 2 | 1 | 1 |
.li | 2 | 2 | 1 | |
.cd | 2 | 1 | 1 | 1 |
.vi | 2 | 2 | 1 | |
.kh | 2 | 2 | 1 | |
.uz | 2 | 2 | | |
.ht | 2 | 1 | 1 | 1 |
.gy | 2 | 2 | 1 | |
.vc | 2 | 2 | | |
.ug | 1 | 1 | 1 | |
.af | 1 | 1 | 1 | 1 |
.ls | 1 | 1 | | |
.la | 1 | 1 | | |
.ai | 1 | 1 | 1 | |
.kg | 1 | 1 | | |
.me | 1 | 1 | 1 | 1 |
.vg | 1 | 1 | 1 | |
.as | 1 | 1 | | |
.bj | 1 | 1 | | |
.sm | 1 | 1 | | 1 |
.dj | 1 | 1 | | |
.gg | 1 | 1 | 1 | |
.ki | 1 | 1 | 1 | 1 |
.biz | 1 | 1 | 1 | |
.gl | 1 | 1 | 1 | |
.nu | 1 | 1 | 1 | |
.st | 1 | 1 | | |
.ck | 1 | 1 | | |
.ao | 1 | 1 | | |
.tm | 1 | 1 | | |
.ms | 1 | 1 | 1 | |
.pn | 1 | 1 | | |
.sb | 1 | 1 | | |
.gs | 1 | 1 | | |
.cg | 1 | 1 | | |
.sl | 1 | 1 | | |
.tv | 1 | 1 | 1 | 1 |
.us | 1 | 1 | 1 | |
.bi | 1 | 1 | | |
.tk | 1 | 1 | | |
.sc | 1 | 1 | 1 | |
.nr | 1 | 1 | | |
.vu | 1 | 1 | | |
.gf | 1 | 1 | | |
.edu | 1 | 1 | | |
.nf | 1 | 1 | | |
.mw | 1 | 1 | | |
.gm | 1 | 1 | | 1 |
.to | 1 | 1 | | |
.gov | 1 | | 1 | |
.je | 1 | 1 | | |
.cc | 1 | 1 | 1 | 1 |
.sh | 1 | 1 | | |
.tl | 1 | 1 | 1 | |
.tj | 1 | 1 | | |
.im | 1 | 1 | | |
.mk | 1 | 1 | | 1 |
.fm | 1 | 1 | | |
.ws | 1 | 1 | 1 | |
.ad | 1 | 1 | 1 | |
undefined | 39,041 | 21,411 | 6,819 | 10,812 |
Total | 133,969 | 113,082 | 9,491 | 11,398 |
|
In alphabetical order |
Requests originating from a Google ip address |
Service | Total | Pages | Images | Other |
Desktop | 2 | 1 | | 2 |
Earth | 1 | | 1 | |
FeedFetcher | 350 | 126 | 1 | 223 |
GoogleBot | 19,527 | 16,833 | 520 | 2,175 |
Image search | 1 | 1 | 1 | |
KeywordTool | 1 | 1 | | |
Mail | 1 | | 1 | |
Maps | 1 | 1 | 1 | |
Other | 191 | 106 | 33 | 53 |
Toolbar | 1 | 1 | | |
Translate | 387 | 387 | 1 | 1 |
Web search | 27 | 19 | 7 | 1 |
Wireless | 2,825 | 1,726 | 1,085 | 14 |
Total | 23,310 | 19,198 | 1,646 | 2,468 |
|
Requests originating from elsewhere |
Service | Total | Pages | Images | Other |
Desktop | 2,242 | 58 | 1 | 2,183 |
Earth | 819 | 158 | 654 | 8 |
FeedFetcher | 1 | 1 | | 1 |
GoogleBot? | 1,992 | 1,922 | 5 | 64 |
Image search | 3,118 | 1,797 | 1,274 | 48 |
Mail | 63 | 12 | 44 | 6 |
Maps | 191 | 53 | 135 | 3 |
Other | 10,221 | 222 | 4,526 | 5,472 |
Toolbar | 973 | 226 | 77 | 670 |
Translate | 29 | 15 | 8 | 6 |
Web search | 91,009 | 89,407 | 1,124 | 478 |
Wireless | 11 | 10 | 1 | |
Total | 110,668 | 93,880 | 7,847 | 8,938 |
|
Top level domains |
.ad | 1 | 1 | 1 | |
.ae | 84 | 82 | 2 | 1 |
.af | 1 | 1 | 1 | 1 |
.ag | 3 | 3 | 1 | |
.ai | 1 | 1 | 1 | |
.am | 4 | 4 | 1 | 1 |
.ao | 1 | 1 | | |
.ar | 1,304 | 1,272 | 24 | 8 |
.as | 1 | 1 | | |
.at | 612 | 588 | 22 | 3 |
.au | 1,951 | 1,876 | 59 | 16 |
.az | 8 | 8 | 1 | 1 |
.ba | 53 | 51 | 2 | 1 |
.bd | 30 | 29 | 1 | 1 |
.be | 671 | 650 | 19 | 2 |
.bg | 257 | 249 | 7 | 1 |
.bh | 18 | 17 | 1 | 1 |
.bi | 1 | 1 | | |
.biz | 1 | 1 | 1 | |
.bj | 1 | 1 | | |
.bn | 9 | 8 | 1 | 1 |
.bo | 114 | 111 | 2 | 1 |
.br | 3,681 | 3,601 | 64 | 17 |
.bs | 7 | 7 | 1 | 1 |
.bw | 4 | 4 | 1 | 1 |
.by | 18 | 18 | 1 | 1 |
.bz | 3 | 3 | | |
.ca | 2,766 | 2,674 | 82 | 10 |
.cat | 4 | 4 | 1 | 1 |
.cc | 1 | 1 | 1 | 1 |
.cd | 2 | 1 | 1 | 1 |
.cg | 1 | 1 | | |
.ch | 653 | 632 | 19 | 2 |
.ci | 6 | 6 | 1 | 1 |
.ck | 1 | 1 | | |
.cl | 954 | 927 | 20 | 6 |
.cn | 268 | 219 | 46 | 3 |
.co | 1,740 | 1,648 | 23 | 69 |
.com | 31,030 | 29,686 | 1,106 | 238 |
.cr | 132 | 129 | 2 | 1 |
.cu | 8 | 8 | 1 | 1 |
.cz | 186 | 176 | 10 | 1 |
.de | 7,369 | 7,168 | 178 | 23 |
.dj | 1 | 1 | | |
.dk | 314 | 303 | 11 | 1 |
.dm | 2 | 2 | 1 | |
.do | 183 | 179 | 3 | 1 |
.dz | 4 | 4 | 1 | |
.ec | 173 | 170 | 3 | 1 |
.edu | 1 | 1 | | |
.ee | 68 | 66 | 2 | 1 |
.eg | 111 | 105 | 5 | 1 |
.es | 2,285 | 2,227 | 48 | 10 |
.et | 6 | 6 | 1 | 1 |
.fi | 706 | 694 | 11 | 2 |
.fj | 6 | 6 | 1 | 1 |
.fm | 1 | 1 | | |
.fr | 3,646 | 3,527 | 100 | 19 |
.ge | 10 | 9 | 1 | 1 |
.gf | 1 | 1 | | |
.gg | 1 | 1 | 1 | |
.gh | 13 | 12 | 1 | 1 |
.gi | 2 | 2 | 1 | 1 |
.gl | 1 | 1 | 1 | |
.gm | 1 | 1 | | 1 |
.gov | 1 | | 1 | |
.gp | 4 | 4 | 1 | |
.gr | 257 | 247 | 9 | 1 |
.gs | 1 | 1 | | |
.gt | 188 | 183 | 4 | 1 |
.gy | 2 | 2 | 1 | |
.hk | 169 | 163 | 4 | 1 |
.hn | 53 | 51 | 1 | 1 |
.hr | 184 | 178 | 6 | 1 |
.ht | 2 | 1 | 1 | 1 |
.hu | 410 | 386 | 18 | 6 |
.id | 492 | 467 | 20 | 5 |
.ie | 288 | 282 | 6 | 1 |
.il | 374 | 365 | 7 | 2 |
.im | 1 | 1 | | |
.in | 1,734 | 1,670 | 52 | 11 |
.is | 22 | 21 | 1 | 1 |
.it | 3,023 | 2,950 | 63 | 10 |
.je | 1 | 1 | | |
.jm | 24 | 23 | 1 | 1 |
.jo | 28 | 26 | 1 | 1 |
.jp | 4,836 | 4,764 | 55 | 16 |
.ke | 29 | 28 | 1 | 1 |
.kg | 1 | 1 | | |
.kh | 2 | 2 | 1 | |
.ki | 1 | 1 | 1 | 1 |
.kr | 85 | 68 | 17 | 1 |
.kw | 25 | 24 | 1 | 1 |
.kz | 16 | 15 | 1 | 1 |
.la | 1 | 1 | | |
.lb | 13 | 13 | 1 | 1 |
.li | 2 | 2 | 1 | |
.lk | 41 | 39 | 2 | 1 |
.ls | 1 | 1 | | |
.lt | 129 | 125 | 4 | 1 |
.lu | 22 | 21 | 1 | 1 |
.lv | 61 | 59 | 2 | 1 |
.ly | 7 | 7 | 1 | 1 |
.ma | 76 | 73 | 3 | 1 |
.md | 17 | 16 | 1 | 1 |
.me | 1 | 1 | 1 | 1 |
.mg | 3 | 3 | 1 | 1 |
.mk | 1 | 1 | | 1 |
.mn | 4 | 4 | 1 | 1 |
.ms | 1 | 1 | 1 | |
.mt | 21 | 21 | 1 | 1 |
.mu | 17 | 17 | 1 | 1 |
.mv | 8 | 7 | 1 | 1 |
.mw | 1 | 1 | | |
.mx | 3,168 | 3,101 | 53 | 14 |
.my | 407 | 394 | 10 | 3 |
.mz | 5 | 5 | 1 | |
.na | 4 | 4 | 1 | |
.net | 4 | 2 | 2 | 1 |
.nf | 1 | 1 | | |
.ng | 27 | 26 | 1 | 1 |
.ni | 33 | 32 | 1 | 1 |
.nl | 1,064 | 1,030 | 30 | 3 |
.no | 282 | 271 | 10 | 1 |
.np | 11 | 11 | 1 | 1 |
.nr | 1 | 1 | | |
.nu | 1 | 1 | 1 | |
.nz | 345 | 333 | 10 | 2 |
.om | 8 | 8 | 1 | |
.org | 4 | 1 | 3 | 1 |
.pa | 14 | 13 | 1 | 1 |
.pe | 830 | 810 | 16 | 4 |
.ph | 783 | 761 | 13 | 9 |
.pk | 146 | 141 | 5 | 1 |
.pl | 2,540 | 2,480 | 50 | 10 |
.pn | 1 | 1 | | |
.pr | 116 | 113 | 3 | 1 |
.ps | 6 | 5 | 1 | |
.pt | 397 | 385 | 11 | 2 |
.py | 59 | 58 | 1 | 1 |
.qa | 22 | 22 | 1 | 1 |
.ro | 373 | 355 | 16 | 2 |
.rs | 25 | 24 | 1 | 1 |
.ru | 598 | 578 | 17 | 3 |
.rw | 3 | 3 | | 1 |
.sa | 92 | 87 | 4 | 1 |
.sb | 1 | 1 | | |
.sc | 1 | 1 | 1 | |
.se | 808 | 781 | 25 | 3 |
.sg | 367 | 360 | 6 | 2 |
.sh | 1 | 1 | | |
.si | 114 | 111 | 3 | 1 |
.sk | 133 | 124 | 8 | 1 |
.sl | 1 | 1 | | |
.sm | 1 | 1 | | 1 |
.sn | 6 | 6 | 1 | 1 |
.st | 1 | 1 | | |
.sv | 144 | 141 | 2 | 1 |
.th | 642 | 623 | 15 | 5 |
.tj | 1 | 1 | | |
.tk | 1 | 1 | | |
.tl | 1 | 1 | 1 | |
.tm | 1 | 1 | | |
.to | 1 | 1 | | |
.tr | 866 | 833 | 30 | 3 |
.tt | 27 | 26 | 1 | 1 |
.tv | 1 | 1 | 1 | 1 |
.tw | 180 | 171 | 8 | 1 |
.tz | 4 | 4 | 1 | |
.ua | 215 | 209 | 5 | 1 |
.ug | 1 | 1 | 1 | |
.uk | 4,787 | 4,637 | 134 | 15 |
.us | 1 | 1 | 1 | |
.uy | 127 | 125 | 2 | 1 |
.uz | 2 | 2 | | |
.vc | 2 | 2 | | |
.ve | 429 | 419 | 9 | 2 |
.vg | 1 | 1 | 1 | |
.vi | 2 | 2 | 1 | |
.vn | 297 | 290 | 6 | 1 |
.vu | 1 | 1 | | |
.ws | 1 | 1 | 1 | |
.za | 245 | 237 | 7 | 1 |
.zm | 3 | 3 | | |
.zw | 4 | 4 | 1 | |
undefined | 39,041 | 21,411 | 6,819 | 10,812 |
Total | 133,969 | 113,082 | 9,491 | 11,398 |
|
IP ranges: known ip ranges for Google are 64.233.[160.0-191.255], 66.249.[64.0-95.255], 66.102.[0.0-15.255], 72.14.[192.0-255.255], 74.125.[0.0-255.255], 209.085.[128.0-255.255], 216.239.[32.0-63.255] and a few minor other subranges
Agents: as for genuine agent strings: too many crawlers indentify themselves as 'GoogleBot' to take this at face value. They are accepted as genuine Google crawler requests only when the ip address matches a known range (see above). Other records that mention GoogleBot are counted as GoogleBot? (question mark, as this may include partners, like DoCoMo). However when the agent string mentions Google Desktop or Google Earth this is always accepted Service: the service name is based on the agent string (plus for GoogleBot check for ip address, see above), if this is inconclusive it is based on the referer string. Here is detailed breakdown per service of indicators that pointed to Google (total ≥ 3)
Service | Total | Originating from Google ip address | Referer mentions Google url | Agent mentions Google service |
Desktop | 2,241 | - | - | Y | Earth | 819 | - | - | Y | FeedFetcher | 350 | Y | - | Y | GoogleBot | 19,527 | Y | - | Y | GoogleBot? | 1,990 | - | - | Y | Image search | 3,118 | - | Y | - | Mail | 63 | - | Y | - | Maps | 191 | - | Y | - | Other | 52 | - | - | Y | Other | 10,164 | - | Y | - | Other | 5 | - | Y | Y | Other | 113 | Y | - | - | Other | 78 | Y | - | Y | Toolbar | 771 | - | - | Y | Toolbar | 154 | - | Y | - | Toolbar | 48 | - | Y | Y | Translate | 29 | - | Y | - | Translate | 131 | Y | - | Y | Translate | 256 | Y | Y | Y | Web search | 91,009 | - | Y | - | Web search | 27 | Y | Y | - | Wireless | 11 | - | - | Y | Wireless | 2,820 | Y | - | Y | Wireless | 5 | Y | Y | Y |
Top Level Domain 'undefined': requests with top level domain 'undefined' are nearly all requests from anonymous ip addresses (crawler and other services) Note: averages below 1 are always rounded up to 1
|