Wikistats unique visitors (by ip) and total visits for April 2015

Note: only html requests

Bots/crawlers have been removed pretty aggressively, so this is rather conservative assessment of Wikistats usage. For more on this see below

Data sources: I: wikimedia dumps, II: hourly page view totals, derived from this, III: 1:1000 sampled squid logs (non public), IV: monthly aggregations of hourly page view counts, E: external, *: misc. sources, M: manual

See also Wikistats Overview

Dump Reports


Folders:
/PP = project name (wikibooks, wiktionary, etc ; empty for wikipedia)
/LL = target language code (EN for English, DE for German, etc)
/AA = animations (/wikimedia/animations/growth/..
/CC = continent (EN_Asia, EN_Europe, etc)
[green] generic description
[red] unexpected requests

Part of file name:
[XX] = wiki language code (EN for English, DE for German, etc)
[CC] = continent (Asia, Europe, etc)
[yyyy-mm] year month

uniq
ip		 unique
pages		 total
requests		 srce generic path most frequently requested pages (hover over link for top 10 list)
70741697 12879I[data about one wiki, both about database content and content editors]../EN/TablesWikipediaEN.htm(1374) (297) (291) (278) (184) (162) (155) (153) (151) (143) etc
24471003872I[sitemaps: list of wikis per project, plus basic metrics]../EN/Sitemap.htm(2050) (194) (167) (137) (90) (90) (77) (72) (66) (65) etc
16902643343I[comparison reports for all wikis within one project]../EN/TablesWikipediansEditsGt5.htm(1221) (307) (185) (167) (128) (102) (101) (92) (69) (62) etc
10681531705I[totally outdated, still linked, category hierarchies]../wikibooks/EN/CategoryOverview_EN_Complete.htm(396) (198) (127) (86) (68) (55) (43) (42) (41) (34) etc
9952521456I[set of bar charts for one wiki (up to date, but awkwardly large, see also summaries)]../FR/ChartsWikipediaFR.htm(197) (123) (88) (70) (59) (50) (43) (38) (28) (27) etc
69126976I[totally outdated, only linked elsewhere?, EasyTimeline stats]../EN/TimelinesFR.htm(351) (334) (76) (54) (27) (22) (20) (16) (14) (13) etc
440165725I[summary per wiki: MoM, YoY and monthly trends]../EN/SummaryEN.htm(126) (43) (32) (21) (19) (18) (17) (15) (13) (12) etc
19616325I[summary per group wikis: MoM, YoY and monthly trends]../EN/ReportCardTopWikis.htm(199) (20) (18) (11) (10) (10) (9) (8) (8) (7) etc
1954293M[meta pages: portal, index, about]../index.html(220) (39) (24) (10)
18872269I[most edited articles per wiki (some are empty)]../EN/TablesWikipediaArticleEditsEN.htm(53) (21) (13) (10) (9) (8) (8) (7) (7) (6) etc
17539259I[totally outdated, still linked, weekly trend plots]../wikiquote/RU/PlotsPngArticlesTotal.htm(60) (39) (27) (26) (14) (9) (6) (6) (5) (5) etc
15220199I[recent months for all wikis within one project, table + bar chart]../EN/TablesRecentTrends.htm(70) (28) (27) (17) (10) (7) (6) (6) (4) (4) etc
1381370I[active editors for all Wikimedia project combined, deduplicated]../EN/TablesWikimediaAllProjects.htm(370)
9910217I[animations:project growth]../wikimedia/animations/growth/AnimationProjectsGrowthWp.html(114) (53) (11) (8) (8) (6) (5) (5) (5) (2)
9294175I[edit & revert history tables and plots per wiki]../wiktionary/EN/EditsRevertsAN.htm(14) (10) (8) (6) (5) (4) (4) (4) (3) (3) etc
531471I[current status per wiki: wide variety of metrics]../EN/TablesCurrentStatusVerbose.htm(41) (7) (5) (3) (3) (2) (2) (2) (1) (1) etc
421878I[edit & revert history tables and plots per group wikis]../EN/PlotsPngEditHistoryTop.htm(20) (13) (6) (6) (4) (3) (3) (3) (3) (3) etc
33946I/PP/LL/BotActivityMatrix.htm../EN/BotActivityMatrix.htm(25) (6) (4) (3) (3) (2) (1) (1) (1)
33438I/PP/LL/BotActivityMatrixCreates.htm../EN/BotActivityMatrixCreates.htm(34) (2) (1) (1)
31243I/PP/LL/BotActivityMatrixEdits.htm../EN/BotActivityMatrixEdits.htm(42) (1)
27440I/PP/LL/TablesUsagePageRequest.htm../EN/TablesUsagePageRequest.htm(35) (3) (1) (1)
2744104*[original report card]../reportcard/RC_2012_02_detailed.html(16) (13) (9) (9) (5) (4) (3) (3) (2) (2) etc
13230I[dump progress reports (generation/processing)]../WikiCountsJobProgress.html(22) (8)
11115I/PP/LL/TableRankArticleHistoryByArchiveSize.html../EN/TableRankArticleHistoryByArchiveSize.html(15)
9111I/PP/LL/TableRankArticleHistoryByTotalEdits.html../EN/TableRankArticleHistoryByTotalEdits.html(11)
7111?/index_tabbed_new.html../index_tabbed_new.html(11)

Traffic Reports


Folders:

/SS = traffic reports: /archive/squid_reports/yyyy-mm/.. or /wikimedia/squids/..

[green] generic description
[red] unexpected requests
uniq
ip		 unique
pages		 total
requests		 srce generic path most frequently requested pages (hover over link for top 10 list)
1222571846III/SS/SquidReportClients.htm../wikimedia/squids/SquidReportClients.htm(392) (345) (217) (180) (121) (105) (68) (51) (41) (40) etc
713341302II[page view totals, per wiki/project]../EN/TablesPageViewsMonthlyCombined.htm(355) (298) (187) (155) (97) (87) (26) (14) (13) (8) etc
6491671028IV[outdated (linked on many external sites), most requested pages per wiki]../wikimedia/pagecounts/reports/2012-12/most-requested-pages-2012-12-wikimedia-COMMONS.html(98) (95) (63) (61) (34) (34) (28) (25) (24) (22) etc
51857874III/SS/SquidReportOperatingSystems.htm../wikimedia/squids/SquidReportOperatingSystems.htm(394) (47) (47) (42) (33) (22) (21) (19) (17) (16) etc
33431600III[page views per country, overview]../wikimedia/squids/SquidReportPageViewsPerCountryOverview.htm(201) (191) (108) (28) (14) (11) (4) (3) (3) (3) etc
29069524III/SS/SquidReportCrawlers.htm../wikimedia/squids/SquidReportCrawlers.htm(179) (28) (24) (22) (14) (13) (12) (10) (9) (9) etc
17177227IV[page views per category subtree]../wikimedia/pageviews/categorized/wp-en/2013-10/categories_wp-en_cat_Lists_2013-10.html(52) (25) (11) (6) (6) (6) (5) (4) (4) (4) etc
1524230III[page views per language, breakdown per country]../wikimedia/squids/SquidReportPageViewsPerLanguageBreakdown.htm(197) (30) (2) (1)
1314174III[page edits per language, breakdown per country]../wikimedia/squids/SquidReportPageEditsPerLanguageBreakdown.htm(167) (4) (2) (1)
11631214III[page edits per country, overview, trends, breakdown per language]../wikimedia/squids/SquidReportPageEditsPerCountryTrends.htm(69) (53) (29) (21) (5) (2) (2) (2) (2) (2) etc
11537283III/SS/SquidReportOrigins.htm../wikimedia/squids/SquidReportOrigins.htm(47) (43) (33) (18) (18) (14) (13) (9) (8) (8) etc
61884III/SS/SquidReportCountryOs.htm../wikimedia/squids/SquidReportCountryOs.htm(67) (6) (4) (2) (2) (1) (1) (1)
571100III[animations:edits per day]../wikimedia/animations/requests/AnimationEditsOneDayWp.html(100)
553377III/SS/SquidReportMethods.htm../wikimedia/squids/SquidReportMethods.htm(20) (6) (4) (3) (3) (3) (3) (2) (2) (2) etc
551480III/SS/SquidReportGoogle.htm ../wikimedia/squids/SquidReportGoogle.htm(26) (18) (13) (8) (5) (2) (1) (1) (1) (1) etc
441158III/SS/SquidReportRequests.htm../wikimedia/squids/SquidReportRequests.htm(28) (15) (3) (2) (2) (2) (2) (1) (1) (1) etc
421130III/archive/squid_reports_edits//[yyyy-mm]//SquidReportCrawlers.htm../archive/squid_reports_edits/2012-11/SquidReportCrawlers.htm(130)
422067III/SS/SquidReportScripts.htm../archive/squid_reports/2013-04/SquidReportScripts.htm(18) (15) (5) (4) (3) (2) (2) (2) (2) (2) etc
37955III/SS/SquidReportCountryData.htm../wikimedia/squids/SquidReportCountryData.htm(26) (15) (4) (4) (2) (1) (1) (1) (1)
331044III/SS/SquidReportUserAgents.htm../wikimedia/squids/SquidReportUserAgents.htm(20) (6) (6) (4) (2) (2) (1) (1) (1) (1)
17624III/SS/SquidReportCountryBrowser.htm../wikimedia/squids/SquidReportCountryBrowser.htm(7) (6) (5) (3) (2) (1)
16319III/SS/SquidReportSkins.htm../wikimedia/squids/SquidReportSkins.htm(17) (1) (1)
15621III/SS/SquidReportBrowsersTimed.htm../archive/squid_reports/2015-01/SquidReportBrowsersTimed.htm(8) (7) (2) (2) (1) (1)
13116III/PP/LL/VisitorsSampledLogOperatingSystems.htm../EN/VisitorsSampledLogOperatingSystems.htm(16)
13115III/archive/squid_reports//[yyyy-mm]/-new/SquidReportOperatingSystems.htm../archive/squid_reports/2015-01-new/SquidReportOperatingSystems.htm(15)

Bots/crawlers have been removed pretty aggressively, so this is rather conservative assessment of Wikistats usage.

Phase 1, removed all lines where user agent string contains 'bot', 'crawl', 'spider', 'http' or 'slurp' plus 2 ultra-active ip addresses-> 146,010 of 1,150,888 lines kept (13%), 17,664 of 21,088 ip addresses kept (84%)
Lines where user agent string contains 'bot': 6.0%, 'crawl': 1.1%, 'spider': 4.7%, 'http': 16.0%, 'slurp': 0.1%
Lines with 2 ultra active ip addresses': 25.8%

Phase 2, removed ip addresses with bot-like behavior (> 50 page views per month, at least once > 6 page views per minute, same page more than 10 times in a month, requesting too many non existing pages or too many very old pages etc -> 36,662 lines kept (3.2%) 16,421 ip addresses kept (78%)

Only rows with at least 10 total requests are shown.

Some Wikistats users may have used dynamic ip addresses, thus affecting overall uniq ip count. ;

Hive query used: USE wmf_raw ; SELECT uri_host, uri_path, http_status, ip, dt, referer, user_agent FROM webrequest WHERE uri_host = 'stats.wikimedia.org' AND uri_path LIKE '%htm%' AND year=2015 AND month=4 ORDER BY ip ASC LIMIT 10000000 ;