While around the world, statistical authorities in the public sector are still mostly regarded as the primary source for data and indicators, the private sector is increasingly moving into the business of providing data analyses and projecting trends.?Currently, stock markets still look anxiously forward to press releases by statistical authorities such as the US Bureau of Labor Statistics and others on unemployment rates, consumer price indices or growth rates of international trade. These trends and analyses are generally released with a delay of 3 to 4 weeks.
As the expectations in the current data rich societies are changing, the private sector is grabbing the opportunity to fill in the gap between a changing demand and the traditional sources of indicators. For example, companies like INDEED provide US industry employment trends, Zillow publishes a US House Value Index and shipping company Maersk released a study on trade and economic growth in China. These indicators from the private sector are published faster, more frequent and with more detail than the indicators from the public sector.
This play between public and private sector in the field of official statistics raises the question of what national statistical offices have to do to stay relevant.
Evidently, the private sector is moving much faster than the public sector in using available Big Data sources. Partnerships with the private sector in the use of Big Data for official statistics seem to be the only way forward. If you can’t beat them, join them
The third global conference on Big Data for official statistics
The United Nations Statistical Commission did respond to these changing market dynamics in 2014, by creating a Global Working Group (GWG) mandated to provide strategic vision, direction and coordination on the use of Big Data for official statistics.
The GWG has 32 members (22 countries and 10 international organizations) and works on 6 thematic areas (partnerships, capacity building, Sustainable Development Goals (SDGs) indicators, mobile phone data, social media data and satellite data).
It organized two global conferences in Beijing in 2014 and in Abu Dhabi in 2015, in which it showcased a variety of examples of Big Data projects for official statistics, such as mobile phone data for tourism and daytime population statistics, scanner data for price statistics, satellite data for agriculture statistics and social media data for consumer confidence indicators.
The statistical community has so far used Big Data only in pilot projects and has yet to implement its use in daily statistical production. Innovations are needed to incorporate those non-traditional sources of data, such as Big Data, in the production processes. This requires real partnerships with the private sector, new skills and infrastructure, and clear links between the available Big Data sources and the SDG indicators.
With these priorities in mind, the GWG organizes its third global conference on Big Data from 30 August to 1 September 2016 in Dublin, Ireland. This conference will focus on (1) Data access and partnerships with private sector and other communities, (2) Capacity building strategies related to the use of Big Data, and (3) using Big Data in the compilation of SDG indicators.
From the private sector, Google, Amazon, Proximus, IBM, LinkedIn, SAS and others will come to Dublin. From the side of the statistical community many developed and developing countries will participate and many of them represented by their senior managers, as they understand the clear need to bring the use of Big Data to the next level.