Big Data
The volume of data in the world is increasing exponentially. In 2020, 64.2 zettabytes of data were created, that is a 314 percent increase from 2015. An increased demand for information due to the COVID-19 pandemics also contribute to higher-than-expected growth. A large share of this output is “data exhaust,” or passively collected data deriving from everyday interactions with digital products or services, including mobile phones, credit cards, and social media. This deluge of digital data is known as big data. Data is growing because it is increasingly being gathered by inexpensive and numerous information‐sensing, mobile devices and because the world’s capacity for storing information has roughly doubled every 40 months since the 1980s.
The Data Revolution
The data revolution -- which encompasses the open data movement, the rise of crowdsourcing, new ICTs for data collection, and the explosion in the availability of big data, together with the emergence of artificial intelligence and the Internet of Things -- is already transforming society. Advances in computing and data science now make it possible to process and analyse big data in real time. New insights gleaned from such data mining can complement official statistics and survey data, adding depth and nuance to information on human behaviours and experiences. The integration of this new data with traditional data should produce high-quality information that is more detailed, timely and relevant.
Opportunities
Data is the lifeblood of decision-making and the raw material for accountability. Today, in the private sector, analysis of big data is commonplace, with consumer profiling, personalised services, and predictive analysis being used for marketing, advertising and management. Similar techniques could be adopted to gain real-time insights into people’s wellbeing and to target aid interventions to vulnerable groups. New sources of data - such as satellite data -, new technologies, and new analytical approaches, if applied responsibly, can enable more agile, efficient and evidence-based decision-making and can better measure progress on the Sustainable Development Goals (SDGs) in a way that is both inclusive and fair.
Risks
Fundamental elements of human rights must be safeguarded to realize the opportunities presented by big data: privacy, ethics and respect for data sovereignty require us to assess the rights of individuals along with the benefits of the collective. Much new data is collected passively – from the ‘digital footprints’ people leave behind and from sensor-enabled objects – or is inferred via algorithms. Because big data is the product of unique patterns of behaviour of individuals, removal of explicit personal information may not fully protect privacy. Combining multiple datasets may lead to the re-identification of individuals or groups of individuals, subjecting them to potential harms. Proper data protection measures must be put in place to prevent data misuse or mishandling.
There is also a risk of growing inequality and bias. Major gaps are already opening up between the data haves and have-nots. Without action, a whole new inequality frontier will split the world between those who know, and those who do not. Many people are excluded from the new world of data and information by language, poverty, lack of education, lack of technology infrastructure, remoteness or prejudice and discrimination. There is a broad range of actions needed, including building the capacities of all countries and particularly the Least Developed Countries (LDCs), Land-locked Developing Countries (LLDCs), and Small Island Developing States (SIDS).
Big Data for Development and Humanitarian Action
In 2015, the world embarked on a new development agenda underpinned by the Sustainable Development Goals (SDGs). Achieving these goals requires integrated action on social, environmental and economic challenges, with a focus on inclusive, participatory development that leaves no one behind.
Critical data for global, regional and national development policymaking is still lacking. Many governments still do not have access to adequate data on their entire populations. This is particularly true for the poorest and most marginalized, the very people that leaders will need to focus on if they are to achieve zero extreme poverty and zero emissions by 2030, and to ‘leave no one behind’ in the process.
Big data can shed light on disparities in society that were previously hidden. For example, women and girls, who often work in the informal sector or at home, suffer social constraints on their mobility, and are marginalized in both private and public decision-making.
Much of the big data with the most potential to be used for public good is collected by the private sector. As such, public-private partnerships are likely to become more widespread. The challenge will be ensuring they are sustainable over time, and that clear frameworks are in place to clarify roles and expectations on all sides.
Here is one example for each of the UN's Sustainable Development Goals showing how big data could be used to help achieve the SDGs:
SDG 1: No Poverty
Spending patterns on mobile phone services can provide proxy indicators of income levels
SDG 2: Zero Hunger
Crowdsourcing or tracking of food prices listed online can help monitor food security in near real-time
SDG 3: Good Health and Well-Being
Mapping the movement of mobile phone users can help predict the spread of infectious diseases
SDG 4: Quality Education
Citizen reporting can reveal reasons for student drop-out rates
SDG 5: Gender Equality
Analysis of financial transactions can reveal the spending patterns and different impacts of economic shocks on men and women
SDG 6: Clean Water and Sanitation
Sensors connected to water pumps can track access to clean water
SDG 7 Affordable and Clean Energy
Smart metering allows utility companies to increase or restrict the flow of electricity, gas or water to reduce waste and ensure adequate supply at peak periods
SDG 8: Decent Work and Economic Growth
Patterns in global postal traffic can provide indicators such as economic growth, remittances, trade and GDP
SDG 9: Industry, Innovation and Infrastructure
Data from GPS devices can be used for traffic control and to improve public transport
SDG 10: Reduced Inequality
Speech-to-text analytics on local radio content can reveal discrimination concerns and support policy response
SDG 11: Sustainable Cities and Communities
Satellite remote sensing can track encroachment on public land or spaces such as parks and forests
SDG 12: Responsible Consumption and Production
Online search patterns or e-commerce transactions can reveal the pace of transition to energy efficient products
SDG 13: Climate Action
Combining satellite imagery, crowd-sourced witness accounts and open data can help track deforestation
SDG 14: Life Below Water
Maritime vessel tracking data can reveal illegal, unregulated and unreported fishing activities
SDG 15: Life on Land
Social media monitoring can support disaster management with real-time information on victim location, effects and strength of forest fires or haze
SDG 16: Peace, Justice and Strong Institutions
Sentiment analysis of social media can reveal public opinion on effective governance, public service delivery or human rights
SDG 17: Partnerships for the Goals
Partnerships to enable the combining of statistics, mobile and internet data can provide a better and real-time understanding of today’s hyper-connected world
The role of the UN
One of the key roles of the UN and other international or regional organisations is setting principles and standards to guide collective action around the safe use of big data for development and humanitarian action within a global community and according to common norms. These standards seek to increase the usefulness of data through a much greater degree of openness and transparency, avoid invasion of privacy and abuse of human rights from misuse of data on individuals and groups, and minimise inequality in production, access to and use of data. Achievement of the SDGs in our digital world will require recognition of the need not only to prevent misuse of data, but also to ensure that when data can be used responsibly for the public good, it is.
The Secretary-General’s Independent Expert Advisory Group on a Data Revolution for Sustainable Development (IEAG) has made on how to address these challenges, calling for a UN-led effort to mobilise the data revolution for sustainable development, by:
- Fostering and promoting innovation to fill data gaps.
- Mobilising resources to overcome inequalities between developed and developing countries and between data-poor and data-rich people.
- Leadership and coordination to enable the data revolution to play its full role in the realisation of sustainable development.
Uptake of big data analytics is accelerating across the UN system with a growing number of UN agencies, funds and programmes implementing and scaling operational applications for development and humanitarian use.
The UN Development Group has issued on data privacy, data protection and data ethics concerning the use of big data, collected in real time by private sector entities as part of their business offerings, and shared with UNDG members for the purposes of strengthening operational implementation of their programmes to support the achievement of the 2030 Agenda.
The first held in January 2017 brought together over 1,400 data users and producers from the public and private sectors, policy makers, academia and civil society to explore ways to harness the power of data for sustainable development. It produced important outcomes, including the launch of the .
The UN World Data Forum community has grown significantly since 2017, from 2,000 attendees at the first and second Forums held in Cape Town and Dubai, to an active list of over 20,000 interested stakeholders. This growth was due to the more open and accessible virtual Forums held in 2020, and a hybrid format in Bern, Switzerland in 2021, and Hangzhou, China in 2023. The stakeholder community is diverse, representing various sectors, including governments, civil society, the private sector, donors, philanthropic bodies, international and regional agencies, among other actors.
Every UN World Data Forum leads to the release of an outcome document, which tracks the progress of discussions around data and statistics and expresses the ambitions of the stakeholder community. The Cape Town Global Action Plan (CTGAP) was launched at the first UN World Data Forum. The CTGAP was followed by the (2018), which called for an innovative funding mechanism to support the implementation of the CTGAP.
More recently, the Global data community’s response to Covid-19 (2020) and (2021) were launched to position official statistics and National Statistical Offices (NSOs) during Covid-19 and in the wider data ecosystem generally. At the most recent fourth Forum, the (2023) was launched to recommit the global community to accelerating progress in the implementation of the Cape Town Global Action Plan for Sustainable Development Data.
UN Global Pulse
is an innovation initiative of the UN Secretary-General on data science. Global Pulse promotes awareness of the opportunities big data presents for sustainable development and humanitarian action, develops high-impact analytics solutions for UN and government partners through its network of data science innovation centres, or Pulse Labs, in Indonesia (Jakarta), Uganda (Kampala) and the UN Headquarters (New York), and works to lower barriers to adoption and scaling.
To safely and responsibly unlock the value of data, Global Pulse established a data privacy programme, part of which involves ongoing research into privacy-protective uses of big data for humanitarian and development purposes. Global Pulse set up a Data Privacy Advisory Group, comprised of privacy experts from the regulatory community, private sector and academia, that engages in dialogue on the critical issues around big data and advises on the development of privacy tools and guidelines across the UN. To better understand the risks linked to big data, Global Pulse developed a two-phase “Risk, Harms and Benefits Assessment” tool, which includes guidelines to help practitioners assess the proportionality of the risks, harms, and utility in a data-driven project.
Global Pulse was also involved in the organization of the , an initiative led by UNICEF and WFP. Consisting of five thematic workshops, the series aimed to understand existing data innovation capabilities and needs within the UN system.
Public - Private Partnerships
To ensure that access to insights from big data across many industries is widely available, Global Pulse has been working with the private sector to operationalize the concept of ‘data philanthropy,’ whereby companies' data can be safely and responsibly used for sustainable development and humanitarian action. For example, in 2016, Global Pulse formed a partnership with the social media network Twitter.
Every day, people around the world send hundreds of millions of tweets in dozens of languages. Such social conversations contain real-time information on many issues, including food costs, the availability of jobs, access to health care, quality of education, and reports of natural disasters. The partnership will allow UN development and humanitarian agencies to turn the public data into actionable information to aid communities around the globe.
Other examples of partnerships include the GSMA’s “” initiative, which leverages mobile operators’ big data capabilities to address humanitarian crises, including epidemics and natural disasters; , a competition which connected researchers around the world with data and tools from leading companies to enable data-driven climate solutions; and , a new form of collaboration beyond the public-private partnership model, in which participants from different sectors? (and companies in particular) exchange their data to create public value.
