Data equity and official statistics in the age of private sector data proliferation

Curated on Posted on by Stefaan Verhulst

Paper by Pietro Gennari: “Over the last few years, the private sector has become a primary generator of data due to widespread digitisation of the economy and society, the use of social media platforms, and advancements of technologies like the Internet of Things and AI. Unlike traditional sources, these new data streams often offer real-time information and unique insights into people’s behaviour, social dynamics, and economic trends. However, the proprietary nature of most private sector data presents challenges for public access, transparency, and governance that have led to fragmented, often conflicting, data governance arrangements worldwide. This lack of coherence can exacerbate inequalities, limit data access, and restrict data’s utility as a global asset.

Within this context, data equity has emerged as one of the key principles at the basis of any proposal of new data governance framework. The term “data equity” refers to the fair and inclusive access, use, and distribution of data so that it benefits all sections of society, regardless of socioeconomic status, race, or geographic location. It involves making sure that the collection, processing, and use of data does not disproportionately benefit or harm any particular group and seeks to address disparities in data access and quality that can perpetuate social and economic inequalities. This is important because data systems significantly influence access to resources and opportunities in society. In this sense, data equity aims to correct imbalances that have historically affected various groups and to ensure that decision-making based on data does not perpetuate these inequities…(More)”.

The Data Innovation Toolkit

Curated on Posted on by Stefaan Verhulst

Toolkit by Maria Claudia Bodino, Nathan da Silva Carvalho, Marcelo Cogo, Arianna Dafne Fini Storchi, and Stefaan Verhulst: “Despite the abundance of data, the excitement around AI, and the potential for transformative insights, many public administrations struggle to translate data into actionable strategies and innovations. 

Public servants working with data-related initiatives, need practical, easy-to-use resources designed to enhance the management of data innovation initiatives. 

In order to address these needs, the iLab of DG DIGIT from the European Commission is developing an initial set of practical tools designed to facilitate and enhance the implementation of data-driven initiatives. The main building blocks of the first version of the of the Digital Innovation Toolkit include: 

  1. A Repository of educational materials and resources on the latest data innovation approaches from public sector, academia, NGOs and think tanks 
  2. An initial set of practical resources, some examples: 
  3. Workshop Templates to offer structured formats for conducting productive workshops that foster collaboration, ideation, and problem-solving. 
  4. Checklists to ensure that all data journey aspects and steps are properly assessed. 
  5. Interactive Exercises to engage team members in hands-on activities that build skills and facilitate understanding of key concepts and methodologies. 
  6. Canvas Models to provide visual frameworks for planning and brainstorming….(More)”.

When forecasting and foresight meet data and innovation: toward a taxonomy of anticipatory methods for migration policy

Curated on Posted on by Stefaan Verhulst

Paper by Sara Marcucci, Stefaan Verhulst and María Esther Cervantes: “The various global refugee and migration events of the last few years underscore the need for advancing anticipatory strategies in migration policy. The struggle to manage large inflows (or outflows) highlights the demand for proactive measures based on a sense of the future. Anticipatory methods, ranging from predictive models to foresight techniques, emerge as valuable tools for policymakers. These methods, now bolstered by advancements in technology and leveraging nontraditional data sources, can offer a pathway to develop more precise, responsive, and forward-thinking policies.

This paper seeks to map out the rapidly evolving domain of anticipatory methods in the realm of migration policy, capturing the trend toward integrating quantitative and qualitative methodologies and harnessing novel tools and data. It introduces a new taxonomy designed to organize these methods into three core categories: Experience-based, Exploration-based, and Expertise-based. This classification aims to guide policymakers in selecting the most suitable methods for specific contexts or questions, thereby enhancing migration policies…(More)”

Combine AI with citizen science to fight poverty

Curated on Posted on by Stefaan Verhulst

Nature Editorial: “Of the myriad applications of artificial intelligence (AI), its use in humanitarian assistance is underappreciated. In 2020, during the COVID-19 pandemic, Togo’s government used AI tools to identify tens of thousands of households that needed money to buy food, as Nature reports in a News Feature this week. Typically, potential recipients of such payments would be identified when they apply for welfare schemes, or through household surveys of income and expenditure. But such surveys were not possible during the pandemic, and the authorities needed to find alternative means to help those in need. Researchers used machine learning to comb through satellite imagery of low-income areas and combined that knowledge with data from mobile-phone networks to find eligible recipients, who then received a regular payment through their phones. Using AI tools in this way was a game-changer for the country.Can AI help beat poverty? Researchers test ways to aid the poorest people

Now, with the pandemic over, researchers and policymakers are continuing to see how AI methods can be used in poverty alleviation. This needs comprehensive and accurate data on the state of poverty in households. For example, to be able to help individual families, authorities need to know about the quality of their housing, their children’s diets, their education and whether families’ basic health and medical needs are being met. This information is typically obtained from in-person surveys. However, researchers have seen a fall in response rates when collecting these data.

Missing data

Gathering survey-based data can be especially challenging in low- and middle-income countries (LMICs). In-person surveys are costly to do and often miss some of the most vulnerable, such as refugees, people living in informal housing or those who earn a living in the cash economy. Some people are reluctant to participate out of fear that there could be harmful consequences — deportation in the case of undocumented migrants, for instance. But unless their needs are identified, it is difficult to help them.Leveraging the collaborative power of AI and citizen science for sustainable development

Could AI offer a solution? The short answer is, yes, although with caveats. The Togo example shows how AI-informed approaches helped communities by combining knowledge of geographical areas of need with more-individual data from mobile phones. It’s a good example of how AI tools work well with granular, household-level data. Researchers are now homing in on a relatively untapped source for such information: data collected by citizen scientists, also known as community scientists. This idea deserves more attention and more funding.

Thanks to technologies such as smartphones, Wi-Fi and 4G, there has been an explosion of people in cities, towns and villages collecting, storing and analysing their own social and environmental data. In Ghana, for example, volunteer researchers are collecting data on marine litter along the coastline and contributing this knowledge to their country’s official statistics…(More)”.

How tax data unlocks new insights for industrial policy

Curated on Posted on by Stefaan Verhulst

OECD article: “Value-added tax (VAT) is a consumption tax applied at each stage of the supply chain whenever value is added to goods or services. Businesses collect and remit VAT. The VAT data that are collected represent a breakthrough in studying production networks because they capture actual transactions between firms at an unprecedented level of detail. Unlike traditional business surveys or administrative data that might tell us about a firm’s size or industry, VAT records show us who does business with whom and for how much.

This data is particularly valuable because of its comprehensive coverage. In Estonia, for example, all VAT-registered businesses must report transactions above €1,000 per month, creating an almost complete picture of significant business relationships in the economy.

At least 15 countries now have such data available, including Belgium, Chile, Costa Rica, Estonia, and Italy. This growing availability creates opportunities for cross-country comparison and broader economic insights…(More)”.

Farmers Sue Over Deletion of Climate Data From Government Websites

Curated on Posted on by Stefaan Verhulst

Article by Karen Zraick: “Organic farmers and environmental groups sued the Agriculture Department on Monday over its scrubbing of references to climate change from its website.

The department had ordered staff to take down pages focused on climate change on Jan. 30, according to the suit, which was filed in the United States District Court for the Southern District of New York. Within hours, it said, information started disappearing.

That included websites containing data sets, interactive tools and funding information that farmers and researchers relied on for planning and adaptation projects, according to the lawsuit.

At the same time, the department also froze funding that had been promised to businesses and nonprofits through conservation and climate programs. The purge then “removed critical information about these programs from the public record, denying farmers access to resources they need to advocate for funds they are owed,” it said.

The Agriculture Department referred questions about the lawsuit to the Justice Department, which did not immediately respond to a request for comment.

The suit was filed by lawyers from Earthjustice, based in San Francisco, and the Knight First Amendment Institute at Columbia University, on behalf of the Northeast Organic Farming Association of New York, based in Binghamton; the Natural Resources Defense Council, based in New York; and the Environmental Working Group, based in Washington. The latter two groups relied on the department website for their research and advocacy, the lawsuit said.

Peter Lehner, a lawyer for Earthjustice, said the pages being purged were crucial for farmers facing risks linked to climate change, including heat waves, droughts, floods, extreme weather and wildfires. The websites had contained information about how to mitigate dangers and adopt new agricultural techniques and strategies. Long-term weather data and trends are valuable in the agriculture industry for planning, research and business strategy.

“You can purge a website of the words climate change, but that doesn’t mean climate change goes away,” Mr. Lehner said…(More)”.

Governing in the Age of AI: Building Britain’s National Data Library

Curated on Posted on by Stefaan Verhulst

Report by the Tony Blair Institute for Global Change: “The United Kingdom should lead the world in artificial-intelligence-driven innovation, research and data-enabled public services. It has the data, the institutions and the expertise to set the global standard. But without the right infrastructure, these advantages are being wasted.

The UK’s data infrastructure, like that of every nation, is built around outdated assumptions about how data create value. It is fragmented and unfit for purpose. Public-sector data are locked in silos, access is slow and inconsistent, and there is no system to connect and use these data effectively, or any framework for deciding what additional data would be most valuable to collect given AI’s capabilities.

As a result, research is stalled, AI adoption is held back, and the government struggles to plan services, target support and respond to emerging challenges. This affects everything from developing new treatments to improving transport, tackling crime and ensuring economic policies help those who need them. While some countries are making progress in treating existing data as strategic assets, none have truly reimagined data infrastructure for an AI-enabled future…(More)”

On the Shoulders of Others: The Importance of Regulatory Learning in the Age of AI

Curated on Posted on by Stefaan Verhulst

Paper by Urs Gasser and Viktor Mayer-Schonberger: “…International harmonization of regulation is the right strategy when the appropriate regulatory ends and means are sufficiently clear to reap efficiencies of scale and scope. When this is not the case, a push for efficiency through uniformity is premature and may lead to a suboptimal regulatory lock-in: the establishment of a rule framework that is either inefficient in the use of its means to reach the intended goal, or furthers the wrong goal, or both.


A century ago, economist Joseph Schumpeter suggested that companies have two distinct strategies to achieve success. The first is to employ economies of scale and scope to lower their cost. It’s essentially a push for improved efficiency. The other strategy is to invent a new product (or production process) that may not, at least initially, be hugely efficient, but is nevertheless advantageous because demand for the new product is price inelastic. For Schumpeter this was the essence of innovation. But, as Schumpeter also argued, innovation is not a simple, linear, and predictable process. Often, it happens in fits and starts, and can’t be easily commandeered or engineered.


As innovation is hard to foresee and plan, the best way to facilitate it is to enable a wide variety of different approaches and solutions. Public policies in many countries to foster startups and entrepreneurship stems from this view. Take, for instance, the policy of regulatory sandboxing, i.e. the idea that for a limited time certain sectors should not or only lightly be regulated…(More)”.

The Preventative Shift: How can we embed prevention and achieve long term missions

Curated on Posted on by Stefaan Verhulst

Paper by Demos (UK): “Over the past two years Demos has been making the case for a fundamental shift in the purpose of government away from firefighting in public services towards preventing problems arriving. First, we set out the case for The Preventative State, to rebuild local, social and civic foundations; then, jointly with The Health Foundation, we made the case to change treasury rules to ringfence funding for prevention. By differentiating between everyday spending, and preventative spending, the government could measure what really matters.

There has been widespread support for this – but also concerns about both the feasibility of measuring preventative spending accurately and appropriately but also that ring-fencing alone may not lead to the desired improvements in outcomes and value for money.

In response we have developed two practical approaches, covered in two papers:

  • Our first paper, Counting What Matters, explores the challenge of measurement and makes a series of recommendations, including the passage of a “Public Investment Act”, to show how this could be appropriately achieved.
  • This second paper, The Preventative Shift, looks at how to shift the culture of public bodies to think ‘prevention first’ and target spending at activities which promise value for money and improve outcomes…(More)”.

A.I. Is Prompting an Evolution, Not an Extinction, for Coders

Curated on Posted on by Stefaan Verhulst

Article by Steve Lohr: “John Giorgi uses artificial intelligence to make artificial intelligence.

The 29-year-old computer scientist creates software for a health care start-up that records and summarizes patient visits for doctors, freeing them from hours spent typing up clinical notes.

To do so, Mr. Giorgi has his own timesaving helper: an A.I. coding assistant. He taps a few keys and the software tool suggests the rest of the line of code. It can also recommend changes, fetch data, identify bugs and run basic tests. Even though the A.I. makes some mistakes, it saves him up to an hour many days.

“I can’t imagine working without it now,” Mr. Giorgi said.

That sentiment is increasingly common among software developers, who are at the forefront of adopting A.I. agents, assistant programs tailored to help employees do their jobs in fields including customer service and manufacturing. The rapid improvement of the technology has been accompanied by dire warnings that A.I. could soon automate away millions of jobs — and software developers have been singled out as prime targets.

But the outlook for software developers is more likely evolution than extinction, according to experienced software engineers, industry analysts and academics. For decades, better tools have automated some coding tasks, but the demand for software and the people who make it has only increased.

A.I., they say, will accelerate that trend and level up the art and craft of software design.

“The skills software developers need will change significantly, but A.I. will not eliminate the need for them,” said Arnal Dayaratna, an analyst at IDC, a technology research firm. “Not anytime soon anyway.”

The outlook for software engineers offers a window into the impact that generative A.I. — the kind behind chatbots like OpenAI’s ChatGPT — is likely to have on knowledge workers across the economy, from doctors and lawyers to marketing managers and financial analysts. Predictions about the technology’s consequences vary widely, from wiping out whole swaths of the work force to hyper-charging productivity as an elixir for economic growth…(More)”.