Data equity and official statistics in the age of private sector data proliferation

Curated on Posted on by Stefaan Verhulst

Paper by Pietro Gennari: “Over the last few years, the private sector has become a primary generator of data due to widespread digitisation of the economy and society, the use of social media platforms, and advancements of technologies like the Internet of Things and AI. Unlike traditional sources, these new data streams often offer real-time information and unique insights into people’s behaviour, social dynamics, and economic trends. However, the proprietary nature of most private sector data presents challenges for public access, transparency, and governance that have led to fragmented, often conflicting, data governance arrangements worldwide. This lack of coherence can exacerbate inequalities, limit data access, and restrict data’s utility as a global asset.

Within this context, data equity has emerged as one of the key principles at the basis of any proposal of new data governance framework. The term “data equity” refers to the fair and inclusive access, use, and distribution of data so that it benefits all sections of society, regardless of socioeconomic status, race, or geographic location. It involves making sure that the collection, processing, and use of data does not disproportionately benefit or harm any particular group and seeks to address disparities in data access and quality that can perpetuate social and economic inequalities. This is important because data systems significantly influence access to resources and opportunities in society. In this sense, data equity aims to correct imbalances that have historically affected various groups and to ensure that decision-making based on data does not perpetuate these inequities…(More)”.

The Data Innovation Toolkit

Curated on Posted on by Stefaan Verhulst

Toolkit by Maria Claudia Bodino, Nathan da Silva Carvalho, Marcelo Cogo, Arianna Dafne Fini Storchi, and Stefaan Verhulst: “Despite the abundance of data, the excitement around AI, and the potential for transformative insights, many public administrations struggle to translate data into actionable strategies and innovations. 

Public servants working with data-related initiatives, need practical, easy-to-use resources designed to enhance the management of data innovation initiatives. 

In order to address these needs, the iLab of DG DIGIT from the European Commission is developing an initial set of practical tools designed to facilitate and enhance the implementation of data-driven initiatives. The main building blocks of the first version of the of the Digital Innovation Toolkit include: 

  1. A Repository of educational materials and resources on the latest data innovation approaches from public sector, academia, NGOs and think tanks 
  2. An initial set of practical resources, some examples: 
  3. Workshop Templates to offer structured formats for conducting productive workshops that foster collaboration, ideation, and problem-solving. 
  4. Checklists to ensure that all data journey aspects and steps are properly assessed. 
  5. Interactive Exercises to engage team members in hands-on activities that build skills and facilitate understanding of key concepts and methodologies. 
  6. Canvas Models to provide visual frameworks for planning and brainstorming….(More)”.

How tax data unlocks new insights for industrial policy

Curated on Posted on by Stefaan Verhulst

OECD article: “Value-added tax (VAT) is a consumption tax applied at each stage of the supply chain whenever value is added to goods or services. Businesses collect and remit VAT. The VAT data that are collected represent a breakthrough in studying production networks because they capture actual transactions between firms at an unprecedented level of detail. Unlike traditional business surveys or administrative data that might tell us about a firm’s size or industry, VAT records show us who does business with whom and for how much.

This data is particularly valuable because of its comprehensive coverage. In Estonia, for example, all VAT-registered businesses must report transactions above €1,000 per month, creating an almost complete picture of significant business relationships in the economy.

At least 15 countries now have such data available, including Belgium, Chile, Costa Rica, Estonia, and Italy. This growing availability creates opportunities for cross-country comparison and broader economic insights…(More)”.

Farmers Sue Over Deletion of Climate Data From Government Websites

Curated on Posted on by Stefaan Verhulst

Article by Karen Zraick: “Organic farmers and environmental groups sued the Agriculture Department on Monday over its scrubbing of references to climate change from its website.

The department had ordered staff to take down pages focused on climate change on Jan. 30, according to the suit, which was filed in the United States District Court for the Southern District of New York. Within hours, it said, information started disappearing.

That included websites containing data sets, interactive tools and funding information that farmers and researchers relied on for planning and adaptation projects, according to the lawsuit.

At the same time, the department also froze funding that had been promised to businesses and nonprofits through conservation and climate programs. The purge then “removed critical information about these programs from the public record, denying farmers access to resources they need to advocate for funds they are owed,” it said.

The Agriculture Department referred questions about the lawsuit to the Justice Department, which did not immediately respond to a request for comment.

The suit was filed by lawyers from Earthjustice, based in San Francisco, and the Knight First Amendment Institute at Columbia University, on behalf of the Northeast Organic Farming Association of New York, based in Binghamton; the Natural Resources Defense Council, based in New York; and the Environmental Working Group, based in Washington. The latter two groups relied on the department website for their research and advocacy, the lawsuit said.

Peter Lehner, a lawyer for Earthjustice, said the pages being purged were crucial for farmers facing risks linked to climate change, including heat waves, droughts, floods, extreme weather and wildfires. The websites had contained information about how to mitigate dangers and adopt new agricultural techniques and strategies. Long-term weather data and trends are valuable in the agriculture industry for planning, research and business strategy.

“You can purge a website of the words climate change, but that doesn’t mean climate change goes away,” Mr. Lehner said…(More)”.

Governing in the Age of AI: Building Britain’s National Data Library

Curated on Posted on by Stefaan Verhulst

Report by the Tony Blair Institute for Global Change: “The United Kingdom should lead the world in artificial-intelligence-driven innovation, research and data-enabled public services. It has the data, the institutions and the expertise to set the global standard. But without the right infrastructure, these advantages are being wasted.

The UK’s data infrastructure, like that of every nation, is built around outdated assumptions about how data create value. It is fragmented and unfit for purpose. Public-sector data are locked in silos, access is slow and inconsistent, and there is no system to connect and use these data effectively, or any framework for deciding what additional data would be most valuable to collect given AI’s capabilities.

As a result, research is stalled, AI adoption is held back, and the government struggles to plan services, target support and respond to emerging challenges. This affects everything from developing new treatments to improving transport, tackling crime and ensuring economic policies help those who need them. While some countries are making progress in treating existing data as strategic assets, none have truly reimagined data infrastructure for an AI-enabled future…(More)”

On the Shoulders of Others: The Importance of Regulatory Learning in the Age of AI

Curated on Posted on by Stefaan Verhulst

Paper by Urs Gasser and Viktor Mayer-Schonberger: “…International harmonization of regulation is the right strategy when the appropriate regulatory ends and means are sufficiently clear to reap efficiencies of scale and scope. When this is not the case, a push for efficiency through uniformity is premature and may lead to a suboptimal regulatory lock-in: the establishment of a rule framework that is either inefficient in the use of its means to reach the intended goal, or furthers the wrong goal, or both.


A century ago, economist Joseph Schumpeter suggested that companies have two distinct strategies to achieve success. The first is to employ economies of scale and scope to lower their cost. It’s essentially a push for improved efficiency. The other strategy is to invent a new product (or production process) that may not, at least initially, be hugely efficient, but is nevertheless advantageous because demand for the new product is price inelastic. For Schumpeter this was the essence of innovation. But, as Schumpeter also argued, innovation is not a simple, linear, and predictable process. Often, it happens in fits and starts, and can’t be easily commandeered or engineered.


As innovation is hard to foresee and plan, the best way to facilitate it is to enable a wide variety of different approaches and solutions. Public policies in many countries to foster startups and entrepreneurship stems from this view. Take, for instance, the policy of regulatory sandboxing, i.e. the idea that for a limited time certain sectors should not or only lightly be regulated…(More)”.

A.I. Is Prompting an Evolution, Not an Extinction, for Coders

Curated on Posted on by Stefaan Verhulst

Article by Steve Lohr: “John Giorgi uses artificial intelligence to make artificial intelligence.

The 29-year-old computer scientist creates software for a health care start-up that records and summarizes patient visits for doctors, freeing them from hours spent typing up clinical notes.

To do so, Mr. Giorgi has his own timesaving helper: an A.I. coding assistant. He taps a few keys and the software tool suggests the rest of the line of code. It can also recommend changes, fetch data, identify bugs and run basic tests. Even though the A.I. makes some mistakes, it saves him up to an hour many days.

“I can’t imagine working without it now,” Mr. Giorgi said.

That sentiment is increasingly common among software developers, who are at the forefront of adopting A.I. agents, assistant programs tailored to help employees do their jobs in fields including customer service and manufacturing. The rapid improvement of the technology has been accompanied by dire warnings that A.I. could soon automate away millions of jobs — and software developers have been singled out as prime targets.

But the outlook for software developers is more likely evolution than extinction, according to experienced software engineers, industry analysts and academics. For decades, better tools have automated some coding tasks, but the demand for software and the people who make it has only increased.

A.I., they say, will accelerate that trend and level up the art and craft of software design.

“The skills software developers need will change significantly, but A.I. will not eliminate the need for them,” said Arnal Dayaratna, an analyst at IDC, a technology research firm. “Not anytime soon anyway.”

The outlook for software engineers offers a window into the impact that generative A.I. — the kind behind chatbots like OpenAI’s ChatGPT — is likely to have on knowledge workers across the economy, from doctors and lawyers to marketing managers and financial analysts. Predictions about the technology’s consequences vary widely, from wiping out whole swaths of the work force to hyper-charging productivity as an elixir for economic growth…(More)”.

Generative AI for data stewards: enhancing accuracy and efficiency in data governance

Curated on Posted on by Stefaan Verhulst

Paper by Ankush Reddy Sugureddy: “The quality of data becomes an essential component for the success of an organisation in a world that is largely influenced by data, where data analytics is becoming increasingly popular in the process of informing strategic decisions. The failure to improve the quality of the data can lead to undesirable outcomes such as poor decisions, ineffective strategies, dysfunctional operations, lost commercial prospects, and abrasion of the consumer. In the process of organisations shifting their focus towards transformative methods such as generative artificial intelligence, several use cases may emerge that have the potential to aid the improvement of data quality. Streamlining procedures such as data classification, metadata management, and policy enforcement can be accomplished by the incorporation of generative artificial intelligence into data governance frameworks. This, in turn, reduces the workload of human data stewards and minimises the possibility of human error. In order to ensure compliance with legal standards such as the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA), generative artificial intelligence may analyse enormous datasets by utilising machine learning algorithms to discover patterns, inconsistencies, and compliance issues…(More)”.

California Governor Launches New Digital Democracy Tool

Curated on Posted on by Stefaan Verhulst

Article by Phil Willon: “California Gov. Gavin Newsom on Sunday announced a new digital democracy initiative that will attempt to connect residents directly with government officials in times of disaster and allow them to express their concerns about matters affecting their day-to-day lives.

The web-based initiative, called Engaged California, will go live with a focus on aiding victims of the deadly wildfires in Pacific Palisades and Altadena who are struggling to recover. For example, comments shared via the online forum could potentially prompt government action regarding insurance coverage, building standards or efforts to require utilities to bury power lines underground.

In a written statement, Newsom described the pilot program as “a town hall for the modern era — where Californians share their perspectives, concerns, and ideas geared toward finding real solutions.”


“We’re starting this effort by more directly involving Californians in the LA firestorm response and recovery,” he added. “As we recover, reimagine, and rebuild Los Angeles, we will do it together.”

The Democrat’s administration has ambitious plans for the effort that go far beyond the wildfires. Engaged California is modeled after a program in Taiwan that became an essential bridge between the public and the government at the height of the COVID-19 pandemic. The Taiwanese government has relied on it to combat online political disinformation as well…(More)”.

The Missing Pieces in India’s AI Puzzle: Talent, Data, and R&D

Curated on Posted on by Stefaan Verhulst

Article by Anirudh Suri: “This paper explores the question of whether India specifically will be able to compete and lead in AI or whether it will remain relegated to a minor role in this global competition. The paper argues that if India is to meet its larger stated ambition of becoming a global leader in AI, it will need to fill significant gaps in at least three areas urgently: talent, data, and research. Putting these three missing pieces in place can help position India extremely well to compete in the global AI race.

India’s national AI mission (NAIM), also known as the IndiaAI Mission, was launched in 2024 and rightly notes that success in the AI race requires multiple pieces of the AI puzzle to be in place.3 Accordingly, it has laid out a plan across seven elements of the “AI stack”: computing/AI infrastructure, data, talent, research and development (R&D), capital, algorithms, and applications.4

However, the focus thus far has practically been on only two elements: ensuring the availability of AI-focused hardware/compute and, to some extent, building Indic language models. India has not paid enough attention to, acted toward, and put significant resources behind three other key enabling elements of AI competitiveness, namely data, talent, and R&D…(More)”.