Article by Aivin Solatorio: “As artificial intelligence (AI) becomes a new gateway to development data, a quiet but significant risk has emerged. Large language models (LLMs) can now summarize reports, answer data queries, and interpret indicators in seconds. But while these tools promise convenience, they also raise a fundamental question: How can we ensure that the numbers they produce remain true to the official data they claim to represent?
AI access does not equal data integrity
Many AI systems today use retrieval-augmented generation (RAG), a technique that feeds models with content from trusted sources or databases. While it is widely viewed as a safeguard against hallucinations, it does not eliminate them. Even when an AI model retrieves the correct data, it may still generate outputs that deviate from it. It might round numbers to sound natural, merge disaggregated values, or restate statistics in ways that subtly alter their meaning. These deviations often go unnoticed because the AI still appears confident and precise to the end user.
Developers often measure such errors through evaluation experiments (or “evals”), reporting aggregate accuracy rates. But those averages mean little to a policymaker, journalist, or citizen interacting with an AI tool. What matters is not whether the model is usually correct, but whether the specific number it just produced is faithful to the official data.
Where Proof-Carrying Numbers come in
Proof-Carrying Numbers (PCN), a novel trust protocol developed by the AI for Data – Data for AI team, addresses this gap. It introduces a mechanism for verifying numerical faithfulness — that is, how closely the AI’s numbers match the trusted data they are based on — in real time.
Here’s how it works:
- The data passed to the LLM must include a claim identifier and a policy that defines acceptable behavior (e.g., exact match required, rounding allowed, etc.).
- The model is instructed to follow the PCN protocol when generating numbers based on that data.
- Each numeric output is checked against the reference data on which it was conditioned.
- If the result satisfies the policy, PCN marks it as verified [✓].
- If it deviates, PCN flags it for review [⚠️].
- Any numbers produced without explicit marks are assumed unverified and should be treated cautiously.
This is a fail-closed mechanism, a built-in safeguard that errs on the side of caution. When faithfulness cannot be proven, PCN does not assume correctness; instead, it makes that failure visible. This feature changes how users interact with AI: instead of trusting responses blindly, they can immediately see whether a number aligns with official data…(More)”.
