Responsible AI and Algorithmic Adoption in Methodology Development for National Statistical Offices

Paper by Siu-Ming Tam: “To meet growing demand for granular demographic and socioeconomic indicators under tighter budgets, national statistical offices must continually develop new methods. These include using big data, satellite imagery, and transactional sources to improve or redesign data collection. Artificial intelligence can support this work, but algorithms generated with AI should not be trusted for production without rigorous verification.
This paper focuses on two foundations of trust in the use of AI in official statistics: independent statistical verification before production use, and disciplined protection of respondent confidentiality during development and testing. The approach is illustrated through the author’s experience directing AI to construct and implement a Mini Max Hierarchical Bayes sampling algorithm. Applied to a synthetic labour force population, the method met all specified precision targets while reducing the required sample size by 80 percent, as confirmed by a Monte Carlo study with 1000 replications. Applied to 2021 Australian Census microdata, it achieved a 90 percent reduction while producing national point estimates accurate to well below 1 percent.
The paper concludes with a practical evaluation checklist aligned with the UN Fundamental Principles of Official Statistics and the HLG MOS Quality Framework for Statistical Algorithms…(More)”.