Article by David S. Johnson, Maggie Meinhardt, and John Sabelhaus: “For many household surveys in the United States, response rates have been steadily declining for at least the past two decades.” This is a quote from a National Academies of Sciences report from 2013. It is still true today, and it is true for all wealthy countries. Suffering from low response rates and increasing costs, surveys are often described as 20th century technology that needs to be replaced.
But surveys capture things we cannot get from administrative data, as Census Bureau Deputy Director Ron Jarmin noted at a recent event. While administrative data could provide a person’s employment and earnings, only surveys can determine (for example) whether someone was looking for work, which is key for measuring the unemployment rate.
Declining survey participation, both in the U.S. and abroad, is often raised as a large challenge for the statistical system and cited as a reason to eliminate surveys in favor of other measurement strategies. But rather than discard this important data source, researchers should seek to understand how response rates impact the statistics we care about and why response rates are falling in the first place.
The key issues for how survey participation impacts economic statistics is whether lower response rates lead to less statistical precision and whether they actually create statistical bias. Lower response rates lead to lower sample sizes and thus less precision, but the statistics may well remain unbiased so long as differences in survey participation are not correlated with the economic outcome we are measuring. Statistical bias is a larger concern because policymakers would be reading economic signposts that are literally pointing in the wrong direction.
There are many plausible reasons why survey response rates are declining. Among these are the difficulty of contacting the individuals who are (randomly) chosen for the survey sample, respondent concerns about the time burden of completing a survey, and respondent fears about the privacy of their personal data. These difficulties are not unique to government economic surveys, and although the challenges may be getting worse, the unique role of economic surveys means we need to move forward using tried and true methods for improving survey participation…(More)”.
