Courtney Kennedy and George Elliott Morris at Pew Research: “People polled by telephone are slightly less likely than those interviewed online to say their personal finances are in “poor shape” (14% versus 20%, respectively), a Pew Research Center survey experiment has found.
The experiment, conducted in February and March, is part of a line of research at the Center looking into “mode effects” – in this case, whether findings from self-administered web surveys differ from those of interviewer-administered phone surveys.
In particular, survey researchers have long known that Americans may be more likely to give a “socially desirable” response (and less likely to give a stigmatized or undesirable answer) in an interviewer-administered survey than in one that is self-administered. Mode effects can also result from other differences in survey design, such as seeing the answer choices visually on the web versus hearing them over the phone.
The Center’s experiment randomly assigned respondents to a survey method (online or telephone). Although it found that political questions, such as whether respondents approve of President Donald Trump, don’t elicit significant mode effects, some other, more personal items clearly do. When asked whether or not they had received financial assistance from a family member in the past year, for instance, just 15% of phone respondents say yes. That share is significantly higher (26%) among web respondents….
While the findings from this experiment suggest that self-administered surveys may be more accurate than interviewer-administered approaches as a way to measure financial stress (all else being equal), this does not mean that past telephone-based research arrived at erroneous conclusions regarding financial stress – for example, what predicts it or how the likelihood varies across subgroups. That said, researchers studying financial stress should consider that phone surveys have, at least to some degree, been understating the share of Americans experiencing economic hardship….(More)”.
Note: Survey methodology can be found here, and the topline is available here.