Presidential Address. Far From Ordinary Questions: Task Difficulty, Motivation, and Measurement Practice

In: The public opinion quarterly: POQ, Band 84, Heft 3, S. 798-812

Abstract

As Howard Schuman noted in his presidential address in 1986, survey respondents expect to be asked "ordinary questions," but our questions often make extraordinary demands. The impact of the difficult tasks we give to respondents on their motivation to provide complete and accurate information is an aspect of the infrastructure of measurement in surveys that has received little attention. When questions are particularly difficult for some groups in a target population, the comparability of measurement for all units in a population is reduced. For measurement to be comparable across all groups in a population, questions must be equally accessible—and equally motivating—to all. Some challenges to improving the practice of measurement in surveys arise because—under the constraints of limited time, budget, and staff—we borrow questions, give them authority based on their source, assert that we need to replicate them to make comparisons, and fail to fully examine claims that they have been validated. Although the constraints are real, these practices limit our ability to take full advantage of advances in research about instrument design to improve the quality of measurement in surveys.