Suchergebnisse

Relatively little attention has been paid to the effects of nonparticipation on data quality in population-based studies that use accelerometers to measure physical activity. We examine these issues using data from the 2013 Longitudinal Internet Studies for the Social Sciences (LISS) panel and 2013–2014 National Health and Nutrition Examination Survey (NHANES) accelerometer studies, both of which collected survey data in advance and therefore permit comparisons of self-reported physical activity between participants and nonparticipants to the accelerometer studies. While individuals with high levels of self-reported physical activity are overrepresented in the participant samples, the differences are modest in both studies. However, in the LISS panel this difference led to overestimates of physical activity that are not fully corrected by propensity score weighting adjustments (i.e., non-ignorable selection bias). This finding underscores the importance of assessing the potential influence of nonparticipation on accelerometer-derived estimates of physical activity.

Abstract
Relatively little is known about the performance of open questions in self-administered questionnaires compared to interviewer-administered ones. We examined this issue using the responses to the 2016 American National Election Study's Web self-administered questionnaires and face-to-face computer-assisted personal interviews (CAPI) from independently drawn samples that asked three sets of open questions: most important problem(s) facing the country; candidate for and against reasons; and political party likes and dislikes. Although Web respondents gave shorter answers with fewer codeable ideas than face-to-face respondents, the two modes produced similar distributions of substantive answers. Despite our expectation that open answers to interviewers would be more civil than those expressed on the Web, there was no difference in civility between the modes. However, the construct validity of the four candidate for and against items (predicting vote choice of Clinton vs. Trump) was higher on the Web than in the interviewer-administered CAPI. These findings call into question the conventional wisdom that open questions are not well suited for self-administration and suggest that researchers should consider more frequent use of open questions in Web surveys.

This article compares the factors affecting completion times (CTs) to web survey questions when they are answered using two different devices: personal computers (PCs) and smartphones. Several studies have reported longer CTs when respondents use smartphones than PCs. This is a concern to survey researchers because longer CTs may increase respondent burden and the risk of breakoff. However, few studies have analyzed the specific reasons for the time difference. In this analysis, we analyzed timing data from 836 respondents who completed the same web survey twice, once using a smartphone and once using PC, as part of a randomized crossover experiment in the Longitudinal Internet Studies for the Social Sciences panel. The survey contained a mix of questions (single choice, numeric entry, and text entry) that were displayed on separate pages. We included both page-level and respondent-level factors that may have contributed to the time difference between devices in cross-classified multilevel models. We found that respondents took about 1.4 times longer when using smartphones than PCs. This difference was larger when a page had more than one question or required text entry. The difference was also larger among respondents who had relatively low levels of familiarity and experience using smartphones. Respondent multitasking was associated with slower CTs, regardless of the device used. Practical implications and avenues for future research are discussed.

AbstractSurvey data producers increasingly provide estimates of nonresponse bias in several variables when they release or analyze data. Researchers understand that sample estimates of population values should be reported with appropriate measures of uncertainty, such as standard errors or confidence intervals. However, few studies acknowledge that nonresponse bias estimates vary across samples. Using simulations, we study the precision of estimates of nonresponse bias in means and how that precision is affected by features such as clustering and response rates. Results show that low response rates and clustering increase the variability of bias estimates. We then evaluate three methods to estimate the sampling variance of nonresponse bias in means: a method developed by Lee, jackknife replication, and linearization. We find that the Lee approach works well for simple random samples but overestimates variability for clustered samples. Linearization and replication work well with all populations studied, and we give an algorithm for the implementation of these approaches. We also apply the three methods to the LISS (Longitudinal Internet Studies for the Social Sciences) panel and the General Social Survey, providing practical confirmation of the simulation results.

Design principles for survey questionnaires viewed on desktop and laptop computers are increasingly being seen as inadequate for the design of questionnaires viewed on smartphones. Insights gained from empirical research can help those conducting mobile surveys to improve their questionnaires. This article reports on a systematic literature review of research presented or published between 2007 and 2016 that evaluated the effect of smartphone questionnaire design features on indicators of response quality. The evidence suggests that survey designers should make efforts to "optimize" their questionnaires to make them easier to complete on smartphones, fit question content to the width of smartphone screens to prevent horizontal scrolling, and choose simpler types of questions (single-choice questions, multiple-choice questions, text-entry boxes) over more complicated types of questions (large grids, drop boxes, slider questions). Based on these results, we identify design heuristics, or general principles, for creating effective smartphone questionnaires. We distinguish between five of them: readability, ease of selection, visibility across the page, simplicity of design elements, and predictability across devices. They provide an initial framework by which to evaluate smartphone questionnaires, though empirical testing and further refinement of the heuristics is necessary.

Survey researchers often assume that "professional" respondents, those who complete a large number of surveys in opt-in online panels, are more likely than others to provide low-quality responses because their primary motivation is to earn rewards with minimal effort. However, there is little empirical evidence for this assumption. It could also be that professional respondents are willing to expend effort in order to be compensated. We investigated this issue using data from four independent surveys of opt-in panelists with about 2,400 respondents in each survey. We classified panelists into three groups—"professional", "average," and "novice"—according to the number of surveys they had previously completed and the number of panels they belonged to. We then compared the groups with respect to their demographic characteristics, reasons for joining a panel, and response behaviors. Professional respondents were the oldest group and, as expected, the most likely to report "for money" as the main reason for joining a panel. However, novices were actually the most likely to provide low-quality responses. Professional respondents appear to take the task of completing surveys more seriously than previously thought.

The rising penetration of smartphones now gives researchers the chance to collect data from smartphone users through passive mobile data collection via apps. Examples of passively collected data include geolocation, physical movements, online behavior and browser history, and app usage. However, to passively collect data from smartphones, participants need to agree to download a research app to their smartphone. This leads to concerns about nonconsent and nonparticipation. In the current study, we assess the circumstances under which smartphone users are willing to participate in passive mobile data collection. We surveyed 1,947 members of a German nonprobability online panel who own a smartphone using vignettes that described hypothetical studies where data are automatically collected by a research app on a participant's smartphone. The vignettes varied the levels of several dimensions of the hypothetical study, and respondents were asked to rate their willingness to participate in such a study. Willingness to participate in passive mobile data collection is strongly influenced by the incentive promised for study participation but also by other study characteristics (sponsor, duration of data collection period, option to switch off the app) as well as respondent characteristics (privacy and security concerns, smartphone experience).