Suchergebnisse

This paper examines the effects of survey mode on patterns of survey response, paying special attention to the conditions under which mode effects are more or less consequential. We use the Youth Participatory Politics survey, a study administered either online or over the phone to 2920 young people. Our results provide consistent evidence of mode effects. The internet sample exhibits higher rates of item non-response and "no opinion" responses, and considerably lower levels of differentiation in the use of rating scales. These differences remain even after accounting for how respondents selected into the mode of survey administration. We demonstrate the substantive implications of mode effects in the context of items measuring political knowledge and racial attitudes. We conclude by discussing the implications of our results for comparing data obtained from surveys conducted with different modes, and for the design and analysis of multi-mode surveys.

Longitudinal surveys traditionally conducted by interviewers are facing increasing pressures to explore alternatives such as sequential mixed-mode designs, which start with a cheaper self-administered mode (online) then follow up using more expensive methods such as telephone or face-to-face interviewing. Using a designed experiment conducted as part of the 2018 wave of the Health and Retirement Study (HRS) in the US, we compare a sequential mixed-mode design (web then telephone) with the standard telephone-only protocol. Using an intent-to-treat analysis, we focus on response quality and response distributions for several domains key to HRS: physical and psychological health, financial status, expectations and family composition.
Respondents assigned to the sequential mixed-mode (web) had slightly higher missing data rates and more focal responses than those assigned to telephone-only. However, we find no evidence of differential quality in verifying and updating roster information. We find slightly lower rates of asset ownership reported by those assigned to the web mode. Conditional on ownership, we find no detectable mode effects on the value of assets. We find more negative (pessimistic) expectations for those assigned to the web mode. We find little evidence of poorer health reported by those assigned to the web mode.
We find that effects of mode assignment on measurement are present, but for most indicators the effects are small. Finding ways to remediate the differences in item-missing data and focal values should help reduce mode effects in mixed-mode surveys or those transitioning from interviewer- to self-administration.

"Landline surveys suffer from an increasing risk of excluding a relevant share of the population.
To analyze and correct telephone coverage issues, face-to-face surveys are often used, which contain questions about landline ownership and registration. Others use dual frame approaches and compare results from the landline with another mode. However, such surveys lack information about unobserved sample members. In this article we analyze representation bias using a household survey with a sample drawn from a population register, where landline is used for households with a matched landline, and face-to-face for those without. We distinguish between the different components of nonobservation, including landline undercoverage, non-contact, and non-cooperation, by either incorporating face-to-face sample members or not, and by the fieldwork phases to recruit households and individuals. Our main interest is how biases from each of these components add up to a final representation bias in the responding sample. In
addition, we analyze income and deprivation differences by either including face-to-face
sample members or not. The strongest representation bias in the telephone sample on the household level is caused by telephone undercoverage. The combined sample suffers much less from representation bias, which mostly stems from noncooperation. In terms of income and deprivation differences, our results show that the face-to-face sample is poorer than the telephone
sample and needs to be considered for unbiased estimates. Based on these findings we offer some fieldwork recommendations to help reduce selection bias based on the different reasons for nonobservation." (author's abstract)

Celem artykułu jest przedstawienie obecnego stanu wiedzy na temat możliwości i celowości wykorzystania techniki ankiety internetowej w przeprowadzaniu badań sondażowych w schemacie mixed-mode. Ze względu na rosnące trudności w realizacji sondaży w sposób tradycyjny konieczne jest poszukiwanie nowych rozwiązań. Artykuł przedstawia główne trudności związane z zastosowaniem techniki internetowej, oraz opisane w literaturze skuteczne sposoby ich minimalizacji w niektórych obszarach, przede wszystkim poprzez mieszanie technik na wszystkich etapach badania. Wskazuje także obszary, w których obecny stan wiedzy jest niewystarczający i konieczne jest przeprowadzenie dalszych badań eksperymentalnych.

This article reports research exploring the benefits of adding text messages to the contact strategy in the context of a sequential mixed-mode design where telephone interviewer administration follows a web phase. In a web-first mixed-mode survey, supplementing the contact strategy with text messages can help increase the response rate at the web phase and, consequently, reduce fieldwork efforts at the interviewer-administered phase. We present results from a survey experiment embedded in wave 11 of Understanding Society in which the usual contact strategy of emails and letters was supplemented with text messages. Effects of the text messages on survey response and fieldwork efforts were assessed. In addition, we also investigated the impact of SMS on the device selected to complete the survey, time to response, and sample balance. The results show a weak effect of the SMS reminders on response during the web fieldwork. However, this positive effect did not significantly reduce fieldwork effort.

Early promise for the Internet as a tool to make social research questionnaires easier and cheaper to deliver is not fully realized. This study reports a mixed-mode survey of 1,100 Australian researchers. When respondents were given the choice to complete either web- or paper-based versions of a questionnaire, the majority chose the paper-based mode. Web respondents were more likely to be young, male, middle ranking, and working in information technology—related sectors. The authors highlight the need to determine how far alternate delivery modes increase response rates. For mixed-mode surveys to be financially and methodologically worthwhile, the authors propose that the initial sample size be at least 1,000 individuals, this figure depending on the demographic characteristics of the sample.

Abstract
This article presents the results of an adaptive design experiment in the recruitment of households and individuals for a two-stage national probability web or mail mixed-mode survey, the American Family Health Study (AFHS). In the screening stage, we based the adaptive design's subgroup differentiation on Esri Tapestry segmentation. We used tailored invitation materials for a subsample where a high proportion of the population was Hispanic and added a paper questionnaire to the initial mailing for a subsample with rural and older families. In the main-survey stage, the adaptive design targeted the households where a member other than the screening respondent was selected for the survey. The adaptations included emailing and/or texting, an additional prepaid incentive, and seeking screening respondents' help to remind the selected individuals. The main research questions are (i) whether the adaptive design improved survey production outcomes and (ii) whether combining adaptive design and postsurvey weighting adjustments improved survey estimates compared to performing postsurvey adjustments alone. Unfortunately, the adaptive designs did not improve the survey production outcomes. We found that the weighted AFHS estimates closely resemble those of a benchmark national face-to-face survey, the National Survey of Family Growth, although the adaptive design did not additionally change survey estimates beyond the weighting adjustments. Nonetheless, our experiment yields useful insights about the implementation of adaptive design in a self-administered mail-recruit web or mail survey. We were able to identify subgroups with potentially lower response rates and distinctive characteristics, but it was challenging to develop effective protocol adaptations for these subgroups under the constraints of the two primary survey modes and the operational budget of the AFHS. In addition, for self-administered within-household selection, it was difficult to obtain contact information from, reach, and recruit selected household members that did not respond to the screening interview.

Linking survey responses with administrative data is a promising practice to increase the range of research questions to be explored, at a limited interview burden, both for respondents and interviewers. We describe the protocol for asking consent to data linkage on nine different sources in a large-scale nationally representative longitudinal survey of young adults in England: the Next Steps Age 25 Survey. We present empirical evidence on consent to data linkage from qualitative interviews, a pilot study, and the mainstage survey. To the best of our knowledge, this is the first study that discusses the practicalities of implementing a data linkage protocol asking consent both retrospectively and prospectively, on multiple domains, and in the context of a mixed-mode survey.

Mixed-mode surveys allow researchers to combine the advantages of multiple modes, for example, the low cost of the web mode with the higher coverage of offline modes. One drawback of combining modes is that there might be systematic differences in measurement across modes. Thus, it would be useful to know which measurement methods work best in all employed modes. This study sets out to find a method that results in the highest measurement quality across self-administered web mode questionnaires (web mode) and self-administered paper questionnaires sent out by mail (mail mode). Two Multitrait-Multimethod (MTMM) experiments employing questions on environmental attitudes and supernatural beliefs were implemented in the GESIS Panel, a probability-based panel in Germany. The experiments were designed to estimate the measurement quality of three different response scales: A seven-point fully labelled scale, a 101-point numerical openended scale and an eleven-point partially labelled scale. Our results show that the elevenpoint partially labelled scale consistently leads to the highest measurement quality across both modes. We thus recommend using eleven-point partially labelled scales when measuring attitudes or beliefs in mixed-mode surveys combining web and mail mode.

Mixed-mode surveys need to determine a number of design parameters that may have a strong influence on costs and errors. In a sequential mixed-mode design with web followed by telephone, one of these decisions is when to switch modes. The web mode is relatively inexpensive but produces lower response rates. The telephone mode complements the web mode in that it is relatively expensive but produces higher response rates. Among the potential negative consequences, delaying the switch from web to telephone may lead to lower response rates if the effectiveness of the prenotification contact materials is reduced by longer time lags, or if the additional e-mail reminders to complete the web survey annoy the sampled person. On the positive side, delaying the switch may decrease the costs of the survey. We evaluate these costs and errors by experimentally testing four different timings (1, 2, 3, or 4 weeks) for the mode switch in a web–telephone survey. This experiment was conducted on the fourth wave of a longitudinal study of the mental health of soldiers in the U.S. Army. We find that the different timings of the switch in the range of 1–4 weeks do not produce differences in final response rates or key estimates but longer delays before switching do lead to lower costs.

"Survey researchers increasingly use mixed-mode surveys for general population data collection because mixed-mode surveys are argued to provide lower selection error at constant budgets or lower variable costs at constant selection error. Nevertheless, the advantage of lower selection error and variable costs might be counteracted by higher measurement error and by higher fixed costs. This trade-off between selection error, measurement error, variable costs and fixed costs has hardly been studied within the existing literature about mixed-mode surveys. This paper discusses a possible procedure for evaluating this trade-off by comparing the performance (mean squared error) of mixed-mode survey designs against single-mode survey designs. The procedure is further illustrated by real example data stemming from a mixed-mode mailface-to-face survey. This illustration yields smaller errors for single-mode designs under low budgets but smaller errors for mixed-mode designs under large budgets or, alternatively, a budgetary advantage of single-mode designs when the allowed error is relatively high but a budgetary advantage of mixed-mode designs when the allowed error is relatively small. However, the validity of these results depend on several modelling assumptions which may be topics for future research." (publisher's description)

Abstract
The use of mixed-mode surveys has grown rapidly in recent years, due to both technological advances and the COVID-19 pandemic. The increased mixing of modes (and the adoption of newer digital modes like web and text messaging) necessitates an evaluation of the impact of these newer designs on survey errors and costs, as well as new techniques for disaggregating and adjusting for nonresponse and measurement errors. This special issue highlights recent innovations, applications, and evaluations of mixed-mode survey designs and identifies areas where additional research is required.

Implementing innovations in surveys often results in uncertainty concerning how different design decisions will affect key performance indicators such as response rates, nonresponse bias, or survey costs. Thus, responsive survey designs have been developed to better cope with such situations. In the present study, we propose a responsive survey design that relies on experimentation in the earlier phases of the survey to decide between different design choices of which - prior to data collection - their impact on performance indicators is uncertain. We applied this design to the European Values Study 2017/2018 in Germany that advanced its general social survey-type design away from the traditional face-to-face mode to self-administered modes. These design changes resulted in uncertainty as to how different incentive strategies and mode choice sequences would affect response rates, nonresponse bias, and survey costs. We illustrate the application and operation of the proposed responsive survey design, as well as an efficiency issue that accompanies it. We also compare the performance of the responsive survey design to a traditional survey design that would have kept all design characteristics static during the field period.