Suchergebnisse

Reliable survey data is needed to be able to infer survey findings to the general population. However, self-selection or panel attrition of the survey respondents may bias survey estimations. To tackle these challenges, weighting adjustments have been established to correct for different inclusion probabilities and to reduce bias in the survey. These strategies adjust the survey data to match known population statistics (e.g., means and proportions). The usefulness of weighting strategies depends on the benchmarks of the variables available from official statistics or other highly reliable sources, for instance, whether population information on the weighting variables is available as joint distributions of all variables or as margins only. While complex weighting strategies have been developed for poststratification using joint distributions (for example multilevel regression and poststratification), these methods are not applicable when only population margins are available. In this paper, we propose two practical approaches that combine the multilevel regression weighting method with weighting algorithms using marginal population distributions only. In a simulation study, we applied both approaches to volunteer samples.

The multilevel and poststratification approach is commonly used to draw valid inference from (non-probabilistic) surveys. This Bayesian approach includes varying regression coefficients for which prior distributions of their variance parameter must be specified. The choice of the distribution is far from being trivial and many contradicting recommendations exist in the literature. The prior choice may be even more challenging when data results from a highly selective inclusion mechanism, such as applied by volunteer panels. We conduct a Monte Carlo simulation study to evaluate the effect of different distribution choices on bias in the estimation of a proportion based on a sample that is subject to a highly selective inclusion mechanism.

The ease at which online paradata can be captured in web surveys seems to increase social researchers' desire to collect such data. Yet little attention is paid to whether respondents actually approve of their collection. This article, therefore, studies online survey respondents' acceptance of automatically collecting their geographical locations. In wave 4 of the German Internet Panel, we asked respondents for their consent to automatically track their location using a JavaScript. Respondents were also asked to report their location in a set of traditional survey questions. About 62 percent of respondents consented to the automated collection of their location whereas 97 percent provided their location manually. With respect to consent biases, we find evidence that the composition of the achieved sample of geo-located respondents is biased and that the personal characteristics associated with respondents' willingness to be geo-located differ between the automated tracking and manual provision of geo-information.

The ease at which online paradata can be captured in web surveys seems to increase social researchers' desire to collect such data. Yet little attention is paid to whether respondents actually approve of their collection. This article, therefore, studies online survey respondents' acceptance of automatically collecting their geographical locations. In wave 4 of the German Internet Panel, we asked respondents for their consent to automatically track their location using a JavaScript. Respondents were also asked to report their location in a set of traditional survey questions. About 62 percent of respondents consented to the automated collection of their location whereas 97 percent provided their location manually. With respect to consent biases, we find evidence that the composition of the achieved sample of geo-located respondents is biased and that the personal characteristics associated with respondents' willingness to be geo-located differ between the automated tracking and manual provision of geo-information.

Previous studies have shown many instances where nonprobability surveys were not as accurate as probability surveys. However, because of their cost advantages, nonprobability surveys are widely used, and there is much debate over the appropriate settings for their use. To contribute to this debate, we evaluate the accuracy of nonprobability surveys by investigating the common claim that estimates of relationships are more robust to sample bias than means or proportions. We compare demographic, attitudinal, and behavioral variables across eight German probability and nonprobability surveys with demographic and political benchmarks from the microcensus and a high-quality, face-to-face survey. In the analyses, we compare three types of statistical inference: univariate estimates, bivariate Pearson's r coefficients, and 24 different multiple regression models. The results indicate that in univariate comparisons, nonprobability surveys were clearly less accurate than probability surveys when compared with the population benchmarks. These differences in accuracy were smaller in the bivariate and the multivariate comparisons across surveys. In addition, the outcome of those comparisons largely depended on the variables included in the estimation. The observed sample differences are remarkable when considering that three nonprobability surveys were drawn from the same online panel. Adjusting the nonprobability surveys somewhat improved their accuracy.

Even though the proportion of individuals who are not equipped to participate in online surveys is constantly decreasing, many surveys face an under-representation of individuals who do not feel IT literate enough to participate. Using experimental data from a probability-based online panel, we study which recruitment survey mode strategy performs best in recruiting less IT-literate persons for an online panel. The sampled individuals received postal invitations to conduct the recruitment survey in a self-completion mode. We experimentally vary four recruitment survey mode strategies: one online mode strategy, two sequential mixed-mode strategies, and one concurrent mode strategy. We find the recruitment survey mode strategies to have a major effect on the sample composition of the recruitment survey, but the differences between the strategies vanish once respondents are asked to proceed with the panel online.

Survey scientists increasingly face the problem of high-dimensionality in their research as digitization makes it much easier to construct high-dimensional (or "big") data sets through tools such as online surveys and mobile applications. Machine learning methods are able to handle such data, and they have been successfully applied to solve predictive problems. However, in many situations, survey statisticians want to learn about causal relationships to draw conclusions and be able to transfer the findings of one survey to another. Standard machine learning methods provide biased estimates of such relationships. We introduce into survey statistics the double machine learning approach, which gives approximately unbiased estimators of parameters of interest, and show how it can be used to analyze survey nonresponse in a high-dimensional panel setting. The double machine learning approach here assumes unconfoundedness of variables as its identification strategy. In high-dimensional settings, where the number of potential confounders to include in the model is too large, the double machine learning approach secures valid inference by selecting the relevant confounding variables.

Survey scientists increasingly face the problem of high-dimensionality in their research as digitization makes it much easier to construct high-dimensional (or "big") data sets through tools such as online surveys and mobile applications. Machine learning methods are able to handle such data, and they have been successfully applied to solve predictive problems. However, in many situations, survey statisticians want to learn about causal relationships to draw conclusions and be able to transfer the findings of one survey to another. Standard machine learning methods provide biased estimates of such relationships. We introduce into survey statistics the double machine learning approach, which gives approximately unbiased estimators of parameters of interest, and show how it can be used to analyze survey nonresponse in a high-dimensional panel setting. The double machine learning approach here assumes unconfoundedness of variables as its identification strategy. In high-dimensional settings, where the number of potential confounders to include in the model is too large, the double machine learning approach secures valid inference by selecting the relevant confounding variables.

The literature on the effects of incentives in survey research is vast and covers a diversity of survey modes. The mode of probability-based online panels, however, is still young and so is research into how to best recruit sample units into the panel. This paper sheds light on the effectiveness of a specific type of incentive in this context: a monetary incentive that is paid conditionally upon panel registration within two weeks of receiving the initial postal mail invitation. We tested early bird cash incentives in a large-scale recruitment experiment for the German Internet Panel (GIP) in 2018. We find that panel response rates are significantly higher when offering early bird cash incentives and that fieldwork progresses considerably faster, leading to fewer reminders and greater cost-effectiveness. Furthermore, sample representativeness is similarly high with or without early bird incentives.

The literature on the effects of incentives in survey research is vast and covers a diversity of survey modes. The mode of probability-based online panels, however, is still young and so is research into how to best recruit sample units into the panel. This paper sheds light on the effectiveness of a specific type of incentive in this context: a monetary incentive that is paid conditionally upon panel registration within two weeks of receiving the initial postal mail invitation. We tested early bird cash incentives in a large-scale recruitment experiment for the German Internet Panel (GIP) in 2018. We find that panel response rates are significantly higher when offering early bird cash incentives and that fieldwork progresses considerably faster, leading to fewer reminders and greater cost-effectiveness. Furthermore, sample representativeness is similarly high with or without early bird incentives.

Once recruited, probability-based online panels have proven to enable high-quality and high-frequency data collection. In ever faster-paced societies and, recently, in times of pandemic lockdowns, such online survey infrastructures are invaluable to social research. In absence of email sampling frames, one way of recruiting such a panel is via postal mail. However, few studies have examined how to best approach and then transition sample members from the initial postal mail contact to the online panel registration. To fill this gap, we implemented a large-scale experiment in the recruitment of the 2018 sample of the German Internet Panel (GIP) varying panel recruitment designs in four experimental conditions: online-only, concurrent mode, online-first, and paper-first. Our results show that the online-only design delivers higher online panel registration rates than the other recruitment designs. In addition, all experimental conditions led to similarly representative samples on key socio-demographic characteristics.

Before conducting a survey, researchers frequently ask themselves how large the resulting sample of respondents needs to be to answer their research questions. In this guideline, we discuss how sample size calculation is affected by the sampling design. We give practical advice on how to conduct sample size calculation for complex samples.

Once recruited, probability-based online panels have proven to enable high-quality and high-frequency data collection. In ever faster-paced societies and, recently, in times of pandemic lockdowns, such online survey infrastructures are invaluable to social research. In absence of email sampling frames, one way of recruiting such a panel is via postal mail. However, few studies have examined how to best approach and then transition sample members from the initial postal mail contact to the online panel registration. To fill this gap, we implemented a large-scale experiment in the recruitment of the 2018 sample of the German Internet Panel (GIP) varying panel recruitment designs in four experimental conditions: online-only, concurrent mode, online-first, and paper-first. Our results show that the online-only design delivers higher online panel registration rates than the other recruitment designs. In addition, all experimental conditions led to similarly representative samples on key socio-demographic characteristics.

In recent years, an increasing number of online panel participants respond to surveys on smartphones. As a result, survey practitioners are faced with a difficult decision: Either they hold the questionnaire design constant over time and thus stay with the original desktop-optimized design; or they switch to a smartphone-optimized format and thus accommodate respondents who prefer participating on their smartphone. Even though this decision is all but trivial, little research thus far has been conducted on the effect of such an adjustment on panel members' survey participation and device use. We report on the switch to a smartphone-optimized design in the German Internet Panel (GIP), an ongoing probability-based online panel that started in 2012 with a desktop-optimized design. We investigate whether the introduction of a smartphone-optimized design affected overall response rates and smartphone use in the GIP. Moreover, we examine the effect of different ways of announcing the introduction of the smartphone-optimized design in the invitation email on survey participation using a smartphone.

Digital trace data are increasingly used in the social sciences. Given the risks associated with data access via application programming interfaces (APIs) as well as ethical discussions around the use of such data, data donations have been proposed as a methodologically reliable and ethically sound way of collecting digital trace data. While data donations have many advantages, study participants may be reluctant to share their data, for example, due to privacy concerns. To assess which factors in a data donation request are relevant for participants' acceptance and decisions, we conducted a vignette experiment investigating the general acceptability and personal willingness to donate various data types (i.e., data from GPS, web browsing, LinkedIn/Xing, Facebook, and TikTok) for research purposes. The preregistered study was implemented in the probability-based German Internet Panel (GIP) and gathered responses from n = 3821 participants. Results show that people rate the general acceptability of data donation requests higher than their own willingness to donate data. Regarding the different data types, respondents indicated that they would be more willing to donate their LinkedIn/Xing, TikTok, and GPS data compared to web browsing and Facebook data. In contrast, information about whether the donated data would be shared with other researchers and data security did not affect the responses to the respective donation scenarios. Based on these results, we discuss implications for studies employing data donations.