Suchergebnisse

The use of randomised controlled trials (RCTs), most commonly a medical sciences research tool, is a hotly debated topic in Education. This book examines the controversial aspects of RCTs in Education and sets out the potential and pitfalls of the method. Drawing on their own extensive experience of running RCTs, the authors provide a thorough practical introduction to the use of randomised controlled trials in education. Using real data sets, chapters equip the reader with all of the key knowledge and skills required to design, run, analyse and report an RCT. Coverage includes: . Step-by-step guidance on analysing data . How to assess the reliability and validity of results . Advice on balancing the demands of various stakeholders Essential reading for postgraduate and more experienced researchers, as well as teachers and educationalists seeking to increase their knowledge and understanding of the use of such methods in education.

Randomized controlled trials (RCTs) are designed to answer causal questions with internal validity. However, threats to internal validity exist for even well-designed RCTs. In this article, we focus on how preregistration can help address some specific threats to internal validity related to the reporting of results. Preregistration involves researchers publicly posting critical decision points in a study prior to conducting it for the purpose of making researcher plans transparent, making deviations from those plans discoverable, and improving the validity of tests of significance. We provide a brief overview of null-hypothesis significance testing; consider how questionable research practices (e.g., p-hacking) and conducting data-dependent analysis threaten the validity of significance tests; discuss how preregistration can help address these threats and how preregistration works for RCTs; note limitations and challenges to preregistration; and provide recommendations for increasing the use of preregistration by researchers conducting RCTs in social work, education, and related fields.

Background: The Scandinavian countries have a long history of implementing social interventions, but the interventions have not been examined using randomised controlled trials until relatively recently compared with countries like the United States and the United Kingdom. Purpose: The purpose of this paper is to examine the history of randomised controlled trials in Scandinavian compulsory schools (grades 0–10; pupil ages 6–15). Specifically, we investigate drivers and barriers for randomised controlled trials in educational research and the differences between the three Scandinavian countries Denmark, Norway, and Sweden. Methods: To locate relevant trials, we performed a systematic search of four bibliographic databases and a search for grey literature. Results were combined with trials located through direct contact with researchers and government officials. A trial was included if one or more interventions were randomly assigned to groups of students and carried out in a school setting with the primary aim of improving the academic performance of children aged 6–15 in grades 0–10 in Denmark, Norway, or Sweden. We included both conducted and ongoing trials. Publications that seemed relevant were screened based on full-text versions. Data extraction included information from the included studies on grade level, study period, sample size (N), project owner, funding source, and theme. In addition, we conducted two semi-structured interviews by phone or in person with central employees in funding agencies and ministries and 25 correspondences with researchers and policymakers. Findings and conclusion: RCTs in grades 0–10 were few in all of Scandinavia until about 2011, after which there was an increase in all three countries, although at different rates. The largest number of trials has been conducted in Denmark, and the increase is more marked in Denmark and Norway compared with Sweden. International trends towards more impact evaluations and results from international comparisons such as PISA have likely affected the development in all countries, but while many trials in Denmark and Norway are the result of policy initiatives, only one such example in Sweden was identified. We believe the lack of government initiatives to promote RCTs in Sweden is the most likely explanation for the differences across the Scandinavian countries. Funding and coordination from the government are often crucial for the implementation of RCTs and are likely more important in smaller countries such as the Scandinavian ones. Supporting institutions have now been established in all three countries, and we believe that the use of RCTs in Scandinavian educational research is likely to continue. ; publishedVersion

Background: The Scandinavian countries have a long history of implementing social interventions, but the interventions have not been examined using randomised controlled trials until relatively recently compared with countries like the United States and the United Kingdom. Purpose: The purpose of this paper is to examine the history of randomised controlled trials in Scandinavian compulsory schools (grades 0-10; pupil ages 6-15). Specifically, we investigate drivers and barriers for randomised controlled trials in educational research and the differences between the three Scandinavian countries Denmark, Norway, and Sweden. Methods: To locate relevant trials, we performed a systematic search of four bibliographic databases and a search for grey literature. Results were combined with trials located through direct contact with researchers and government officials. A trial was included if one or more interventions were randomly assigned to groups of students and carried out in a school setting with the primary aim of improving the academic performance of children aged 6-15 in grades 0-10 in Denmark, Norway, or Sweden. We included both conducted and ongoing trials. Publications that seemed relevant were screened based on full-text versions. Data extraction included information from the included studies on grade level, study period, sample size (N), project owner, funding source, and theme. In addition, we conducted two semi-structured interviews by phone or in person with central employees in funding agencies and ministries and 25 correspondences with researchers and policymakers. Findings and conclusion: RCTs in grades 0-10 were few in all of Scandinavia until about 2011, after which there was an increase in all three countries, although at different rates. The largest number of trials has been conducted in Denmark, and the increase is more marked in Denmark and Norway compared with Sweden. International trends towards more impact evaluations and results from international comparisons such as PISA have likely affected the development in all countries, but while many trials in Denmark and Norway are the result of policy initiatives, only one such example in Sweden was identified. We believe the lack of government initiatives to promote RCTs in Sweden is the most likely explanation for the differences across the Scandinavian countries. Funding and coordination from the government are often crucial for the implementation of RCTs and are likely more important in smaller countries such as the Scandinavian ones. Supporting institutions have now been established in all three countries, and we believe that the use of RCTs in Scandinavian educational research is likely to continue.

Background: The Scandinavian countries have a long history of implementing social interventions, but the interventions have not been examined using randomised controlled trials until relatively recently compared with countries like the United States and the United Kingdom. Purpose: The purpose of this paper is to examine the history of randomised controlled trials in Scandinavian compulsory schools (grades 0–10; pupil ages 6–15). Specifically, we investigate drivers and barriers for randomised controlled trials in educational research and the differences between the three Scandinavian countries Denmark, Norway, and Sweden. Methods: To locate relevant trials, we performed a systematic search of four bibliographic databases and a search for grey literature. Results were combined with trials located through direct contact with researchers and government officials. A trial was included if one or more interventions were randomly assigned to groups of students and carried out in a school setting with the primary aim of improving the academic performance of children aged 6–15 in grades 0–10 in Denmark, Norway, or Sweden. We included both conducted and ongoing trials. Publications that seemed relevant were screened based on full-text versions. Data extraction included information from the included studies on grade level, study period, sample size (N), project owner, funding source, and theme. In addition, we conducted two semi-structured interviews by phone or in person with central employees in funding agencies and ministries and 25 correspondences with researchers and policymakers. Findings and conclusion: RCTs in grades 0–10 were few in all of Scandinavia until about 2011, after which there was an increase in all three countries, although at different rates. The largest number of trials has been conducted in Denmark, and the increase is more marked in Denmark and Norway compared with Sweden. International trends towards more impact evaluations and results from international comparisons such as PISA have likely affected the development in all countries, but while many trials in Denmark and Norway are the result of policy initiatives, only one such example in Sweden was identified. We believe the lack of government initiatives to promote RCTs in Sweden is the most likely explanation for the differences across the Scandinavian countries. Funding and coordination from the government are often crucial for the implementation of RCTs and are likely more important in smaller countries such as the Scandinavian ones. Supporting institutions have now been established in all three countries, and we believe that the use of RCTs in Scandinavian educational research is likely to continue. ; publishedVersion

Background: The Scandinavian countries have a long history of implementing social interventions, but the interventions have not been examined using randomised controlled trials until relatively recently compared with countries like the United States and the United Kingdom. Purpose: The purpose of this paper is to examine the history of randomised controlled trials in Scandinavian compulsory schools (grades 0–10; pupil ages 6-15). Specifically, we investigate drivers and barriers for randomised controlled trials in educational research and the differences between the three Scandinavian countries Denmark, Norway, and Sweden. Methods: To locate relevant trials, we performed a systematic search of four bibliographic databases and a search for grey literature. Results were combined with trials located through direct contact with researchers and government officials. A trial was included if one or more interventions were randomly assigned to groups of students and carried out in a school setting with the primary aim of improving the academic performance of children aged 6-15 in grades 0–10 in Denmark, Norway, or Sweden. We included both conducted and ongoing trials. Publications that seemed relevant were screened based on full-text versions. Data extraction included information from the included studies on grade level, study period, sample size (N), project owner, funding source, and theme. In addition, we conducted two semi-structured interviews by phone or in person with central employees in funding agencies and ministries and 25 correspondences with researchers and policy makers. Findings and conclusion: RCTs in grades 0–10 were few in all of Scandinavia until about 2011, after which there was an increase in all three countries, although at different rates. The largest number of trials has been conducted in Denmark, and the increase is more marked in Denmark and Norway compared with Sweden. International trends towards more impact evaluations and results from international comparisons such as PISA have likely affected the development in all countries, but while many trials in Denmark and Norway are the result of policy initiatives, only one such example in Sweden was identified. We believe the lack of government initiatives to promote RCTs in Sweden is the most likely explanation for the differences across the Scandinavian countries. Funding and coordination from the government are often crucial for the implementation of RCTs and are likely more important in smaller countries such as the Scandinavian ones. Supporting institutions have now been established in all three countries, and we believe that the use of RCTs in Scandinavian educational research is likely to continue. ; Background: The Scandinavian countries have a long history of implementing social interventions, but the interventions have not been examined using randomised controlled trials until relatively recently compared with countries like the United States and the United Kingdom. Purpose: The purpose of this paper is to examine the history of randomised controlled trials in Scandinavian compulsory schools (grades 0–10; pupil ages 6-15). Specifically, we investigate drivers and barriers for randomised controlled trials in educational research and the differences between the three Scandinavian countries Denmark, Norway, and Sweden. Methods: To locate relevant trials, we performed a systematic search of four bibliographic databases and a search for grey literature. Results were combined with trials located through direct contact with researchers and government officials. A trial was included if one or more interventions were randomly assigned to groups of students and carried out in a school setting with the primary aim of improving the academic performance of children aged 6-15 in grades 0–10 in Denmark, Norway, or Sweden. We included both conducted and ongoing trials. Publications that seemed relevant were screened based on full-text versions. Data extraction included information from the included studies on grade level, study period, sample size (N), project owner, funding source, and theme. In addition, we conducted two semi-structured interviews by phone or in person with central employees in funding agencies and ministries and 25 correspondences with researchers and policy makers. Findings and conclusion: RCTs in grades 0–10 were few in all of Scandinavia until about 2011, after which there was an increase in all three countries, although at different rates. The largest number of trials has been conducted in Denmark, and the increase is more marked in Denmark and Norway compared with Sweden. International trends towards more impact evaluations and results from international comparisons such as PISA have likely affected the development in all countries, but while many trials in Denmark and Norway are the result of policy initiatives, only one such example in Sweden was identified. We believe the lack of government initiatives to promote RCTs in Sweden is the most likely explanation for the differences across the Scandinavian countries. Funding and coordination from the government are often crucial for the implementation of RCTs and are likely more important in smaller countries such as the Scandinavian ones. Supporting institutions have now been established in all three countries, and we believe that the use of RCTs in Scandinavian educational research is likely to continue.

Randomized controlled trials: the basics -- Types of randomized controlled trials -- Bias in randomized controlled trials -- Assessing the quality of randomized controlled trials: why, what, how, and by whom? -- Reporting and interpreting individual trials: the essentials -- From individual trials to groups of trials: reviews, meta-analyses and guidelines -- From trials to decisions: the basis of evidence-based health care -- Ethics and clinical trials: are they compatible? -- Reprise: more musings

Back in January, I posted about an article that was getting some attention in my world. Megan T. Stevenson is an active researcher in the criminal-justice-and-economics literature. She argues that when you look at the published studies that use randomized control trial methods to evaluate ways of reducing crime, most of the studies don't show a … Continue reading Pushback on Pessimism About Randomized Controlled Trials
The post Pushback on Pessimism About Randomized Controlled Trials first appeared on Conversable Economist.

Abstract
We report the results of a forecasting experiment about a randomized controlled trial that was conducted in the field. The experiment asks Ph.D. students, faculty, and policy practitioners to forecast (1) compliance rates for the RCT and (2) treatment effects of the intervention. The forecasting experiment randomizes the order of questions about compliance and treatment effects and the provision of information that a pilot experiment had been conducted which produced null results. Forecasters were excessively optimistic about treatment effects and unresponsive to item order as well as to information about a pilot. Those who declare themselves expert in the area relevant to the intervention are particularly resistant to new information that the treatment is ineffective. We interpret our results as suggesting that we should exercise caution when undertaking expert forecasting, since experts may have unrealistic expectations and may be inflexible in altering these even when provided new information.