Suchergebnisse

A case is made for using the technique of evaluability assessment to introduce students to the field of evaluation. Procedures for directing, instructing, and supervising evaluability assessments in the classroom setting are described, along with techniques for adapting the method to the students' level of training, mix of experiences, and duration of the class. Evaluability assessment is a useful exercise that assists students in applying their research skills to real program needs. Twenty local evaluability assessments, course evaluations, and subsequent contact indicate that this approach is beneficial for student professional development and is a useful service for program stakeholders.

AbstractEvaluability assessment involves key policymakers, managers, and staff in developing program theory and clarifying intended uses of evaluation information, thus helping solve problems that inhibit useful program evaluation.

AbstractThe debate between Semantic Minimalism and Radical Contextualism is standardly characterized as concerning truth-evaluability—specifically, whether or not sentences require rich contextualization in order to express complete, truth-evaluable contents. In this paper, I examine the notion of truth-evaluability, considering which kinds of mappings it might require from worldly states of affairs to truth-values. At one end of the spectrum, an exhaustive notion would require truth-evaluable contents to map all possible states of affairs to truth-values. At the other end, a liberal notion would require only that truth-evaluable contents map at least one possible state of affairs to at least one truth-value. I show that both Minimalists and Radical Contextualists rely on some intermediate, moderately strict notion of truth-evaluability, falling between these two poles. I consider four ways in which such a notion could be defined. However, I argue that each of these is ultimately implausible, giving us no reason to favour a moderately strict notion of truth-evaluability over the liberal alternative. This suggests that the debate must shift to more moderate ground; rather than concerning the in principle possibility of truth-evaluable contents, it fundamentally hinges on their explanatory value. More generally, paying close attention to the notion of truth-evaluability allows us to tease apart distinct strands in the Minimalism-Contextualism debate, and gain a better appreciation of what is at stake.

AbstractThe authors describe training of a network of professionals to conduct evaluability assessments for the Early Assessment of Programs and Policies on Childhood Obesity Initiative (Early Assessment Initiative). We learned that (1) the professionals trained had a diverse set of skills and expertise, (2) a training refresher session solidified methods and concepts, (3) evaluability assessments were improved when training included content on qualitative interviewing and development and application of logic models, and (4) comprehensive and consistent training methods helped to ensure consistency and methodological integrity. © Wiley Periodicals, Inc., and the American Evaluation Association.

Evaluability assessment focuses on the readiness of organisations to carry out evaluations. Scholars argue that evaluability assessment needs to focus on internal evaluation systems and tools and their capability to measure programmes and services reliably and credibly. Even so, literature on best practice guidelines on evaluability assessment within the context of the not-for-profit sector appears to be rare. We seek to begin to fill this gap by presenting lessons learned from Ngala, Western Australia, when we reviewed the organisation's evaluation practice and culture in 2018/2019. The Service Model and Outcomes Measurement Audit project assessed the extent to which service models within Ngala aligned with the organisation's standardised service model and individual service contracts, as well as consistency of outcomes, data collection and reporting practices. Insights obtained from the project and their implications for evaluability assessment practice are discussed.

This paper presents the results of an evaluability assessment (EA) conducted in Spain. EAs are assessments that minimize the uncertainty and risk inherent in a subsequent evaluation (e.g. an impact or results assessment) that may require excessive resources. We designed our own EA model by combining three theoretical approaches, with a holistic and pedagogic vision of the process and a flexible methodology to enable adaptation to different programs and contexts, thus fostering transferability. The EA was carried using a qualitative methodology: documentary analysis, seven semi-structured interviews with stakeholders, eight direct observation sessions and a check-list for politicians and experts. The outcome report helped promote organizational learning, provided the program with a logical and assessable theoretical model, detected its strengths and helped overcome the barriers to efficient assessment. Based on these achievements, an evaluation of the results was recommended, with repeated pretest-posttest measurements and an equivalent control group.

Evaluation is essential to understand whether and how policies and other interventions work, why they sometimes fail, and whether they represent a good use of resources. Evaluability assessment (EA) is a means of collaboratively planning and designing evaluations, seeking to ensure they generate relevant and robust evidence that supports decision-making and contributes to the wider evidence base. This article reports on the context, the process undertaken and evidence from participants in an EA facilitated with public service workers involved in implementing a complex, area-based community improvement initiative. This is a novel context in which to conduct an EA. We show how the process allows practitioners at all levels to identify activities for evaluation and co-produce the theory of change developed through the EA. This enables evaluation recommendations to be developed that are relevant to the implementation of the programme, and which take account of available data and resources for evaluation.