Suchergebnisse

This article examines the concept of credible evidence in Extension evaluations with specific attention to the measures and measurement strategies used to collect and create data. Credibility depends on multiple factors, including data quality and methodological rigor, characteristics of the stakeholder audience, stakeholder beliefs about the information source, and the evaluation context. Measurement planning involves a process of making thoughtful decisions about choosing study variables, measurement strategies, and specific measures that adequately reflect the content and goals of the program being evaluated. The use of specific measures may also entail implicit assumptions, e.g., that the respondent is being truthful and accurate, which must be accepted if resulting data are to be viewed as credible. The article discusses aspects of measurement quality, including reliability and validity, for both quantitative and qualitative forms of data. Program stakeholders should be encouraged to be attentive, reflective, and critical in their analysis of evaluation evidence, and their views on what makes data credible must be understood and considered. The use of common measures in evaluating multi-site programs can be valuable, but only if the measures are fully appropriate for local sites. The article concludes with a summary of implications and recommendations for Extension evaluation practice.

Situational complexity is the distinction between simple, technically complicated, socially complicated, and complex situations. Programs that operate in simple situations are usually able to follow a prescribed course of action, or recipe, while programs operating in more complicated or complex situations must be flexible and responsive. In this article, the authors present findings from an exploratory, multiple-case study of the credibility of evidence in four distinct program situations ranging from simple to complex. Key informant interviews were conducted with 16 key informants, both internal and external to Extension. The findings were generally that the more complex the situation, the more likely that flexible or mixed-methods approaches were employed to strengthen program credibility. Across all the cases, the relationships that Extension educators have built with stakeholders played a pivotal role in building credibility of evidence. We conclude that sometimes situational complexity matters, sometimes methods matter, sometimes reporting style matters, but what always matters is the trusting relationship between the delivery organization and the stakeholder.

A review of the book What Counts as Credible Evidence in Applied Research and Evaluation Practice? by Stewart I. Donaldson, Christina A. Christie, & Melvin M. Mark, published in 2008 by Sage.

Purpose: This article builds on the Practice Research Collaboratives (PRCs) as an initiative that developed from the Fifth Conference on Practice Research to provide a platform for practice researchers to engage actively around impact and influence. Research question: The unique features of research activities that enable transformational impact in three cases in social work practice research involving long-term community and university research partnerships. Methodology: Literature review and comparison of case studies. Results: The cases show how the processes of implementation are not only seen as linear, but constantly evolving at the same time as intervention fidelity in social work is crucial to improving outcomes for people, which can have transformative impacts for individuals and systems alike. Conclusion: We conclude by describing the importance of understanding the feasibility of complex problems and complex social situation which requires meaningful communication between partners, transparency and involvement of all players throughout the process.

This article provides insight into how an adequate approach to selecting methods can establish credible and actionable evidence. The authors offer strategies to effectively support Extension professionals, including program developers and evaluators, in being more deliberate when selecting appropriate qualitative and quantitative methods. In addition, several examples of commonly used measures are described to help in determining their applicability for evaluating Extension programs. Benefits and challenges of select methods are discussed as well as pitfalls to avoid that can derail the evaluative process. Lastly, a few cases are shared to present how Extension is aiming to establish credible evidence through state efforts and at the national level. The authors discuss the use of practical designs (e.g., common measures) that offer a more uniform way of evaluating programs. Examples are also included to highlight the effective use of Extension reporting systems that aim to streamline data collection, evaluation, and reporting as a means to ensure more credibility.

AbstractAccording to the Directive 2014/95/EU on non‐financial information (NFI), from 2017 onwards, large companies of member states must provide social, environmental, and governance disclosures. This paper, focusing on the evaluation of the credibility of NFI in Italy after the implementation of the EU Directive, aims to investigate whether making non‐financial disclosures (NFDs) obligatory affect their credibility. From a theoretical perspective of material legitimacy theory, we investigated the NFDs of the 31 FTSE MIB Italian listed companies for the 2017 fiscal year. Through a meaning‐oriented content analysis, we developed a self‐constructed credibility index applying an operational framework based on the Habermas idealism. Our framework conceives credibility as a multidimensional construct, constituted of three sub‐dimensions: truth, sincerity and appropriateness, and understandability. Our findings show that the NFDs of Italian FTSE MIB companies tend to be credible, contributing to widen the empirical researches addressed to investigate the credibility of NFDs in mandatory contexts.

The debate over what counts as credible evidence often occurs on a methodological level (i.e., about what technical applications of systematic inquiry provide believable, justifiable claims about a program). Less often, it occurs on an epistemological level (i.e., about what ways of knowing are appropriate for making claims about a program). Even less often, the debate touches on ontological concerns (i.e., about what conceptualizations of reality, in general or in relation to a specific program, are in play when we wish to make claims about that program). For example, whether we understand Extension to be a vehicle for the dissemination of scientific knowledge or a site of grassroots democracy matters when we seek to evaluate Extension with credibility. The purpose of this paper is to examine the credible evidence debates through an ontological lens, showing why and how different narratives (or different realities) of Extension must be considered when we seek credible evidence about Extension.

Determining what constitutes practically relevant, statistically significant evidence for animal sentience, under the precautionary principle, could be enhanced through Bayesian statistics. A Bayesian approach allows the incorporation of multiple evidence sources through prior probabilities, the tracking of changing evidence across time, and a principled means of adjusting evidentiary bars via Bayes factors.

AbstractCredibility in evaluation is a multifaceted concept that involves consideration of diverse stakeholders' perspectives and purposes. The use of a transformative lens is proposed as a means to bringing issues of social justice and human rights to the foreground in decisions about methodology, credibility of evidence, and use of evaluation findings. ©Wiley Periodicals, Inc., and the American Evaluation Association.