Suchergebnisse

Official development agencies are increasingly supporting civil society lobby and advocacy (L&A) to address poverty and human rights. However, there are challenges in evaluating L&A. As programme objectives are often to change policies or practices in a single institution like a Government Ministry, L&A programmes are often not amenable to large-n impact evaluation methods. They often work in strategic partnerships to foster change; hence, contribution may be a more relevant evaluation question than attribution. Small-n qualitative approaches are available to measure the effectiveness of L&A which use the theory of change as their analytical framework. We conducted a meta-evaluation of 36 evaluations of multi-component international programmes to support civil society L&A across Asia, Africa and Latin America, comprising the majority of programmatic support from one international donor. We assessed the confidence in causal claims in the evaluations using a new tool that we developed. Assessments of the contribution of the programmes to the changes in outcomes were not provided in many of the evaluations, nor were predictable sources of bias addressed. Given that L&A programmes are likely to adopt an influencing approach where many different inside-track and outside-track engagement objectives, opportunities and strategies are attempted, many of which might be expected to fail, there appeared to be a clear bias in the evaluations towards reporting outcomes that were achieved, ignoring those that were not. We provide guidance on how to improve the design, conduct and reporting of small-n qualitative evaluations of aid effectiveness.

Background In circumstances where vaccine hesitancy is high, like in Tanzania in 2021, measures to control the spread of COVID-19 infection through non-pharmaceutical interventions, specifically mask-wearing, hand-washing and physical distancing, become crucial. This study was undertaken to inform the development of a context-adapted communication campaign to control COVID-19 in Tanzania. Focus of the Article The study examines the effectiveness of three different behaviour change campaign ads created as part of the same control effort, and two of them by the same creative process, with the same creative team, at roughly the same time, in altering behavioural propensities for relevant target behaviours, including the washing of hands, the wearing of masks and social distancing, and key participant attitudes, such as the sense of obligation to others. Research Question The main research question was: what are the effects of behaviour change campaign ads on propensities for engaging in COVID-19 target behaviours? We also examined behavioural attitudes and consumer appreciation of the ads (believability, likeability, relevance and surprise). The purpose of the study was to provide empirical evidence on the effectiveness of different messaging logics, in order to help improve future campaigns. Methods We designed a controlled survey experiment where participants were randomly assigned to receive one of the three campaign ads. These were called: the "Setting" campaign, which was designed using a theory-based Behaviour Centred Design (BCD) approach and produced with local partners; the "Password" campaign, which was the result of a substantial investment by a major international consortium; and the "Balance" campaign, a local adaption of "Password"; as well as an educational public service announcement, which served as an active control. An SMS survey was administered online to 2080 participants in Tanzania in 2022 to evaluate the potential effectiveness and consumer appreciation of the campaigns. Results The results showed that the most costly campaign, "Password", did not perform better than the other campaigns on any test. In particular, "Password" was not more effective than the theory-based "Setting" campaign on any single behavioural indicator or exposure variable. The "Setting" campaign was more effective than other ads on the expected qualities of appreciation (surprise, believability). However, the educational announcement achieved higher average response levels than all of the narrative-based treatments on measures of consumer appreciation, such as likeability, believability and relevance. Recommendations for Practice This study supported the proposition that a campaign specifically designed to elicit particular kinds of psychological responses could do so. It showed that a theory-based campaign, produced locally and at low cost, can compete on standard marketing values with high-quality creative processes and production values.

Non-randomized studies of intervention effects (NRS), also called quasi-experiments, provide useful decision support about development impacts. However, the assumptions underpinning them are usually untestable, their verification resting on empirical replication. The internal replication study aims to do this by comparing results from a causal benchmark study, usually a randomized controlled trial (RCT), with those from an NRS conducted at the same time in the sampled population. We aimed to determine the credibility and generalizability of findings in internal replication studies in development economics, through a systematic review and meta-analysis. We systematically searched for internal replication studies of RCTs conducted on socioeconomic interventions in low- and middle-income countries. We critically appraised the benchmark randomized studies, using an adapted tool. We extracted and statistically synthesized empirical measures of bias. We included 600 estimates of correspondence between NRS and benchmark RCTs. All internal replication studies were found to have at least "some concerns" about bias and some had high risk of bias. We found that study designs with selection on unobservables, in particular regression discontinuity, on average produced absolute standardized bias estimates that were approximately zero, that is, equivalent to the estimates produced by RCTs. But study conduct also mattered. For example, matching using pre-tests and nearest neighbor algorithms corresponded more closely to the benchmarks. The findings from this systematic review confirm that NRS can produce unbiased estimates. Authors of internal replication studies should publish pre-analysis protocols to enhance their credibility.