Suchergebnisse

Evaluation has been designed and used as an instrument of organisational governance for publicly funded research institutions (PRIs). Such an instrument can justify external public support and provide internal evidence for decision-making and organisational learning. Under given national and institutional contexts, defined missions of PRIs shape the characteristics of organisational governance in at least three aspects: strategies, funding, and operation. Governance tensions related to them involve key stakeholders at three organisational levels: central decision-making and supervisory bodies at the macro-level (L1), research institutes at the meso-level (L2) and individual researchers at the micro-level (L3). The thesis offers a conceptual framework, built in a formative way, to answer the research question: How do evaluation mechanisms of PRIs play a balancing role and help to mitigate tensions in organisational governance? Three PRIs were investigated, namely, the Max Planck Society (MPG) and the Helmholtz Association (HGF) in Germany, and the Chinese Academy of Sciences (CAS). Their governance tensions across the three levels are reflected and balanced by the evaluation mechanisms, which facilitate the organisational learning. The balancing processes occur when the individual evaluation activities supplement each other within the institutionalisation of the overarching evaluation mechanisms. During the processes, the stakeholders' interactions and negotiations vis-à-vis their diverging interests are also facilitated. Generalisable balancing roles of evaluation mechanisms in organisational governance are found at each organisational level in the three PRIs studied. Specifically, the balance found at L1 is between external justification of public funding and internal governance of heterogeneous research; at L2 between required alignment of organisational strategies and individual institutes' autonomous development; and at L3 between meeting management instructions and protecting scientists' trust and freedom. A common principle to mitigate governance tensions across the three levels involving various stakeholders is found—a proper contextualisation of evaluation designs and uses. Implications are suggested for useful ways of discovering and balancing governance tensions at PRIs in general and for public policies on research evaluation in wider landscapes of PRIs.

Broader impact of scientific research beyond academia has become increasingly important in research evaluation. To evaluate broader impact of research proposals, some funding agencies compose mixed panels that include peer experts and non-academic stakeholders. Whether and how non-academic reviewers bring any difference to panel discussions has been understudied. We analysed 164 review reports (2014–6) from the Investigators Programme (funding Science, Technology, Engineering, and Mathematics fields) at Science Foundation Ireland, where two types of panels, with and without non-academics, were composed for impact assessments. We find that the mixed panel reviews were longer and touched upon broader and more concrete impact topics. Also, mixed panels commented on causality and attribution of impact towards characteristics of applicants and research process more than scientific excellence. A survey of the same reviewer pool supplements our understanding of the pros and cons of the inclusion of non-academic reviewers. We discuss some policy recommendations for funding agencies to organise review panels.