The workshop was entitled: What should 21st century cross-disciplinary research evaluation look like?

Main aspects covered:

• the global debate on research evaluation today

• evaluation from an institutional perspective

• evaluating research for societal benefits.

The opening keynote was given by Angela Liberatore, head of the scientific department at ERCEA (European Research Council Executive Agency), who launched the new ERC procedure for research evaluation. 

Ex ante evaluation remains firmly anchored to peer-review assessment. In evaluating research proposals the ERC asks its peer reviewers to focus particularly on the scientific merit of the proposal.

What way forward for research evaluation systems?

Representatives of research councils from Norway and Italy, and a former head of the EUA Council for Doctoral Education (now head of international funding policy, Swiss National Science Foundation) were called upon to present their strategies to ensure that governmental evaluation agencies can identify the best contributions of researchers. 

These strategies focused on the evaluation of researchers’ host universities, rather than paying attention to the individual assessment of scholars. Furthermore, they were invited to address the value of research evaluation today for society and policymakers.

Looking ahead, the panel identified that some innovative features are emerging in evaluation. In particular, a wider definition of scientific output, responding to the international recommendation for a proper recognition of the diversity of research activities and practices.

Evaluating projects for societal benefits

Research must be considered in the dimension of social impact rather than economic growth, noted Steven Hill, director of research at UKRI. This is where the SSH have a comparative advantage. Hill pointed out that impact case studies reported by SSH faculties to the REF (Research Excellence Framework) exercise in 2021 demonstrated a much faster impact on policymakers than hard science studies. 

Furthermore, mission-related topics must privilege sustainable ‘value creation’. Not just technology and energy, but healthy societies and well-being. The panel acknowledged that social impact remains difficult to assess but must be a principle in particular for top-down funding programmes.

Public humanities, the new open access journal of Cambridge University Press will be launched at the end of 2024. It will give more visibility to how the humanities address major societal challenges following an interdisciplinary approach.

Where next for the funding and assessment of transdisciplinary research?

James Wilsdon, Director of RORI (Research on Research Institute) at UCL, described the need for a new approach to the evaluation of research in the context of the changing demands by policymakers on the research system.  

Funders increasingly take a mission-driven approach to research investment, which continues to encourage multi- and transdisciplinary research as the route to tackling the major societal challenges. The ‘mission’ approach also seeks to engage non-academic partners into the multi-discipline research collaborations, creating layers of complexity for the evaluation of research and research funding.

Wilsdon argued that the social sciences, by deploying ‘whole systems’ thinking approaches, can provide the ‘secret sauce’ to research evaluation in this multi-discipline, mission-driven research environment.

The workshop is a starting point for the work that EASSH will continue with the CoARA working group on “Evaluating SSH globally” (coming in spring 2024).

More information:

Download the worshop report.

Presentations:

• ANVUR_Research_Evaluation in SSH_MM.pdf
• ERC_Research evaluation_AL.pdf
• RCN_National research assessments in Norway_JH.pdf
• T&F_Challenge-based research_Liz Allen.pdf