[Zurück]

A. Gomez-Sanchez, A. Pacher:
"The Future of Research Evaluation: Current Landscape for New Metrics and New Tools";
Vortrag: Workshop über Möglichkeiten der Leistungsmessung im Hochschulsektor, TU Braunschweig, Deutschland; 24.06.2021 - 25.06.2021.

Research evaluation has been a key aspect for universities for decades. It has been mainly focusing on bibliometric measurements of peer-reviewed document production and impact. To access the most common indicators (productivity, citations to articles, h-index, journal metrics, etc.), the resources used mainly have been subscription-based databases, leading to some significant limitations such as an unequal treatment of subject areas, disadvantages of non-English publications, and other phenomena of exclusions.

The emergence of the models of Responsible Research and Innovation (RRI) and Open Science demands researchers to perform a broader variety of activities, all of which should be rewarded consequently. The alternative metrics have been a good start, but not enough. The past years have witnessed the publication of important proposals, such as the "Next-generation metrics" and the "Indicator Frameworks for Fostering Open Knowledge Practices in Science and Scholarship" published by the European Commission, and more recently, the "Next Generation Metrics" White Paper by the Task Force Benchmark of CESAER . At least some of the proposed indictors, including open science, innovation or teaching activities, should be considered to evaluate our researchers at universities.

It is based on this development that this presentation reflects upon how the current research evaluation landscape should adapt to the needs as outlined in those documents. In addition, this presentation offers an overview of some newer tools that may be suitable towards this goal.

It will argue that, first, efforts to introduce new ways of evaluations at institutional and national levels are required. Organisations and funders should embrace a responsible use of metrics, and develop strategies in line with the Open Science Agenda. Second, proper infrastructures that allow us to perform evaluations are required. It is essential to have reliable sources with curated information and good metadata that assure us to develop most accurate reports. Moreover, evaluation sources should not be limited to the ones provided by major commercial companies, but one should rather likewise consider new tools, many of which are open or free (for example, BASE or Semantic Scholar, OpenCitations, CoCites, etc.).

https://publik.tuwien.ac.at/files/publik_302284.pdf