Latvia – like many countries is trying to evaluate and reward the societal, policy, environmental and economic impact of publicly funded research. For many years there has been a discussion around the use of Altmetrics – various tools have been developed which promise useful insights to research policy-makers. Unfortunately, none of these tools are useful on the operational level due to a mismatch between the underlying data sources and the actual pathways to impact, issues with the recognition of content in the Latvian language and a risk of abuse if funding decisions are based on the scores. We can easily demonstrate this by e.g. noting the low accuracy of research mentions in policy documents in Altmetrics databases.
A comprehensive research impact assessment exercise like the United Kingdom Research Excellence Framework (REF) is expensive and administratively burdensome. In the case of Latvia, the added benefits beyond gathering information for allocating institutional funding currently do not justify the costs. Thus, we ask – would such an exercise meaningfully change the institutional funding allocation compared to the existing institutional funding system? One paper comparing the UK and Italian exercises has found that if REF output submitted had been evaluated with the bibliometric algorithm used in the VQR, with peer review assessing the rest of the institutional submission, the correlation between the actual funding assigned to each institution and the funding it would have received would have exceeded 0.9997. Furthermore, correlations between peer-reviewed impact exercises and traditional assessment methods vary greatly by discipline. Thus, if the impact of research beyond academia is to be rewarded on the institutional level, we would not mimic the gold-standard of REF, but instead focus on areas such as Humanities and Social Sciences where the true value of research is difficult to capture using traditional indicators. In the meantime, our efforts are focused on promoting the societal impact of research on the researcher and project level.
