The best paper award at the 22nd International Conference on Theory and Practice of Digital Libraries (TPDL 2018) went to the paper authored by David Pride and Petr Knoth titled “Peer review and citation data in predicting university rankings, a large-scale analysis.” The paper conducted the largest analysis of REF2014 data so far (data of 145 thousand submitted papers, 7 million citations across all 36 REF Units of Assessment/disciplines), looking at the link between peer review, conducted by REF nominated panels, and bibliometric indicators. The study found surprisingly high correlations of the REF results at an institutional level (Grade Points Average – GPA) with simple bibliometric indicators.  

This indicates that 2014 REF results could have been predicted using automated techniques to a high degree of accuracy for about a third of the disciplines, those with high average citations per paper. If such approach was adopted for just those disciplines, this could result in savings to UK universities and Research England of about £50 million every time a national exercise is run and even more if more disciplines adopted a similar approach. Since the preprint of this study was made available, a number of researchers have made contact with us and confirmed that they have since obtained similar results. This information is now being discussed with Jisc, who finance the project, to advise Research England on the next steps.

TPDL 2018 is the highest regarded conference in the area of digital libraries in Europe and 2nd worldwide. TPDL 2018 took place in Porto, Portugal.

Pride, D. and Knoth, P. (2018) Peer review and citation data in predicting university rankings, a large-scale analysis, Theory and Practice of Digital Libraries (TPDL) 2018, Porto, Portugal Lecture Notes in Computer Science, Springer, https://arxiv.org/abs/1805.08529