Revealing Reviewers’ Identities as Part of Open Peer Review and Analysis of the Review Reports

Abstract

This research article is aimed at comparing review reports, in which the identity of the reviewers is revealed to the authors of the papers, with those where the reviewers decided to remain anonymous. The review reports are gathered as part of the peer review process of the European Scientific Journal (ESJ). This journal maintains a single-blind peer review procedure and optional open review. Reviewers are familiar with the names of the authors but not vice versa. When sending the review reports, the reviewers can opt to reveal their identity to the authors.

The sample of 343 review reports from members of the ESJ editorial board, gathered within the period of May to July 2019, were analysed. The data analysis was performed using the Python programing language based on NumPy, Pandas, and Scipy packages.

Half of the reviewers decided to choose the open option and reveal their names to the authors of the papers. The other half remained anonymous. The results show that female reviewers more often decide to remain anonymous than their male colleagues. However, there is no significant difference in the review reports on the basis of gender or country of institutional affiliation of the reviewers.

Revealing identities did not make a difference in the reviewers’ point appraisal in the review reports. This difference was not significant. However, a majority of the reviewers who recommended rejection in their review reports were not willing to reveal their identities. Even more, those reviewers who revealed their identity were more likely to recommend in their review reports acceptance without revision or a minor revision.