Professor Flaminio Squazzoni is chair of PEERE, a project (‘Action’) funded by the European Union to explore issues around journal and grant peer review, running from 2014 to 2018. I recently spoke with Professor Squazzoni to learn more about PEERE’s mission, milestones, and desired outcomes.
Q. Give us some background on the PEERE Action: how did it begin? What is its raison d’être? How does your current academic position relate to it?
A. The idea behind PEERE started both from my research interests on previous work I have carried out on evaluation processes and peer review and from several discussions with colleagues. I cannot count how many times during these events we shared our personal frustration about the situation of peer review, both at the journal and grant funding level, and especially the lack of solid understanding and systematic analysis of this important scientific institution. Eventually, we decided to stop complaining and start doing something about it. I attended a conference on the future of open science in Oxford in 2011 and realized that only a large-scale, multidisciplinary, inter-sectoral collaboration could make a difference. I pulled together a group of people undertaking scientific research on peer review across various disciplines. Separately I contacted some stakeholders (such as Wiley, Elsevier, and Springer) with the idea of involving them in the most important challenge of PEERE: opening a data sharing initiative for journals to make their peer review data available for research. The EU Cooperation in Science and Technology (COST) panel eventually agreed to support the initiative, and PEERE came to life.
Q. Who is involved in the Action now?
A. We have 28 EU countries represented. The management committee consists of 40 members with another 20 colleagues in three working groups. These working groups are focusing on analysis of peer review, data sharing, and research agenda. The members represent a wide range of specialties, from scientometrics and computer science to behavioral and biomedical sciences, with a good balance of gender and broad range of academic experience. A great feature of PEERE is that most members have never met one another; it is a truly collaborative exercise.
Q. What has PEERE done so far?
A. We meet several times a year in different European locations, with invited speakers and presentations of research undertaken by members of the Action. We have performed a review of the literature on peer review, trying to trace the evolution of this field empirically. The number of publications on this topic clearly demonstrates a growing interest in the area. We have also started to discuss how to measure the quality and efficiency of the peer review process. Initial research on peer review data for some journals has investigated reviewer bias, network effects (how much the position of a scientist in the community may influence his/her judgement), variations in the way peer reviewers assess papers across different disciplines, and the content and quality of peer reviewer reports. More importantly we have started to collaborate with our three publisher stakeholders, Wiley, Elsevier, and Springer, to discuss the logistics and legalities around sharing data. We are in the process of developing a protocol to make peer review data sharing possible while guaranteeing the interests of all involved.
Q. What activities are planned for the future?
A. Our next goal is to create a dataset on peer review among a sample of journals, to study peer review in different fields and contexts, and measure the quality and efficiency of the process. We plan to explore whether the type of peer review model (open vs. closed, pre-publication vs. post-publication) has a significant effect on the quality and efficiency of the process. We are also planning to undertake in-depth analysis of peer review at funding agencies in order to understand better the role of peer review in the way grant panels make funding decisions. Finally we hope to organize a summer school on peer review, probably in 2017, where we will train academics and professionals on approaches, methods, and tools in peer review. More practically we are planning some publications in journals that we hope will set the standard for future research in the area.
Q. What do you find most valuable about PEERE?
A. PEERE is the first, large-scale collaboration on peer review. It consists of a group of smart people who want to learn from each other and make a difference. Each PEERE meeting has a fairly informal atmosphere with plenty of opportunity for debate and dialogue. An expert on computational linguistics might debate with an economist the pros and cons of paying for fast track peer review, or a scientist might discuss some research results into post-publication peer review with a representative from a publisher. This is not what you find in typical workshops and conferences.
Q. What do you hope will be the outcomes of PEERE?
A. PEERE wants to make a difference. I hope that, by the time PEERE has finished, data sharing on peer review will be more the rule than the exception. I hope too that we will have clearer ideas about the pros and cons of different models of peer review and more evidence-based knowledge on how to manage peer review, for example, what types of incentives and guidelines can improve its quality and efficiency. Our conclusions will be relevant not only to journal authors, reviewers, and editors, but also to publishers and funding agencies.
Q. The Action runs until 2018. Can anyone interested in PEERE’s activities join?
A. Any COST Action is open to members of the EU, and we do also have some non-EU partners from North and South America. I recommend looking at the PEERE website (www.peere.org) and, if you are interested, contacting me or the COST Coordinator in your home country.
Flaminio Squazzoni is the chair of the PEERE management committee, a professor at the University of Brescia, and editor of the Journal of Artificial Societies and Social Simulation. He can be reached at firstname.lastname@example.org.