Yesterday Wiley announced a new pilot program with Publons, a new peer review information tool which produces comprehensive reviewer profiles with publisher-verified peer reviews, allowing reviewers to get credit for their reviews. We recently spoke with Publons co-founder Andrew Preston about how Publons got started, what it does, and how it’s been received thus far.
Q. Can you tell us a little about your background and how Publons came about?
A. I was finishing up a postdoc role at Boston University and it was time to decide 'what next'. I love doing science, but academic life can be frustrating at times. A lot of the things we consider normal in this industry seem crazy to outsiders.
At the same time, my Publons co-founder Daniel and I had been dreaming up startup ideas together since we were 10. This one was an intersection between a pressing problem and a real opportunity to have a meaningful impact on the world. So we dived in, learned an awful lot about the academic publishing industry, and the idea evolved into what Publons is today.
I think the critical point leading us to peer review in particular was learning that of the 150 days (on average) it takes to publish a paper, 120 of them are spent in peer review. We figured that we could speed up science by focusing on that problem.
Q. What is Publons and how is it “speeding up science”?
A. Our mission is to speed up science by providing a range of tools and services to reviewers, editors, and publishers that improve the peer review process. That starts with giving reviewers credit for peer review, in the form of a verified reviewer profile (example below) that can be used in performance evaluations etc.
Reviewers have told us they respond to editors' peer review requests quicker, are more willing to review, and offer more constructive feedback knowing they’re going to get credit for it on Publons. That’s the immediate impact on the speed of science that we love to see.
We also work with editors and publishers to help reward their reviewers, to verify and showcase their journals' peer review processes, to identify peer review fraud, and to aid them in finding expert reviewers.
Q. What does Publons offer reviewers?
A. The main thing that motivates reviewers is getting credit that they can put on their CVs and include in performance evaluations. But many reviewers also use Publons to publish their reviews, write post-publication reviews, and discuss the reviews and research of others. Some early career researchers also browse the reviews on Publons as a way to learn how to perform their own reviews.
As a bit of fun, we also run a rewards program, where each quarter the three most prolific reviewers receive a rewards package valued at over $3,000.
Q. What has the reaction from authors been? Is there any hesitancy on the part of authors to have reviews of their papers open to the public?
A. It’s important to note that only a small fraction of the reviews on Publons are published. Many reviewers elect for their reviews to remain blind and unpublished. Behind the scenes we work with publishers and editors to verify these reviews -- with the Wiley partnership we’re able to formalize and automate the verification, which is one of the reasons we’re so excited about it.
With regards to published reviews, it’s a new thing, so there is occasional apprehension -- but this is usually by editors or publishers concerned about how authors might react, rather than apprehension from the authors themselves. To date, no author has ever requested a review of his/her article to be taken down.
Transparency can be scary, but it has its advantages too. When you think about it, most authors would be quite pleased if an expert took the time to read and discuss their publications. This is why we only allow reviewers to publish reviews after the manuscript has been published. Our sample is likely biased since most of the authors we converse with are reviewers on Publons, but many authors are happy to have their experiences with reviewers out in the open for anyone to see and respond.
Another advantage of having the reviews of your articles on Publons is the boost to the readership and Altmetric score of the article (since we integrate directly with Altmetric).
Q. Publons has a scoring system. How does that work?
A. We use reviewer “merit” as a quick indicator of review expertise. Each review is awarded a base merit of 1 with additional merit points awarded for being verified by the editor/journal, for being published, and (if published) for receiving endorsements from other reviewers. A reviewer’s merit is simply the sum of all his/her review merit points.
Q. What does Publons offer to journal editors?
A. One of the biggest things we’ve learned is just how important the role of an academic editor is in organizing and monitoring the peer review process, so we’ve devoted a lot of time in recent months to learning a lot about that process and developing tools to help editors to better perform this vital role.
Obviously we provide editors with a way to formally recognize their reviewers, but we’ve also developed two tools to help editors to:
- Search for and contact reviewers; and
- Screen author-provided reviewer suggestions to ensure they are legitimate and suitable peer reviewers.
This is all part of our mission to speed up science. By the way, we also provide a way for editors to get credit for their editorial work.
Q. Who uses Publons?
A. If you browse our top reviewer list you’ll see researchers from most countries and universities around the world. Lots of early career researchers are also using Publons as a way to build their reputations with the journals and editors in their areas of expertise.
The reviews are for thousands of different journals. The majority are STEM titles but lately we’ve started to see a few more reviews from the social sciences. Reviewers are adding more than 1,000 reviews each week so the variety is growing quickly.
Q. Why did you decide to partner with Wiley?
A. A great deal of Wiley reviewers are already using Publons so this partnership seemed like an excellent way to make the process of getting credit for their reviews even easier. Working together also makes it possible to give other Wiley reviewers (who might not be aware of Publons) credit for the first time.
We also saw that Wiley has shown a willingness to experiment with new things (like Altmetric.com) and it has turned out to be a pleasure to develop this pilot with the team there.