Lachlan Coin completed a Bachelors degree of Science at the Australian National University in 1999, and a PhD at the University of Cambridge, in bioinformatics in 2005. He worked as a research fellow in the Department of Epidemiology and Biostatistics at Imperial College London and in 2012 took up a position as group leader at the Institute for Molecular Bioscience at the University of Queensland. Lachlan is co-founder of Academic Karma, a new platform for peer review.
1. Can you explain the concept of Academic Karma and tell us how it got started?
Academic Karma was born out of 14 years worth of frustration with peer review as a reviewer and author. I realized that peer-review inefficiencies are the main reason academic publishing is both frustratingly slow and expensive. The main reason for this seems to be the difficulty in finding willing, qualified reviewers, because the rewards for publishing are much greater than the rewards for reviewing.
We thought – what if there was a peer-review currency, which we call 'Karma', earned by doing timely review and spent by getting manuscripts reviewed. If every academic did as much peer review as they demand from others in order to maintain a positive Karma, the system would become self-regulating. We think this would make open-access publishing much cheaper and also much faster. We really don't want people to do more reviewing, on average, than they currently do, or to prioritize reviewing over other important time-sensitive work. We are just proposing that academics do as much reviewing as they are consuming review, and that they only agree to do it when they can turn it around in a reasonable time-frame.
2. How does Academic Karma work?
One important aspect is helping users keep a track of their 'Karma' over the last 5 years. We do this by linking up with ORCID to calculate how much Karma has been consumed. We also provide tools for users to upload their reviewing history to calculate how much Karma has been earned. We list all of the researchers who have a positive reviewing Karma at http://academickarma.org/backbone. We think these academics deserve to be rewarded for doing their fair share – so next time you are asked to review a paper from someone on this list please agree to do it, and try to do it in a timely manner!
The second important aspect is helping users indicate their expertise and willingness to peer review to editors. We provide tools for editors to use this information to identify and invite reviewers. We have built a cloud-based peer review platform so that editors can manage the entire peer-review process through Academic Karma.
Reviewers are emailed a review form customized to the requirements of the journal. The completed review is sent simultaneously to the author and to the editor so that the author gets faster feedback.
We wanted our platform to be a place where reviewers could do all their reviewing in one place. To enable this, reviewers can forward a review invitation from any journal to 'firstname.lastname@example.org', and subsequently do the review through Academic Karma.
When the paper is finally published, we email the reviewers to let them know and ask them if they would like to make their review openly accessible on the Academic Karma manuscripts page (http://academickarma.org/manuscripts).
3. Are you measuring peer reviewer performance (speed/quality of review) and if so, what are you noticing?
We have been measuring the time reviewers take between agreeing to review and returning their reviews. This has been averaging about 8 to 9 days. Our users have asked us to measure review quality too – this is one of the next things we intend to implement.
4. What has been the reaction from researchers?
We have a number of `power-users' who regularly use Academic Karma to return their reviews. We are really trying to make sure the platform works well and addresses the needs of a smaller user base before trying to scale up. We also want to avoid mass email marketing campaigns.
5. Does Academic Karma help reviewers to get academic credit for their review efforts?
The main recognition is highlighting reviewers who have done their 'fair-share' of reviewing. We hope that ultimately this will lead to them getting their own papers reviewed faster.
6. What has been the reaction from editors? Is there any hesitancy to accept reviews from reviewers using Academic Karma?
Most editors are supportive of what we are doing and are happy to accept reviews sent using Academic Karma. One issue when we first started was that we used the same review form for all journals, which created problems for editors, but shifting to customized review forms has solved this problem.
7. What is the biggest challenge faced by Academic Karma?
Peer review is currently in a 'vicious cycle' whereby it doesn't matter how diligent a reviewer you are, it doesn't help you get your own papers reviewed well or quickly. The biggest challenge we face is how to turn this 'vicious cycle' into a 'virtuous cycle' in which diligent, timely reviewers get access to fast, high quality peer review for their own papers. To kick-start this, we are encouraging the community to prioritize reviewing papers from reviewers with positive karma (http://academickarma.org/backbone), which will hopefully provide an incentive for all academics to do their fair share of reviewing.