Jill Yablonski-Crepeau
Jill Yablonski-Crepeau
Author Marketing Manager, Wiley

What aspects of peer review do you find the most challenging?  When we asked individuals from around the globe this question, their answers ranged from balancing support and criticism, to accepting the feedback, to finding the time.  Take a look at all the responses in the video below and let us know what you think is challenging about peer review in the comments section or Tweet us @WileyExchanges.  We want to hear from you!

What aspects of peer review.PNG

    Margaret Kearney
Margaret Kearney
Vice Provost, University of Rochester

Picture this. The same article receives these three reviewer recommendations:

Peer-review-pic-3.jpg• Reviewer #1: Recommends rejecting the manuscript for publication. Comments include a need to demonstrate that analyses were two-tailed tests, better defend the use of a certain statistic to judge model fit, and add several points to the Discussion section.

• Reviewer #2: Recommends that the authors revise and resubmit. Comments, which include disparaging language (“surprisingly weak,” “obvious to others in the field”), note cumbersome writing, insufficient literature review, and overly long Discussion section.

• Reviewer #3: Recommends that the editor accept the paper. Comments include suggestions on how to clarify the research questions, details that should be added on the sampling and data collection approach, a recommendation to add a table of correlations, and aspects of the findings that could be highlighted in the Discussion.

As editor of a research journal, if I receive the three hypothetical reviews summarized above, should I focus on the confidential recommendations regarding acceptance or on the review content? Should I rescind one if I find it completely inaccurate or unnecessarily hurtful? What action is best for the journal? What is best for the state of knowledge in the field? What is best for the author?

Editors balance these questions daily as they make final decisions on manuscripts submitted for publication. Their decisions are shaped by the journal and its audience, the nature of the writing talent available, the level of competition for acceptance and page space, and many other factors, but some common principles can be found.

The first principle in my decision-making is that the reviewers’ recommendations are not a vote and I have no obligation to follow the majority, or indeed, any of the recommendations. Instead, I use reviewers as consultants. Reviewers are selected because they have indicated when joining my review panel that they have relevance to the manuscript. Plus, I know the strengths of most reviewers.  If a statistician has a comment about statistics, it carries more weight with me than if a clinical leader makes such a comment. Conversely, if a statistician comments on the relevance of content to health care clinician readers, those comments will be taken with a grain of salt.

The second principle is that the editor is the arbiter and interpreter to the author of contradictions and confusion in review content. That does include the option to rescind, edit, or de-emphasize a review. I do sometimes edit adjectives and adverbs in reviews to make them civil, and communicate with reviewers whose approach is consistently unhelpful. If reviews are very discordant or complex, I include in the decision letter some guidance as to how to prioritize. I generally support the well-reasoned, well-defended, and important comments and put less emphasis on others.

In making a decision based on the reviews above, I would probably keep the paper in the pipeline and ask for revision and resubmission. No reviewer identified a fatal flaw in the study being described that cannot be remedied in revision. In the reviews above, #1 and #3 provide concrete issues that can be addressed, but #2 provides few specifics on where the problems lie.  Authors who are uncertain how to respond to a mix of feedback can and should contact the editor to gain a better understanding of how to proceed.

The peer review process is essential to progress in the literature in any field. If you are a reviewer, please consider yourself one of several important judges of, and contributors to, the quality of presentation of a future contribution to the field, rather than the sole decider of its fate. Whether or not you think the paper has a future in print, provide concrete and polite suggestions without overplaying your own research, and trust that the editor does not expect you to comment on aspects of the paper in which you are not an expert. If you are an author of a work in revision, please use your editor for guidance as needed to respond to conflicting recommendations. If you are an editor, take the time to scan each review for plausibility and civility, and assist the author as needed to separate the wheat from the chaff and give each review the attention it merits.

Credit Image/Source - iStockphoto

    Jo Wixon

On a cold day in Frankfurt, a record number of staff from publishing companies large and small, publishing consultants, suppliers and a smattering of self-publishing societies met for a day to network and catch up on important initiatives in publishing.

After a blue sky keynote speech by Robert Hariri, Co-Founder of Human Longevity Inc. about the time being right to take personalized medicine further, with the aim of extending human lifespan, by early detection of the propensity for disease and treatment or lifestyle modification to tackle it, we came back down to earth to cover important matters directly related to publishing.

Presenting the results of STM’s consultation on article sharing, H Frederick Dylla pointed out that publisher's advice to authors on article sharing needs to be clear, and consider scholarly collaboration networks. He urged everyone to review the STM Principles, and, if you think they are reasonable, to sign up.

Caroline Sutton also appealed for support from scholarly publishers, and societies who own journals, to spread the word to authors: “Before you submit your work (particularly to a journal you have never heard of that has contacted you out of the blue), visit http://ThinkCheckSubmit.org, and follow the guidance to evaluate the journal.” For more information, see our recent post on the organization.

An array of inspiring case studies of advances made by researchers, clinicians and librarians due to Research4Life access to journal content, from Richard Gedye, moved all to agree that we must tell this story more widely, as it shows that what we do as scholarly publishers makes a difference.

Laurel Haak explained that ORCID is a hub that connects identifiers for people, places and things; it provides 'plumbing' and used properly, can lead to trust. For this, publishers must require authentication when asking authors to supply ORCID IDs with their submissions, which could allow for automatic updates of ORCID and other author profiles with their new publications.

Michael Jubb reviewed the findings from Monitoring the Transition to Open Access a study into the effects of RCUK’s open access policy following the Finch Group Report. He too stressed the need for more clarity from publishers on their policies regarding article posting, and stated that the STM Principles are critical for improving this. Key observations from the data were:

  • Gold OA journals made up just <17% of titles in Scopus by 2014
  • UK authors showed a stronger preference for hybrid OA titles over gold or green OA than global authors
  • A strong correlation between mean APC paid by UK universities and the impact factor of the journal
  • Of the 280 learned societies in the UK, 63% published one journal and 22% two or more titles
  • Of those UK learned societies with journals, 24% are self-publishing and 76% work with a publishing partner
  • For more than 50% of UK learned societies with journals, their journals income funds more than 50% of their charitable expenditure
  • 24% of articles had a free version posted online regardless of publishing model, with about half of these being illicit

Next came an amusing, but sobering demonstration from Roger Schonfeld of the frustrations for some academics of off-campus access to newly published journal articles, and the continued difficulties that can be experienced upon reaching campus and using a PC. The takeaways from his report on Streamlining access to scholarly resources were: understand the use of your platform in the library system environment and off-campus; set expectations for how quickly your new content appears in discovery services; eliminate mobile user interfaces; and enhance single user accounts, perhaps using single sign-on.

The day was rounded off by a CEOs panel of Brian Crawford (ACS), Tracey Armstrong (CCC), Ron Mobed (Elsevier), Steven Hall (IOP), and Philip Carpenter (Wiley).

Most of the panelists agreed that the sources of competitive advantage lie in service levels (to authors, readers or society partners) and technology (but only by keeping moving!) This chimed with a question on whether publishers are becoming technology companies, and was it worth investing in new journals if so; here all felt that it was not an either/or situation, but rather that we need to make the best use of technology to help readers find the information they need faster, and also that content, albeit perhaps in a different form than journals in the future, will still be key, as will publishers’ role in filtering it for quality.

CHORUS received multiple plaudits as the most impressive new entrant, with nods also to CrossRef for great collaborative work and continuing to have good ideas; and to the importance of analytics offerings.

On the effects of green OA and 12-month embargoes, it was felt by all that although commonly perceived as workable, a 12-month embargo period does in fact have an impact and there would be some titles that could not survive it, since much of an article’s usage happens beyond 12 months after publication.

Other questions mainly focused on the publishing workforce, including recruiting and retention of top performers. Steven Hall voiced a concern about a dearth of top class talent in our industry; Ron Mobed said they tackle this by recruiting outside our industry, pulling in bright people from other areas who would rather work for a company with a purpose. Philip Carpenter recognized the need to invest in top performers so they’ll want to stay, and for the job to be worthwhile and valued, with their contributions noticeably appreciated; this was echoed by Tracey Armstrong who advised giving people something intellectually stimulating to work on as that is highly motivating, with the whole team being close to the core mission. Brian Crawford felt similarly, that individuals who work for societies want to serve the field, and feel closely tied to the mission, a point also acknowledged by Steven Hall.

Speaking on gender diversity in the workplace, all acknowledged the issue of the scarcity of women at the top, as exemplified by the make-up of the panel itself, and all parties are actively working on this, some with outside help from consultants. Tracey Armstrong quoted an advisor who had told her it could take 7-10 years to diversify. She said that finding great mentors was key, and advised finding out what it’s like for women at lower levels of your organization: “talk to the women below the ceiling”.

The discussions left us all with quite some food for thought, and the women in the room departed with inspiration from Tracey Armstrong about advancing their careers and acting as leaders. Coupled with the insights into key initiatives from the morning, and all the great networking done at lunch and during coffee breaks, we all left richer for the experience.

    Chloe Tuck
Chloe Tuck
Assistant Editor, Technical Editorial

Getting rejected stinks. Wouldn’t it be great if we could appeal people’s decisions in life? Imagine asking someone on a date and getting rejected. What if you could submit an appeal letter explaining your argument with data to back it up? If only. Well, in science, you can.

As an Editorial Assistant, I’ve seen quite a few appeal letters submitted to my editors. While most are well-written scientific responses, some are emotionally driven criticisms of the nonscience kind. Rage and disappointment seep off the page. Scientists have an incredible amount invested in their work but personal attacks on the editors or reviewers will not overturn a rejection. In this brief piece, it is my aim to explore appeals in the scientific community and their place in the peer-review process, and conclude with a bit of advice we can all take away.

Scientific work can be quite difficult. In fact, I’m of the opinion that we should all be out there thanking scientists every day. What they do is hard enough and, on top of that, they’re expected to publish, which is no easy feat.

Dr. Daniel Kohane, Associate Editor of Nano Letters, likens publishing to dating: “Quite frankly, one accepts the fact there is a certain amount of subjectivity. It’s very much like dating. You have a general sense of what league you’re in. Most authors take rejection as part of the game.”

Of course, this is much easier said than done. In an article about her journal’s appeal process, Dr. Jillian Buriak, Editor-in-Chief of Chemistry of Materials, writes that getting rejected isn’t easy: “Having one’s paper declined is far from a pleasant experience and, as we know personally, the experience stimulates a range of emotions and reactions.”

I am certainly not faulting authors for the deep disappointment and frustration that a rejection letter can cause, nor am I suggesting that authors should refrain from appealing. As Dr. Kohane notes, “people have a right to appeal. The paper is the fruit of their hard work. They have a right to advocate for themselves.”

Editors and reviewers are people just like authors. In other words, they make mistakes. Sometimes, they miss a point and an appeal gives the author an opportunity to expand upon that point. As Dr. Paul Weiss, Editor-in-Chief of ACS Nano, explains in an editorial that gives a breakdown of appeals handled by his journal, “Sometimes, we overlook a key aspect of submitted work; we have found that appeals help us identify these papers, and several have ultimately been published.”

Dr. Kohane personally experienced this when he appealed the rejection of a paper he coauthored: “One of the reviewers just completely shot [the paper] down. It was just a rogue reviewer—as if someone said, we don’t need antibiotics. In fact, the paper was accepted.”

If the editors or reviewers have missed a point, it could be that the authors need to revise their work. In an article on the review process for his journal, Dr. Prashant Kamat, Deputy Editor of the Journal of Physical Chemistry Letters (JPCL), writes that “authors need to realize that the origin of this misunderstanding may lie in his/her presentation of the results.” Dr. Weiss echoes this sentiment, noting that, “As scientists and authors, it is up to us to make ourselves understood in our writing and otherwise.”

I spoke with a Managing Editor/Freelance Scientific Writer who talked about the value of appeals from an author’s standpoint. For the authors, appealing is always worth it. Authors may wish to challenge a disappointing decision if the reviews in the decision letter appear bland. It’s possible that the reviewers shared more detailed information with the editors that was not included in the rejection letter. Moreover, the often nuanced language in rejection letters can be a big obstacle to non-native English speakers.

Appeals are helpful to more than just authors. They can occupy an important place in the scientific community. In his article on appeals in ACS Nano, Dr. Weiss highlights their importance: “Appeals effectively give us a series of live case studies of how we understand the state of nanoscience and nanotechnology, as well as a way to move ACS Nano forward.”

We’ve seen that appeals can and should be included in the peer-review process, but what should they look like? As I mentioned, I’ve seen attacks on the editor and the reviewers as well as personal notes that do not deal strictly with the science at hand.

In his article, Dr. Kamat explains that authors should refrain from making nonscientific remarks, such as “the reviewer is not qualified” or “he/she has a biased opinion.” Dr. Kohane expresses similar concern about the importance of being professional: “Appeals that are not successful are those that are scientifically weak and where either side is disrespectful or unreasonable.”

A Managing Editor I spoke with agrees that an effective rebuttal letter should be professional and objective, provide data refuting the reviewers’ concerns, and include evidence supporting any claims of bias. The letter should thank the editors and the Editorial Board for their expertise.

My colleague notes that the appeal letter “gives points not understood by the reviewer, does not take the one positive reviewer comment and run with it. It deals with the real content of the reviews and addresses it now or says we can address it.”

What these letters should avoid are personal attacks, attempts to identify reviewers, emotional appeals, and cosmetic changes. Dr. Kohane echoes this sentiment, explaining that appeals should be polite and constructive, or rather, factual: “Politeness is important because the editor has put a lot of time and effort [into reviewing the paper], and to be rewarded by a rude email, that just doesn’t help. Also, odds are the editor wants to help you but needs rational ammunition to do so.” This ammunition does not come in the form of attacks. It comes directly from the data.

One Managing Editor/Freelance Scientific Writer I spoke with concurs that “generally, the Editor-in-Chief is on your side.” Editors will help if there is a reasonable argument for further review, rooted in the work itself. Dr. Weiss, in his article on how appeals are handled in ACS Nano, recommends highlighting the novelty of the work and the broad interest of the work in addition to addressing referee comments.

So, what should authors know going forward? Being polite is key. Sticking to the facts and avoiding personal attacks is also imperative. Appeal processes may vary depending on the journal. Authors should keep in mind that the response time frame may also vary, as additional input from specific content editors or reviewers may be necessary. It is important to remember that Editorial Offices cannot change the decision on a manuscript; rather, they can be used as a resource for the process.

It should also be stressed that, as pointed out by Dr. Buriak, editors and reviewers aim to treat manuscripts the way they hope their own would be treated. Editors and reviewers understand what it’s like to be on the other side of the decision letter.

As a final takeaway, a Managing Editor/Freelance Scientific Writer I spoke with recommends that authors should wait 24 to 72 hours before responding to a decision letter—then re-read the email. This simple process will remove much of the personal bias that could pollute appeals letters written in rage or disappointment.

What allows the peer-review process to operate honestly and effectively is for authors, reviewers, and editors to respect their distinct roles and to appreciate each other’s profound contribution to published works and the general scientific dialogue. Rejection is disappointing, but remaining polite and professional is essential. Courtesy occupies an important place in the peer-review process. It is my hope that all involved in the peer-review process will pause, think, and reflect on the points raised here before writing or reviewing their next appeal letter.

The article above is republished with permission from Editorial Office News, published on behalf of ISMTE.

    Jaslyn Tan
Jaslyn Tan
Senior Marketing Manager, Wiley Singapore

We recently spoke with Dr. Tadahiro Takada, founding President of the Japanese Society of Hepato-Biliary-Pancreatic Surgery, to get his insights on the use of mobile apps to facilitate clinicians' decision making and improve clinical practice.


Q. Can you tell us more about the rationale/reason for developing an app? What were your objectives?

A. We developed “Biliary Tract Cancers Classification (BTC C 2015)” and “Biliary Tract Cancers Guidelines (BTC GL 2015),” with the aim of allowing biliary tract cancer specialists to make judgments directly linked to clinical practice, and to concurrently facilitate revisions to new international codes.

The content of this mobile app was simplified and clarified, with the intention of promoting an easy way for health professionals to make judgements in clinical settings, and to improve their clinical practice for patients.

Q. Who’s the target audience?
The target audience consists of cancer specialists and other health professionals involved in the clinical practice for patients with biliary tract cancer. We hope that this app will provide more information and deeper insight for patients and their families when they are seeking a decision on the available treatment options.

Q. What are some of the benefits to clinicians?
This app enables users to determine the cancer stage using the checklist function in BTC C 2015, as well as to email the input checklist as findings.

Furthermore, BTC GL 2015 is designed to allow immediate access to necessary information with minimal operation, such as linking to clinical questions corresponding to diagnostic and treatment algorithms.

The app is available in English as well as Japanese; therefore, it can be used by health professionals worldwide.

Q. What challenges did you face as you developed the app?
The most difficult challenges were figuring out how to create an app that allows users to easily and clearly understand important medical matters, and how to promote simplified and accurate medical judgments on diseases.

Q. Why did you chose to partner with Wiley to develop this app?
Wiley is a partner in the publication of the Journal of Hepato-Biliary-Pancreatic Sciences, which is the official journal of the Japanese Society of Hepato-Biliary-Pancreatic Surgery, and has also published papers in this journal that form the basis of this app. Because Wiley already has a shared information system and is skilled at creating mobile apps, we commissioned Wiley to develop this app.

Q. What is your advice to fellow society officers who might be interested in developing an app?
An important role of medical societies is considering how to clearly convey the latest medical information to health professionals and patients. Therefore, clinical practice guidelines and classifications should be widely disseminated. It is also important that such information is readily available in clinical settings; therefore, potential developers must bear all of this in mind when they work on the development of a mobile app.

Q. Did the final product achieve your objectives?
We are satisfied with this app, but globally speaking, there are not enough clinical practice guidelines or evidence for biliary tract cancer. In addition, as scientific knowledge continuously advances, new insights that are not included in this app may emerge. At the Japanese Society of Hepato-Biliary-Pancreatic Surgery, we aim to provide the latest information by continually updating these guidelines and classifications. We strongly hope that this app will benefit many patients suffering from biliary tract cancer and assist the clinicians treating these patients by sharing this app worldwide, rather than focusing on a single country.

Both apps are now available for download from the App store or Google Play.

Dr. Tadahiro Takada Source: Japanese Society of Hepato-Biliary-Pancreatic Surgery

Filter Blog

By date:
By tag: