{"objectType":14,"id":2014,"valid":true}
2015

Peer review around the world

Posted Sep 30, 2015
    Alice Meadows
Alice Meadows
Director of Communications, ORCID
Source: Vetta/Getty Images
Source: Vetta/Getty Images

We talk a lot about peer review in the scholarly communications world. Many of us – and our organizations – are working to improve both the process and the experience for researchers, which has led to a significant increase in the range of options available, especially – but not exclusively – for reviewing journal articles. From double blind to completely open review, pre- and/or post-publication, and even transferrable peer review, not to mention the work being done on peer review recognition and validation by organizations like Publons and PRE, there’s a plethora of new approaches and services to choose from.

But what do researchers make of all this? What are their experiences of peer review? How and why do they review themselves, and what do they get from reviews of their own work? In this reflection from researchers around the world, we asked some of them to tell us about their views of peer review.

By and large, their feedback was very positive, with good experiences outweighing bad and universal agreement that peer review is, as Elizabeth Briody of Cultural Keys, USA, says: “a critically important process for evaluating the merit, content, relevance, and usefulness of scholarly publications” – or as Hugh Jarvis, Cybrarian, University at Buffalo, USA, describes it: “Peer review is the glue of academic publishing.” Saurabh Sinha, Executive Dean, Faculty of Engineering & the Built Environment, University of Johannesburg, South Africa agrees that: “it positions our work with respect to the body of already published knowledge. The approach also helps to ensure, as far as possible, the correctness of the work, elimination of potential blind spots, and validity of assumptions for a practical world.”

Pretty much everyone noted the importance of peer review – both as reviewer and author - to them personally as well as professionally. For example, Professor Yongcheng Hu, a medical researcher in China commented that: “Peer review is an essential arbiter of scientific quality, no doubt, it has a great impact on scientific communication and is of great value in determining academic papers’ suitability for publication, while for me, via personal experience, it is also an process of exploration and sublimation.” Erik Ingelson, Professor of Molecular Epidemiology at Uppsala University in Sweden, currently Visiting Professor at Stanford University, USA adds: “Mostly, my experiences of being a reviewer have been positive; I get to think critically about study design and methods and learn new things on the way. Similarly, most of the time the review process is positive also as the author, since you get valuable input and the paper that comes out is often better than the original submission.” Anna Cupani, a Belgian researcher, agrees: “Having someone reading and commenting on your research is beneficial for several reasons: it validates your work, it confirms what you are doing is meaningful not only for you but for a wider scientific audience and it helps you focus and improve your research. You never grasp the meaning of something as deeply as when you have to explain it to someone else!” And Lee Pooi See, Associate Chair (Research), School of Materials Science and Engineering, Nanyang Technological University, Singapore adds: “My personal experience of being reviewed has been interesting; especially in receiving scientific viewpoints from different reviewers on emerging topics. Peer review also steers us to identify those unaddressed aspects of the related research topics

Several people also commented that there are upsides and downsides to peer review. Janine Milbradt, who is currently working on her PhD at the Institute for Human Genetics, University of Cologne, Germany, says: “You never know what is going to happen! All you can be sure about is that you will have to put another 3-6 months of work into your paper. Having a paper reviewed is a nerve-stretching process, filled with hopes and dreams about the reviewers actually liking your research. On a more serious note, the review process is a very important tool to find incomprehensible or knowledge lacking parts of your research to improve your paper.” Professor Wong Limsoon, KITHCT Professor of Computer Science, National University of Singapore comments: “I appreciate very much constructive reviews that gave me really useful suggestions on my work. I am sometimes annoyed by uninformed comments, but fortunately these are few.”

So what improvements to peer review would our group of researchers like to see? To quote Professor Sinha again: “Scholarly peer-review has…the opportunity to improve beyond the past, where today, coupled with data, crowd-sourced reviews/discussion, newer open-access technologies could play a dynamic role of developing credibility of research-work and at the same time increasing competition!” Hugh Jarvis likewise has “great hopes that peer review will develop a much more expanded role in the future, and provide input before and after publication, similar to the role the comments serve in Current Anthropology and the product ratings in sites like Amazon.com.” And Joao Bosco Pesquero, Professor, Federal University of Sao Paulo, Brazil would also like to see a more open approach: “The more openly we produce science and expose our work to criticism, the more it helps to improve what we do.”

Perhaps the best summary of why researchers continue to value peer review – both as authors and as reviewers – comes from PhD student, Grace Pold of UMass – Amherst, USA, who told us: “Although I have had the opportunity to formally review only four or five papers, reviewing papers is one of my favorite things to do. First off, it is a good reminder that not all papers are born perfect, and when I am struggling to try and finish my own work and the prospect of a well-polished manuscript seems too far in the distance, it gives me hope. Second, is there a better opportunity to see what your colleagues are working on and thinking about than by reviewing their work? Third, the idea of being able to help shape the information released into the public sphere is a very enticing. Fourth, it is a great excuse to really think about the assumptions you and others make in your research…when you review, it is your responsibility to stop and think about why this is the way things are done. Fifth, thinking up alternative interpretations and then filtering through the data presented in the paper to determine the robustness of the conclusions is a rewarding challenge. Finally, reviewing papers provides an opportunity to slow-down and formulate a full, well-rounded opinion on something, something which happens unfortunately rarely in the life of the frantic modern scientist stuck in with the nitty gritty details of doing experiments. And I think that from a personal perspective, that final point of generating a sense of accomplishment in doing a good job in thinking things through to the end is probably the greatest motivation for me to review papers.”

Peer review - a discussion

Posted Sep 29, 2015
    Alice Meadows 
Alice Meadows
Director of Communications 

To celebrate Peer Review Week, representatives from ORCID, ScienceOpen, Sense About Science, and Wiley got together (virtually) to discuss why peer review matters - to them and to their organizations - and what they hope this week will achieve.

Participants: Stephanie Dawson (CEO, ScienceOpen), Chris Graf (Wiley), Peter Gregory (Wiley), Laure Haak (ORCID), Emily Jesper (Sense about Science)

 

Source: Stephanie Dawson
Source: Stephanie Dawson

Stephanie Dawson (SD): When I started off as a journal manager at De Gruyter publishers, I used to receive three copies of each manuscript, which I put in envelopes and sent to Japan or Argentina by post and hoped that the chosen peer reviewer would have time to review. Much has changed since then and much not. Some reviewers did an extremely conscientious job in helping authors improve their papers, others sent a rejection in three lines. I really saw firsthand how heterogeneous the process could be and wished I could personally thank each reviewer who put in the effort to really help their colleagues.

With the research and publishing network ScienceOpen, one of our distinct roles besides a Gold Open Access (OA) publisher and content aggregator, is that of a peer review reformer. Our goal is to augment trust in the peer review process by making it entirely transparent. Then everyone can see (and even thank) those reviewers who are doing an exceptional job to improve the quality of scientific communication.

We’ve deployed a novel workflow to demonstrate the efficacy of a different approach to Peer Review and its suitability for the digital future of scholarly communication which we believe will need to speed up and involve the publication of many more digital objects than the current single article unit.

 

Source: Chris Graf
Source: Chris Graf

Chris Graf (CG): Speaking of novel workflows, I’m personally interested in how pre- and post-publication peer review in combination will make a better system.

Pre-publication peer review validates and organizes research, and in the main it does a great job. But – let’s be honest – sometimes it goes spectacularly wrong. And that’s where post-publication peer review comes in.

Post-publication peer review can help contextualize, curate, and – excitingly for me – perhaps also gives a more accessible entry point for readers (eg, comments in PubPeer, PubMed Commons, metrics and more in Kudos, provide a steer for readers and help surface articles). And – yes – post-publication peer review weeds out the occasional bad apple. Which is important.

Last, we need to stop and think – and have a public discussion – about whether we should view a peer reviewed article as absolute, definitive, final and 100% correct just because it’s peer reviewed. Might it be more realistic to think about a peer reviewed article as one step on a (fairly tortuous) research journey, upon which we can expect to make missteps, take wrong turns, stumble, and take new directions before we reach our final destination: The Answer. Many steps make up that journey, and many pieces of research make the evidence we use to inform practice and policy.

 

Source: Laurel Haak
Source: Laurel Haak

Laure Haak (LH): I completely agree with your last point but, in terms of validation, is that really what peer review does? Is it supposed to be the filter that says, this paper used the right methodology and statistics (in the case of a science paper) or are we really looking for the peer reviewer to assess the logical flow of the paper? At the end of the day, whose responsibilty is it if the paper is found to be "fraudulent"? Can we really expect/do we want that weight to be on the reviewer?

 

 

 

 

 

Source: Emily Jesper
Source: Emily Jesper

Emily Jesper (EJ): And to Chris’s point about peer review’s role in informing practice and policy, of course peer review is not just something of significance to scientific researchers. Because it indicates that research has been scrutinized by independent experts in the field, peer review is an important consideration for policy makers, reporters and the public when weighing up research claims and debates about science. Understanding the status of research claims is empowering. It helps us weigh up claims and use evidence to make decisions. Since Sense About Science was set up in 2002, we have been working to popularize an understanding of peer review more widely. Our public guide to peer review I Don’t Know What to Believe encourages people to ask “Is it peer reviewed?” when weighing up claims about science.

 

Source: Peter Gregory
Source: Peter Gregory

Peter Gregory (PSG): Although you’re right that peer review does have a wider significance, it matters to Wiley primarily because it matters to the scholarly community which we serve. In addition to gathering opinion and improving contributions peer review also provides, especially in the sciences and medicine, a safety valve, preventing the distribution of incorrect or even dangerous erroneous articles.

LH: Yes, peer review is a core component of scholarly practice.  It is an established set of methods to gather commentary on scholarly works (papers, books, grants, programs, and more) from peers -- experts in the field -- with the end goal of improving the work.  ORCID is interested in peer review because people are involved, both creators of works and the reviewers.  We are working with the community to develop digital methods to acknowledge the contributions of reviewers by citing peer review activities. Through this we are hoping to encourage scholars to participate as reviewers, and also to support others who are working on issues of trust in the review process.

SD: I love the idea of making ORCID the place where researchers can log their reviewing activities. ScienceOpen is trying to address this with open reviews that have a registered CrossRef DOI.  Publons is also doing a nice job of this as well as supporting reviewers in blind peer review journals. In cases of blind review they are verifying with publishers that a reviewer has indeed done the work that they claim – right now it would be easy for a researcher to add “review for Nature and Cell” to their CV because there is no way to verify this in our current blind review system. Again trust and transparency are key.

PSG: I hope that this Peer Review Week will help intensify discussion of peer review and start sensitizing participants (publishers, authors, editors, referees, funders etc) to the differences in the ways peer review is applied and is useful in different communities. Some areas collect opinions, some are more fact based; some benefit from preprint circulation (prior to peer review and publication I mean) some are damaged by it; for some post publication review is innovative, for some it is dangerous. This leads me to the conclusion that those working to improve/change peer review should carefully consider that one size does not fit all.

SD: I agree that there will not be a “one size fits all” solution but different communities can also learn from each other, especially as an increase in interdisciplinary research draws them together. One great example is the life sciences preprint server BioRxiv which builds on the physics community’s positive preprint experience with arXiv. In the new field of bioinformatics, the computer scientists just put their preprints on arXiv as a matter of fact, and the researchers  from the life sciences asked “what’s that?” So something really interesting may grow up just at those points of friction. Also the new trend towards megajournals such as PLoS One, SpringerOpen, Scientific Reports, requires rethinking peer review to fit a wide range of communities and all of those journals have come down to asking reviewers to just review the scientific soundness of results. So this may give us some clues to what the common denominator of peer review may be moving forward.

LH: The idea of a common denominator is really key and is exactly the sort of discussion I hope Peer Review Week will generate – stimulating and concentrating the ongoing conversation on peer review and, in so doing, bringing forth what is being done in practice to support improvements.

To read more of this discussion, visit the ORCID blog.

Don't forget to join the conversation online using #peerrevwk15.

Do you want to find out more about Trust and Transparency in Peer Review? Join expert speakers from academia and publishing for our FREE 1hr webinar this Thursday, October 1st.

Peer Review Week arrives!

Posted Sep 28, 2015
    Verity Warne
Verity Warne
Senior Marketing Manager, Author Marketing, Wiley

Welcome to  Peer Review Week! To kick off the celebration of all things review, here’s a quick reminder of the central role peer review plays in scholarly communications. Below are our tenets of peer review:

 

Peer review makes science better - Multiple industry wide research reports (the 2008 Ware & Monkman report, Sense about Science’s 2009 survey and CIBER report from 2013) show that the majority of researchers feel that peer review improves the quality of their published papers.

 

Peer review is the central pillar of trust for researchers - Researchers want to be published in journals that have robust peer review and they feel secure in citing peer reviewed material. (CIBER, 2013)

But, our own research shows that peer review is not just important in a broad sense…

 

An author’s experience of peer review shapes his/her overall publishing experience - The quality of the peer review experience is a crucial factor in an author’s decision about where to publish:

Speed to online publication is ranked as the 2nd most important benefit by Wiley authors after an online submission system. And, the perceived quality of the peer review process consistently appears as one of the top five most influential factors for authors in deciding where to submit their paper.

Beyond this, authors who express the most satisfaction with their publishing experiences are those that state they’ve had an easy time with the review process..

On the flip side, the authors expressing the lowest levels of satisfaction are those who experienced a difficult review process and struggled to communicate with their reviewers. The primary problems? The review took longer than expected (35%) and authors struggled to understand what the reviewers wanted (35%). Notably, 53% of reviewers in this category are early career researchers.

But, what about reviewers themselves? It won’t come as a surprise that a lot of researcher time is spent reviewing. Our research indicates that 5 in 10 reviewers are actively reviewing for five or more journals at any given time. Experienced reviewers (those with more than five years of reviewing experience) shoulder even more of the burden with 61% reviewing for five or more journals at any one time. Taking just the top 12 publishers alone, we estimate that 55.2 million researcher hours were spent reviewing papers in 2013. That’s a big time commitment!

It’s our job, as publishers, to protect the integrity and continue to improve the review process - Researchers believe that organizing and managing peer review is the                crucial role of publishers. (CIBER, 2013)and this is a job we take very seriously. Some of the ways in which Wiley is seeking to improve the efficiency of the process, include:

 

    • Investing in best practice peer review processes. In 2014 we consolidated the peer review experience within our business into one global team of 54 colleagues who manage the editorial offices for around 170 journals, working with our academic editors and applying best practice in order to make peer review as efficient and robust as possible.

 

    • Evolving the peer review process: The vast majority of our journals use either single or double blind review, but we also offer newer forms of review across a number of journals. We are exploring how to reduce the amount of repeat reviewing by innovating around transferable peer review. In addition, eight journals offer open review models, four use a collaborative review process and we also have a journal which uses a “bidding” process whereby reviewers bid for the paper they wish to review.. The infographic below provides a birds eye view of the peer review models across Wiley journals.

      peer review infographic

    • Providing training and best practice guidance to peer reviewers: In 2014 alone we ran about 200 author and reviewer in-person workshops in 17 countries. We are also experimenting with mentoring schemes for journals such as Journal of Morphology and Austral Ecology. Last year we launched our peer review webinar series, our peer review resource center and a regular series of peer review posts on this blog.

 

    • Improving recognition of the contribution made by reviewers: The ways reviewers currently receive recognition varies from journal to journal, and include recognition in a journal’s official list of reviewers, letters or certificate of contribution from the journal editor, and book or APC discounts. We’re also looking into other ways to acknowledge a reviewer’s contribution and currently have initiatives underway which look at recording reviewer activity as a measurable research output. For example, we are undertaking a pilot which enables peer reviewers for selected Wiley journals to receive credit for their peer reviews on Publons.

 

This week, we’re looking forward to some great conversation and community-driven discussion about – and celebration of – peer review. Join the conversation at #PeerRevWk15.

Look out for:

 

 

peer review at wiley.PNG

    Helen Eassom
Helen Eassom
Author Marketing, Wiley

Global challenges require a global response. Climate change, energy security, scarcity of water resources, global health and disease, and food supply and nutrition are among the biggest challenges facing humanity today. Tackling such huge challenges requires collaboration on an international scale, sharing resources and knowledge across communities. It is only through working together that we can even begin to hope to address these global problems.

 

global challenges.PNGglobal challenges.PNGThat is why Wiley is excited to announce the launch of Global Challenges, a new, premium open access journal. Global Challenges is a unique journal that will cross disciplines and international borders, bringing science, technology and the social sciences together to mobilize debate and create a platform for directing and setting research and policy agendas. The journal will initially focus on five major challenges, with each area overseen by Chief Editors: Climate Change (Georg Feulner, Potsdam Institute for Climate Impact Research), Energy (Peter Lund, Aalto University), Water (David Butler is Professor of Water Engineering and a Director of the Centre for Water Systems at the University of Exeter), Global Health (Steven J. Hoffman, University of Ottowa and John-Arne Røttingen, Norwegian Institute of Public Health, Harvard University and University of Oslo) and Food, Agriculture & Nutrition (Spencer Henson, University of Guelph). We’ve spoken to some of the Chief Editors about why they decided to get involved in the journal, and why a collaborative approach is so important.

Q. What made you decide to get involved with Global Challenges?

Georg Feulner, Potsdam Institute for Climate Impact Research, Chief Editor- Climate Change:

Traditionally, research on global challenges such as climate change or food security is published in specialized disciplinary journals. I find the approach to include several challenges in one journal fascinating and important, because it allows us to take a truly global look at the most pressing issues and to identify cross-connections between different challenges. And I do like the idea of detailing the relevance of each paper for policy makers as this could help to translate the relevant research findings into policy.

Q1

David Butler, Professor of Water Engineering and a Director of the Centre for Water Systems at the University of Exeter, Chief Editor- Water: My interest in the journal was piqued because I saw the opportunity of cross-fertilization and impact.  We live in our little disciplinary silos and independently try and tackle these global challenges, when if we talked to each other we might well find common solutions.  The journal also has a clear policy/impact agenda, which again is refreshing and quite different for an international journal of this type.

Peter Lund, Aalto University, Chief Editor- Energy: Global Challenges is a highly relevant and timely effort. The publishing world is increasingly filled with highly specialized journals, but seldom do we see efforts to tackle global challenges as a whole. In energy, which is my specialization, looking beyond one’s own discipline is a must to understand how we could solve the global energy challenges. When the opportunity to join Global Challenges was offered to me, it was just a perfect fit to the on-going discourse in energy. For me, getting involved with the journal is to be part of the solution to the grave global problems.

Q2

Q. How important do you think it is to take a collaborative approach when attempting to solve the kind of challenges the world faces today?

Spencer Henson, University of Guelph, Chief Editor- Food, Agriculture and Nutrition: The complexity of many of the challenges faced by the world today requires collaborative thinking across the natural and social sciences and humanities.  No discipline alone can provide effective, practical and sustainable solutions to these challenges, reflecting the multitude of technical, economic, social and political issues that need to be addressed.  The persistence of disciplinary boundaries, that dictate where researchers are housed, who they engage with and where they publish, has stood in the way of this needed collaboration for too long.  The availability of high-impact journals that publish cutting-edge collaborative research will play a substantive part in overcoming these boundaries.

Q3

David Butler: I think a collaborative approach is key.  No one discipline can possibly hope to solve the many challenges the world faces.  Currently there are rather few practical ways of working together across disciplines and indeed between academics, policy makers and practitioners.

Q4

Georg Feulner: A collaborative approach is clearly very important, in particular for a topic like climate change which is connected to each and every one of the other challenges covered by the journal. As I write in the editorial  to the climate change section of Global Challenges: "Climate change has the potential to impact global water supplies, agricultural production, human health, and our energy infrastructure. In turn, the way in which we produce our energy and food has a profound effect on the Earth’s climate system."

Q. How do you think researchers can best organize themselves to tackle global challenges, and how will the journal feed into that overarching goal?

Peter Lund: Organizing collaborative efforts starts by someone taking the initiative and showing the way how to proceed. The Global Challenges journal is such an initiative which helps to create an international platform for multidisciplinary thinking to tackle global problems. Importantly, the journal will help to gather together different stakeholders of the global problems such as scientists, practitioners, and policy makers, among others.

Spencer Henson: Most critically, there is a need for researchers across the natural and social sciences and humanities to engage with one another in order to understand the complexity of factors underlying the challenges facing the world today and to identify effective and practical policy solutions.  While there is a broader need for disciplinary boundaries to be lowered and incentives put for interdisciplinary and policy-focused research, there is a more immediate need for high-ranking journals that publish such research.

Q5

Want to learn more about this exciting new journal? Watch the Global Challenges video and visit www.globalchallenges.com

    David Perlmutter
Dean of the college of Media and Communication, Texas Tech University

Because I have written about promotion and tenure and the academic career track for more than 15 years, whether in books or magazine essays, I have gotten a chance to talk to hundreds of doctoral students and tenure trackers. Their fields run the gamut of the academy, from philosophy to biochemistry to psychology.

501886143_294735893_294735894_256224451.jpgBut over and over one observation has recurred: Doctoral programs usually do at least a decent job in training you to become a researcher and sometimes a good job in training you to be a teacher. But across the board, especially for those who have no family experience of the professoriate, the people and politics of getting a Ph.D., a tenure-track job, and then tenure are surprising, disconcerting, and puzzling. As one bench scientist put it, “I know what to do with my centrifuge, but nobody told me how to deal with the lunatic who’s the head of the promotion-and-tenure committee.''

Just because you feel like a deer in the headlights, however, doesn’t mean that you, a sentient, highly-educated human, have to freeze up. Yes, many factors, wholly and partly out of your control, affect your success in academic careers. For example, you may go on the job market in a year when there are few openings in your area, and several of them are fixed, with no chance of your getting them. But I have seen and heard too many young scholars whose careers have been tripped up not by supervillains or a rigged wheel of fortune but by their own inability or unwillingness to embrace a central wisdom of becoming a professor.

In brief, to paraphrase Dickens, you are the hero of your own story. People may try to trip you up or help you out, but you have to train yourself to become as savvy, shrewd, planning-oriented, realistic, and soberly self-appraising as possible. You will never be able to discount the malevolence of others or bad luck, but just like a blackjack player, knowing how to play the game like an expert can reduce the casino’s odds against you. Specifically:

Find and seek out active mentors and indirect role models…for specific goals. Young academics are often given the advice to find good advisers or mentors. The problem is that they forget the wisdom of the Roman historian Livy: “The gods do not give all their gifts to one man.” A good mentor for classroom teaching may be a poor one for research and an uninformed one for putting together a tenure packet. Alternately, there are ways to take someone’s advice without asking for it. Look at the cohort of successful doctoral students and tenure trackers in front of you via their CVs. What did they do and how often did they do it?

Think ahead. You plan out your research, you prep for your classes, but do you set goals and benchmarks for your career? Make sure you have thought out exactly what you need to achieve and by when you need to achieve it. In terms of productivity for tenure, for example, pace and timing are often as important as raw quantity.

Become your own best critic. Two major character flaws mark those who tend to stumble the most in doctoral programs, the job hunt, and the tenure track. First, they are the “fragile Frankies” (taken from a Seinfeld character) who wilt under any criticism whatsoever, from anyone, including themselves. At the other end of the spectrum are the “full-metal arrogants,” who ignore any criticism, bad or good, useful or unhelpful. Learn to gauge whom to trust and the value of a good critique of your work and even your behavior.

Finally, find the middle path between confidence and delusion. Don’t listen to the Disney movie mantra: You can’t achieve anything you want simply by force of will and pixie dust. Create rational, reasonable objectives—a dissertation that can be completed given the time, resources, and finances available, for instance—and then pour all your strength and effort to making sure the goal is met. Don’t dream the impossible dream, but don’t sell yourself short either.

It is difficult to succeed today in a doctoral program through getting promotion and tenure; so much more the reason to plot your path to success carefully and thoughtfully.

Image Credit:Getty Images

    Nat Winn
Nat Winn
   Marketing Manager, Society Strategy and Marketing

A couple of weeks ago, several Wiley colleagues joined peers from the publishing community at the ALPSP Conference, to share information and knowledge and participate in discussions on both the challenges and the opportunities facing the industry.

We asked a few of these colleagues to tell us what stood out for them among all the information packed into two days. Here’s what they shared:Davina Quarterman, Associate Director, Society Strategy and Marketing at Wiley was particularly interested in discussions around factors affecting the research community:

Changes in behavior

Changes in researcher behavior will impact on the design and usability of content - millennials are taught early on in school to scan and skim long form; so this will inevitably impact the consumption of research content. A number of conference speakers called for improved abstracts and layman summaries.

Discoverability of content

Microsoft is making great strides in improving the discoverability of content for the research community. They’re thinking not only about the semantic web, but the knowledge web, where human intent is factored into the result. Kuansan Wang, of Microsoft Search, talked about the possibility of machines reading content and learning from it in the same way that humans do. Kuansan also showcased developments to Cortana (Windows Phone personal assistant).Integrating data will allow Cortana to provide more relevant content, research and events. new and noteworthy.PNG

Kate Smith, Director, EMEA Library Marketing, Wiley, noted some stats that really are food for thought:

Scholarly Search

University of Illinois Urbana-Champaign (UIUC) cited that 50% of library searches are 1-3 words long at UICU, which isn’t much to go on and 10% of logged UICU searches are single words. These stats really highlight the importance of search behaviors; researchers aren’t framing the questions they have but expecting systems to still deliver results.

Additionally Kate noted that publishers have the challenge of getting their metadata in good shape in order to respond to changing consumption demands and deliver information in a future-proof, flexible way.

OA Predictions – China

In the Global Positioning plenary, speaker Dr Xiaolin Zhang, National Science Library, Chinese Academy of Sciences discussed OA to support Open Science and Open Innovation in China and indicated where they are heading with open access:

 

    • Green OA: They anticipate a wide spread of agency and institutional policies, best operational practice guidelines within three years, a smooth ecosystem for deposit such as iSwitch and a conformance check and enforcement within five years

 

    • Gold OA: They foresee funding for domestic STM journals, experiments and established services to fight against predatory journals within three years and collaborative bargaining about APC levels within five years

 

Jenny Neophytou, Bibliometrics Manager, Market & Publishing Analytics thoroughly enjoyed the keynote talks by Anurag Acharya (Google Scholar) - What happens when your library is worldwide and all articles are easy to find, and by Kunsan Wang (Microsoft Research) – From web publishing to knowledge web publishing. Jenny mentioned the sessions could be characterized as ‘Google vs Bing and underscored some key differences in personalization between the two in regard to research articles.

Google vs Bing

It has long been known that Google ranks your search results according to your internet search history, yet Acharya revealed (to the surprise and, frankly, disbelief, of many attendees) that Google Scholar is free from personalization. Whether this is a benefit or a hindrance depends on your perspective. Yes, this makes it more difficult to locate what you’re looking for – but it has the benefit of returning records without your own views/perspectives coloring the research you retrieve. No academic wants to live in an echo-chamber.

By contrast, Microsoft Research revealed a host of Bing upgrades, including extensive personalization that enables Bing to ‘predict’ what the user is searching for. Not only that, but it is able to summarize research articles for you – not merely provide the abstracts, but actually summarize the paper. How accurate this is remains to be seen, but the demonstration by Wang was extremely impressive.

Funder and Government Policy

And last, but not least, the Global positioning plenary really got Duncan Campbell, Director of Journals Digital Licensing, Wiley, thinking about how funder and government policy is becoming increasingly important in driving change in the industry – perhaps without being aware of (or interested in) possible negative or unintended consequences.

If you attended the ALPSP Conference, we’d love to hear about your highlights in the comments.

Image Credit/Source: Kuansan Wang, Microsoft

Exploring the peer review process

Posted Sep 22, 2015
    Roger Watson
Roger Watson
Editor, Nursing Open and Journal of Advanced Nursing

shutterstock_93625837_271824079_271824080_256224451 (1).jpgThe review process for manuscripts submitted to peer review journals is one of the most common issues I get questioned about when I meet authors. I am asked what type of review process my journal uses and what the purpose and advantages and disadvantages of the various review processes are. I think these questions arise from curiosity and poor experiences with submitted manuscripts. Also, some authors expect a degree of certainty in the process which, as editors, we cannot provide. For example, I am often asked why one journal will reject a manuscript and another will accept it. My answer is that peer reviewing is not a scientific process; it is a process based on people and the judgements they make; people differ in their expertise, opinions, and experience. I also emphasize that reviewers for peer review journals do not make the decisions about which manuscripts to accept or reject; they provide a view on a manuscript which aids the editors in making a decision.

Types of peer review

There are, essentially, two types of peer review: closed and open. The former is more common, but the latter is gaining in popularity. Authors and reviewers will encounter both.

Closed peer review is a system where the identities of the reviewers are not disclosed in the journal or to authors, and the identities of authors may not be disclosed, during the review process, to the reviewers. Of course, the reviewers can identify the authors after publication. Closed review works in two ways: single-blind and double-blind. Single blind review works by revealing the names of authors to reviewers while withholding the names of reviewers from authors. In double-blind peer review—as described above—identities of authors and reviewers are mutually withheld.

Open peer review, in contrast, operates a more transparent approach to peer review. Identities of authors and reviewers are mutually disclosed and, furthermore, reviews are sometimes published alongside the published articles. This system is becoming increasingly popular and is often applied by open access journals.

Pros and cons

There is no consensus on which of these peer review systems is best and it is agreed that both closed and open peer review have good points and bad; likewise single- versus double-blind peer review. The principle behind closed review is to minimize the bias of reviewers who may be influenced by the identity of the authors and to protect the reviewers from authors who may take exception to adverse reviews and rejections. The principle of only protecting reviewers operates in single-blind review. While it is always possible to protect the identity of reviewers during and after the review process, it is often possible for reviewers to identify authors by virtue of the work that is being reviewed.

Moreover, even when the author cannot be identified, reviewers may take exception to a line of work for reasons that are not concerned with the science or because the work competes with or refutes some of their own work. Therefore, the advantages of the system—minimizing bias and protecting identities—may be undermined by prejudice on behalf of reviewers.

As an ‘antidote’ to some of the issues raised by closed review, open review introduces transparency. By mutually revealing identities, the potential for bias by reviewers is attenuated by accountability to authors and readers. The advantages of this system may be outweighed by less-than-honest comments from reviewers who feel unable to be frank about the work.. On the other hand, the potential for unhelpful and inappropriate comments is reduced. Neither of the above systems of review-closed or open-is capable of completely obviating the problems they are designed to address.

Final words

Some modern variants have been proposed and one is post-publication review. To a large extent, post-publication review had always existed as it has always been possible, for example, to correspond with authors and journals about their publications and authors may choose to publish refutations and rejoinders. In recent years the rise of social media has facilitated and accelerated the exchange of views on scientific publications. However, post-publication review is also proposed as a more radical and dynamic process whereby articles are published without pre-publication review and are altered thereafter in response to post-publication comments. This system has not yet ‘caught on’ but has supporters including Richard Smith, the former editor of the BMJ.

Another variation on the theme of closed review is triple blind review whereby, not only are authors and reviewers blind to each other’s identities but where editors are also blind to the identity of both. This is aimed at minimizing bias among editors but does not eliminate the possibility of identifying authors by their work or of bias against competing work. Again, there is little evidence that it is used commonly and it requires journals which can afford the administrative staff and processes.

The process of peer review, mainly in publishing but also in other aspects of academic life, came under the scrutiny of the British government and other bodies after some accusations about biased publishing in the field of climate science. The scrutiny was in-depth and prolonged. The conclusion was that the peer review system in its various manifestations was far from perfect, but that it was the best available and should continue.

For more on peer review processes, listen to my podcast on the subject.

Image Credit:Bizroug/Shutterstock

 

    Anne-Marie Green 
Anne-Marie Green
Communication Manager, Wiley 

We recently asked a few of our regular Exchanges contributors to share their summer reading choices with us, along with their thoughts on what they've read. You may pick up some great additions to your reading list, and feel free to share what you read in the comments below.

 

Source; Richard Threlfall
Source; Richard Threlfall

Richard Threlfall, Editor, Asian Journal of Organic Chemistry

Controversial and tremendously thought-provoking, top of my reading list this summer was Richard Thaler and Cass Sunstein’s Nudge (ISBN 978-0143115267). Thaler and Sunstein set out their ideas on how the “health, wealth, and happiness” of the population at large could potentially be improved by what they term “liberal paternalism”, that is, giving people a slight “nudge” in the right direction when making important choices while still maintaining complete freedom of choice. It’s a compelling idea that seems a powerful tool with which to improve life for a great many people, but the interesting questions that arise are how and when should it be used, and who should be using it? How do we avoid good intentions being hijacked by other interests and how do we ensure that nudging doesn’t become something altogether more sinister? This is a great book that changed my perception of some of the issues facing a modern society and it will have you on the phone to your financial advisor faster that you can say liberal paternalism.

 

Source: Elizabeth Lorbeer
Source: Elizabeth Lorbeer

Elizabeth Lorbeer, Library Director, Western Michigan University School of Medicine

If you’re a parent of a young child, you find that the books that your child selects for their summer reading program become your summer read.  My daughter and I clocked over 720 hours reading young adult non-fiction and fact books.  My daughter is fascinated by the planets and stars. After reading together all the books on the solar system held by our local public library, I realized the amount of discovery that has happened since my youth.   Each night when I look up at the night sky, I can tell you so much more about the world around us.  The capstone to summer reading for my daughter and I was watching the International Space Station fly over our home.  The Spot the Station website is located here.

 

Source: Wiley
Source: Wiley

Mark Allin, President & CEO, Wiley

I was just rereading a novel that I read when I was in Africa, Earthly Powers, by Anthony Burgess. It’s an 800-page romp through the twentieth century. Hugely entertaining, quite impactful, very sad. I read it again and thought, It’s 30 years later, and I still think this novel is stunning.

 

 

 

Source: Robert Dingwall
Source: Robert Dingwall

Robert Dingwall, Editor, Symbolic Interaction

http://onlinelibrary.wiley.com/journal/10.1002/(ISSN)1533-8665I revived a long-standing passion for science fiction, partly in the context of trying to contribute to sociological thinking about the future. There is a big problem with the way governments approach major infrastructure investments by projecting recent trends – ‘upwards and to the right thinking’ – which lacks resilience in the face of technological innovation or environmental change. Advocates of ‘science fiction prototyping’ think one alternative might be to use literature to ask ‘what if’ questions about how relevant big projects will be in fifty years’ time.

Apart from revisiting some old friends, like Dune, I discovered the American writer, Connie Willis, who explores some implications of time travel. History would become an empirical study and graduate students would write dissertations by journeying into the past. Unfortunately the technology is not wholly reliable so they tend to get stranded. Willis is very good at creating a sense of jeopardy around characters who are essentially contemporary young Americans encountering the challenges of everyday life in the past. The best is Doomsday Book whose heroine accidentally lands in the middle of the Black Death in 14th century England. This is a really chilling account of a pandemic without modern medical interventions. Later books include a farce set in late Victorian England and two books set in London during the Blitz. Although the central figures are well drawn, these use more stock types in the surrounding characters. Otherwise meticulous research is also let down by a neglect of the differences between UK and US railway/railroad argot. These grated on a reader whose family has over a century of experience in that industry! Nevertheless, the four books saw me through several long-haul flights and provoked useful methodological thoughts about past, present and future, that may or may not surface in other contexts.

 

Jenny Neophytou
Source: Jenny Neophytou

Jenny Neophytou, Bibliometrics Manager, Wiley

Anyone who’s met me will probably know that I’m a voracious bookworm, so I’m restricting myself to the three books from my Summer library that made the biggest impressions on me this year. Firstly, Fool’s Quest by Robin Hobb. This is the latest of Hobb’s ‘Farseer’ books. While the genre may be fantasy, Hobb’s books read like history, and are structured around some fascinating philosophical debates on the nature of history, context and identity. When it comes to the fantasy genre, you can’t do better than this.

Next, Stephen King’s The Running Man, and Markus Zusak’s The Book Thief. If you think you know The Running Man from the film, think again. This is no action-comedy. It portrays a chilling 1984-style dystopia where governmental control is enforced by misinformation, and by deadly televised games that are the reward for poverty or dissent. Finally, The Book Thief. Like all of Zusak’s books (including the equally powerful I Am the Messenger), you will not finish this unchanged. The Book Thief the story of an orphaned child in Nazi Germany, whose love of books sets her on a dangerous path between governmental compliance and rebellion. The story is narrated by Death, but the personification is powerful rather than childish, and the book itself is heart-breakingly beautiful. Watch out in particular for the moment when Hitler’s Mein Kampf is transformed into a fairy tale by a Jewish refugee. 

    Rowena Murray
Rowena Murray
  Professor, University of the West of Scotland

I’m writing this during the last 90-minute session at a two-day residential writing retreat I organized for my PhD group – 12 students – so that they could focus on writing a journal article. I’ve been writing one too. This Writing for Publication retreat is a new idea I came up with to provide real time and support for publishing during the PhD.

Most of my students have been here before to write thesis chapters, but this is the first time they are here to write journal articles. Because it’s a new writing task for most of them, and because I have experience of students requesting pre-retreat workshops, we had a two-hour ‘Writing for Publication’ workshop last week. I wanted to make sure that they were prepared to make the best use of their time here, and they did too.

During that workshop we covered key points about (1) analyzing their target journals – I’d asked them all to choose a target journal before the workshop and to bring an article from it – and (2) working out the focus of the article, using the device of drafting an inquiry to the editor – which they said they found daunting at first but actually very useful. It made them decide what their article was going to be about – an essential first step and crucial preparation for a writing retreat.

Most of what I have to say about writing journal articles is in my book, Writing for Academic Journals, and all my students have that, but I wanted them to make a start on their articles before this retreat. A Structured Writing Retreat is one of the most productive and positive practices for academic writers, but it only works if you have something to write about. See rowenamurray.org for more information and evidence of impact.

 

Source: Rowena Murray
Source: Rowena Murray

As usual, almost all of our time at this retreat has been spent writing. However, there have also been crucial discussions of focus, giving and receiving feedback on drafts and revisions, conversations about how articles fit with theses, focusing discussions about shaping articles for specific journals and checking and comparing journals’ instructions for authors. I think what’s important about writing in this way is that you can get feedback in one of the breaks and then act on it right away in the next writing session.

As a supervisor, I can check that each student has understood my suggested revision when I see the next draft at the next break. This is not to say that we are all on a writing treadmill, but that we are having on-going discussions of writing-in-progress. I prefer this to receiving whole drafts of articles that might not be as sharply focused as they could have been if we’d had these discussions earlier on.

These on-going discussions are great for focusing, and the fixed time-slots of the retreat program help us to set goals for the time we actually have. Monitoring the extent to which we achieve these goals, as we go along, helps us to develop self-efficacy – in relation to generating writing for publication – and to set realistic writing goals.

One of the goals I encouraged them all to set was to draft a title and abstract for their articles. They all started this last week and have been revising abstracts and drafting sections of articles yesterday and today. The great thing about this for me is that I can help them sharpen the focus of their writing, and it only takes me a minute or two to read each abstract, which means I have been able to provide feedback several times during this retreat. The great thing for them is that they get feedback right away and, once they have an abstract that we’re all happy with, they more or less have the focus for sections of their articles. A focus of our discussions of draft abstracts is the last sentence, where you say what ‘new knowledge’ your article offers. Until you have that, it’s difficult to know how to focus the rest of the article.

All of this helps authors overcome their uncertainty about whether they are ‘ready to write’. We use these writing tasks and discussions to help them develop something in writing and thereby overcome this uncertainty. While we have spent some time talking about the sources of and triggers for their uncertainty, we have all managed to produce text.

As soon as I write or say ‘we produced text’, there is usually someone in the room – and I have clearly internalized this voice – who asks, what about the ‘quality’ of text we produced: surely, they say, the quality is more important than the quantity? To which I say, of course, quality is essential, but it’s not achieved in one draft. It’s not achieved at one writing retreat.

We have made a good start here, but there is still work to do. I suggest they think about producing ‘quality’ writing as a series of stages: which component of quality can they produce at any stage in the writing process? At this writing retreat, they have achieved the qualities of focus, coherence and structure. There is still work to be done on other components of quality over the next few weeks and months.

Finally, I can report that all the PhD students produced a draft journal article in the 10  hours of this writing retreat, and I did too. So, I think we’ll be continuing with this new Writing for Publication Workshop + Structured Writing Retreat next year, if not before.

    Anne-Marie Green
Anne-Marie Green
Communication Manager, Wiley

Titia de Lange is head of the Laboratory of Cell Biology and Genetics at The Rockefeller University and director of the university’s Anderson Center for Cancer Research. In November she will succeed Günter Blobel as chair of the Wiley Prize jury and she spoke to us recently about the prize and its importance.

 

Source: Titia de Lange
Source: Titia de Lange

Q. Can you tell us about your background and your current position?

A. I am from Holland but have been in the US for 30 years. I did my PhD in Amsterdam and then went to UCSF to study with Harold Varmus who received a Nobel Prize for his work on cancer genes. After my work with Harold, I moved to The Rockefeller University to set up my own lab and I have been here for 25 years.

Q. How did you become involved with The Wiley Prize?

A. Outgoing chair of the Wiley Prize, Günter Blobel asked me to join the committee.

Q. Why are prizes such as the Wiley Prize important to the biomedical research community?

A. The Wiley Prize is unusual in that it often is ahead of other award committees. The Wiley Prize can guide other committees to potential candidates. This has happened time and again with Wiley awardees receiving major awards later.

Q. What does the Prize offer scientists?

A. Prizes offer scientists recognition of their research. To be selected by a group of your peers for your discoveries is an important validation ofyour work. . The Wiley Prize in particular offers a $35,000 award and a luncheon in honor of the prizewinner(s). Nominations are open until September 30th, 2015 and can be submitted here.

Q. Can you tell us about a few of the notable prizewinners?

A. As I said, the Wiley has often been ahead of the curve. Notable examples are: Peter Walter and Kazutoshi Mori who received the Wiley in 2005 and the Lasker Award in 2014; Liz Blackburn and Carol Greider who received the Wiley in 2006 and the Nobel in 2009. And Andrew Fire and Craig Mello who received the Wiley Prize in 2003 and the Nobel in 2006. This year’s Wiley Prize winners Evelyn Witkin and Stephen Elledge went on to win the Lasker Award earlier this month.

Q. You have been the recipient of numerous awards in the biomedical sciences as well. What do you feel has been the most significant achievement in your career thus far?

A. I have worked my whole career on one simple problem in biology: how do eukaryotic cells survive with linear chromosomes? The problem is that the ends of the chromosomes look like broken DNA and cells will try to repair them. We have telomeres at our chromosome ends to prevent such repair, which could lead to a disaster in our genome. I have been studying telomeres for 25 years to understand how they work. We have come a long way and understand in principle how they work but much remains to be discovered, thankfully….

Thanks Titia.

    Kerstin Brachhold
Kerstin Brachhold
   Editor, BioEssays

509916917_293052355_293052356_256224451.jpgReviewing a manuscript requires many skills. Of capital importance is that you have enough background knowledge to be able to assess and comment on the science presented in the paper you are evaluating. But how do you communicate your comments back to the authors? How should you phrase your report? Below are some tips on what to watch out for when writing a reviewer report.

Keep it simple. Don’t forget that for many authors English is not their native language. Write your report so that it can also be easily understood by non-native English speakers. Short and clearly structured sentences will help you get your message across. Furthermore, carefully think about word usage. Using complex and unusual words might obscure the meaning. It is not helpful if authors have to consult a dictionary to understand your report.

Proverbs and colloquial language may also pose problems and should therefore be avoided. Moreover, keep in mind that it is not always easy to recognize sarcasm in written form, without the possibility of seeing gestures or facial expressions. Sarcasm may also be received and judged differently depending on the cultural background of a person. Consequently, sarcasm is also a form of expression that should not find its way into a reviewer report.

Be constructive, not destructive. Formulate your comments in a positive, constructive way. Simply damning a manuscript will neither be helpful for the editor who has to make a decision based on the reviewer comments nor for the author. Be direct, especially if you request additional experiments. Use wording that makes it clear which experiments are absolutely necessary and which would simply be nice additions.

Be objective, not subjective. Reviewing a manuscript means that you will also have negative comments for the author(s). Even if you feel strongly about certain shortcomings of a manuscript, remember to voice your criticisms in a courteous way using neutral, objective language. Stay professional and do not convey the impression that you, the reviewer, are a know-it-all while the authors have no clue. Many journals also offer the possibility to enter confidential comments to the editor. While these should be kept to a minimum they will give you the possibility to voice your personal opinions.

Don’t use offensive language. Keep in mind that you are evaluating the science and not the person who wrote the paper. Accusatory, derogatory or even libelous language should be avoided. Editors will normally not edit reviewer reports. The reports will usually be sent to the authors as they are submitted. On rare occasions, however, an editor might choose to remove offensive language.

Authors should also pause before reacting emotionally to the language in a reviewer report. Just as the reviewer should remember that the author is not necessarily a native English speaker, the author should assume the same of the reviewer when reading a report. Criticism expressed in robust language is not necessarily unfair. Tolerance and good will on both sides will be beneficial in the communication between reviewers and authors and will ultimately help to improve a manuscript.

Image Credit/Source:Getty Images

    Verity Warne 
Verity Warne
Senior Marketing Manager, Author Marketing, Wiley 

peer review weekToday we are delighted to announce that, as part of our commitment to peer review and the invaluable role that it plays in maintaining scientific quality, Wiley is partnering with ORCID, Sense about Science and Science Open for a week of peer review celebrations.

Peer Review Week 2015 will run from Monday, September 28th to Friday October 2nd, and will include:

     

    • Daily posts about peer review on each organizaton’s blog

     

    • A Twitter campaign (#peerrevwk15)

     

    • A Webinar on Trust and Transparency in Peer Review with Kent Anderson (AAAS), Verity Brown (St Andrews University), Alexander Grossman (ScienceOpen), Laure Haak (ORCID), and Andrew Preston (Publons)

     

    • A Video of top 10 peer review tips

 

Peer Review Week grew out of informal conversations between ORCID, ScienceOpen and Wiley. Each organization has a different perspective on peer review, and has been working independently to better support its role in scholarly communications. Joining forces enables all three organizations to share their central message - that good peer review, whatever shape or form it might take, is critical to scholarly communications - more widely and powerfully. Sense About Science has also joined the week to ensure the wider benefits of peer review – as a mark of quality and tool for making sense of science claims – are shared with the public.

“At Wiley we believe that peer review is the foundation for safeguarding the quality and integrity of scientific and scholarly research,” comments Philip Carpenter, Executive Vice President, Research, Wiley. “Peer Review Week and our partnership with ORCID, Sense about Science and Science Open allow us to highlight the crucial role that peer review plays in protecting trust in scholarly communication.”

Researchers spend a substantial amount of time reading and reviewing, but are rarely acknowledged for this important contribution to the community,” says Laure Haak, Executive Director of ORCID. “ORCID is pleased to be part of peer review week and the effort to increase recognition for review activities.”

ScienceOpen’s CEO, Stephanie Dawson explains that: “Our goal is to help re-build trust in the peer review process by making it entirely transparent. We facilitate Post-Publication Peer Review from named individual experts to nearly ten million open access articles and toll stubs currently available on the platform. We’re delighted to participate in this inaugural Peer Review Week.”

And Victoria Murphy, Program Manager of Sense About Science, adds: "Peer review is an essential arbiter of scientific quality, and asking 'Is it peer reviewed?' helps people to query the status of science and research reported in the media. So during this inaugural Peer Review Week, we want to share that question as widely as possible. Raising awareness of the value of peer review is vital for maintaining quality in science."

We’d like to bring together everyone who cares about peer review and hope that you help support Peer Review Week by engaging via social media and sharing your thoughts. If you’d like to create your own blog post or editorial you can download the Peer Review Week logo here: landscape or portrait. And don’t forget to Join the conversation on Twitter #PeerRevWk15

    Jenny Neophytou
Jenny Neophytou
Bibliometrics Analyst, Wiley
Source: Steve Mann/Getty Images
Source: Steve Mann/Getty Images

If you're familiar with the concept of citation metrics, it's highly likely that you are also familiar with the concept of anomalous citation patterns. These take many forms, however the most commonly discussed are those relating to self-citation and citation stacking. With every new release of Thomson Reuters'’ Journal Citation Reports, the academic community is flooded by blog posts, articles and editorials discussing the relative merits and flaws of citation metrics – most notably the Impact Factor – and the relative ease with which they can be manipulated. Examples include posts in both scholarly and non-scholarly sources, such as Scientific American, Times Higher Education and Scholarly Kitchen.

Of course, some metrics (such as the Eigenfactor and the SJR) restrict or omit self-citations from their calculations, while Thomson Reuters reserves the right to exclude titles from the Journal Citation Reports if it is found that ‘anomalous citation patterns’ are having an undue influence on their rankings. This exclusion is commonly perceived by the academic community as punishment for misbehavior, and can cause severe damage to the reputation of both editors and journals. Articles discussing these matters frequently associate the terms ‘citation stacking’ (a pattern of citation behavior) and ‘citation cartels’ (one possible cause of citation stacking), further strengthening the perceived link between anomalous citation patterns and culpable behaviour. However, let’s not forgetthat exclusion does not necessarily indicate culpability, and it is this question that draws attention to a disconnect between the perceptions of the academic community, and the policies implemented by Thomson Reuters – who insist that exclusion from the Journal Citation Reports is not a ‘punishment’, but rather an attempt to maintain the integrity of their dataset. As such, it is important that before assigning culpability, we pay attention to the various factors (other than citation manipulation) that can lead to ‘anomalous citation patterns’.

Self-Citation

Self-citation can refer to occasions when authors cite their own papers, or occasions when journals cite their own papers. Both of these forms of self-citation can distort citation metrics, however journal-level self-citation is far easier to detect.

Deliberate attempts to boost citation metrics through self-citation can cause considerable damage to a journal’s reputation, regardless of the benefit to its citation metrics. Practices such as loading editorials with self-citations, or ‘suggesting’ that authors include more citations to the receiving journal, stand to create bad feeling within the academic community. Moreover, they create a false impression of the usefulness of research, and (without appropriate checks) can lead to poor-quality titles dominating the journal rankings.

However, there are two key reasons (other than deliberate manipulation) why journals might have a higher-than-average self-citation rate:

  

    • Niche or archive publications. If your journal is the journal that founded a discipline (and therefore holds the research archive), or if your journal is one of just a handful that covers your subject area, it is natural that many of your papers will cite other papers in your journal.

  

    • Poor discipline coverage. If your competitor journals are not indexed in a given citation database, citations made by those journals will not be indexed. This means not only that your journal metrics will be calculated based on a fraction of your total citations (one of several reasons for the low metrics of SSH titles), but that the majority of visible citations to your journal will be from your journal – leading to a high rate of self-citation.

 

Citation Stacking

The waters become even murkier when we come to the issue of citation stacking. Citation stacking refers to anomalous citation activity that sees a disproportionate number of citations being exchanged between two or more journals. There is typically a ‘donor’ journal (the title giving the citations) and a recipient journal (the title receiving the citations). Such patterns can be evidence of a ‘citation cartel’, whereby groups of journals attempt to inflate their metrics through donating and receiving citations without inflating their self-citation rate.

Notably, in the case of citation stacking, both the donor and the recipient journals are suppressed if Thomson Reuters believes that citation stacking has had an undue influence on the rankings of either title. However, there are many reasons why journals may appear to be guilty of citation stacking, without any wrong-doing on their part.

  

    • Poor discipline coverage. Once again, if only one or two of a journal’s main competitors are indexed, it stands to reason that the vast majority of itscitations will come from those few publications.

  

    • Author self-citation. I was recently unhappy to witness a journal being excluded from the Journal Citation Reports as a result of a single author group extensively citing their own papers from within a different publication. Despite the fact that these papers were swiftly retracted by the other publication, and that the authors in question had no ties to the editorial teams of either journal, the exclusion (for both journals) remains in place.

  

    • Third-Party Influence. It is important to remember that some journals may be unwitting members of a citation cartel. I have seen occasions where author groups, linked to another publication, have acted as citation cartels – publishing papers in a range of journals, yet loading those papers with citations to their own title, thereby creating a donor/recipient scenario. While peer-review should catch the worst of such offenders, these trends can be difficult to spot when multiple reviewers are working on a collection of affected papers.

  

    • Journal size. While all journals are vulnerable to exclusion for ‘anomalous citation patterns’, small journals are the most vulnerable. It is well understood that small journals tend to have the most fluctuating journal metrics, simply because the addition of a small number of citations can have a big influence on the average. The same is true for citation stacking – and, indeed, self-citation. It is difficult to imagine a journal such as Nature (with over 70,000 citations in the 2014 IF calculation) receiving enough citations (whether self-citations or citations from another publication) to be subject to the exclusion policy. In contrast, a small SSH title (with maybe 20 citations in the IF calculation) is vulnerable to excessive attention from a single paper.

 

Why is this important?

The perception of JCR exclusion as a ‘punishment’ has led to a prolific association between anomalous citation patterns and culpability – an association that can severely damage the reputations of editors and journals alike. However, it is an association that cannot always be upheld, and which, particularly in the case of citation stacking, can perpetrate injustices. Thomson Reuters insists that exclusion from the Journal Citation Reports is not a punishment for wrongdoing, knowing, perhaps, that it is impossible to judge culpability without a detailed investigation. As such, it is important that we increase our awareness of the complexities of the issue – not only to avoid unjustly allocating blame, but also to remind ourselves of the factors that could leave our own journals vulnerable.

 

    Michael O'Riordan
Michael O'Riordan
   Editor, Wiley

Oona Schmid, Publishing Director of the American Anthropological Association,  has extensive editorial and entrepreneurial experience that position content and products to maximize their digital opportunities. She specializes in smart adaption to digital challenges, as evidenced by the  win of an Alfred P. Sloan Foundation grant to prototype an all-digital book review workflow and two grants from the National Science Foundation to help anthropologists preserve their digital data sets.

She recently spoke to us about the relaunch of AnthroSource as a society hub on Wiley Online Library and what this means for AAA members and AnthroSource subscribers.

 

AAA_Schmid
Source: Oona Schmid

Q. Can you tell us a bit about your background and the American Anthropological Association?

A. With about 10,000 members, the American Anthropological Association is the world’s largest scholarly society for trained anthropologists. Its mission is to promote a global exchange of anthropological research, and apply that research to the world’s most pressing problems. We do this principally through an ambitious publishing program and also through hosting an annual conference that attracts more than 6,000 anthropologists.

I came to AAA seven years ago as a publishing professional — not as a subject expert in anthropology. While the program has an incredible output of anthropological knowledge in 22 different journals, it was not on stable financial footing and was struggling with digital challenges like how to keep content visible to students or how to make sure non-anthropologists knew about our content.

Q. What is AnthroSource and how is it used by AAA members and researchers?

A. In 2003, the Association did a remarkable thing. It had the prescience to make a large investment—aided with an instrumental grant from the Mellon Foundation—to digitize the complete contents of 30 titles, realizing that the usefulness of its scholarship to members and other researchers required creating an online database of its journals. Initially built and hosted by Atypon, AnthroSource provided full-text 24/7 access to 31 journals of the Association.

That achievement used a design concept from the turn of the century. The site emulated a print experience, in order to promote trust and stability, so that we could address fears that we heard from librarians and members that online publishing would be ephemeral. A decade later, we have two new titles added to AnthroSource and in 2014 our content saw more than 1.5 million full-text downloads, but we also had an interface in dire need of a fresh look and more agile navigation.

Q. What are the new features and benefits of AnthroSource 2.0?

A. AnthroSource 2.0 provides full-text searching of all readable PDFs. It offers responsive

 

Anthrosource 2.0
Anthrosource 2.0

design for all webpages and the HTML content for our journals to make reading on smart  phones and tablets easier. Best of all the redesign gets members and researchers to full-text  content within two clicks.

The site also offers some new paths to uncover new articles, like dynamic lists of the articles that  have been “most cited” and “most accessed,” and lets editors highlight content that is  particularly timely or intriguing.

Q. What were the biggest challenges and opportunities of the re-launch? Did you gain  any new insights into how members engage with AAA content?

A. I am always surprised at the power of Google. When we did user experience testing and  asked members and the larger anthropology community what they do first, we witnessed how  Google has shaped researcher behavior. Users beelined to the search box. We made our  search box larger and easier to spot.

We always knew that the big value of AnthroSource is in its breadth and depth of full-text content. One awesome feature of the new AnthroSource is the “flyout” abstracts, which let scholars hover a mouse over a title (in a search result or within a table of contents) and identify immediately if they want to click on that article. I am very proud of this because it’s a way of emphasizing that digital environments can create added value for researchers. How many of us have had to wade through a huge search result to pan for the gold? It’s frustrating! This feature can help scholars filter out their content gold.

Q. Have you received feedback from members and the larger anthropology community since the launch of AnthroSource?

A. “A great improvement!” –Lynne

“WOW! It is gorgeous and user-friendly and such an improvement!” –Kathi

We are very pleased with our members’ reactions to the site. Of course we look forward to seeing the site continue to develop, but even a critic called the site “glorious.”

Q. What further enhancements would you like to see for the AnthroSource site in the future?

A. Discovery of content is the largest opportunity. I’d love to see the Association make sure the entire holding is digitally visible, whether faceting search results by keywords, deploying a taxonomy like HRAF does of cultural-linguistic groups, or creating new means of associating relevant archival and current content. These initiatives are challenging when content creation is done perfectly, but at AAA our content practices, especially before 2002, were erratic and so our archive is riddled with challenges.

    Jason Markos
Jason Markos
   Director, Content Enrichment, Wiley
Source: Rawpixel/Shutterstock
Source: Rawpixel/Shutterstock

Publishing models are undergoing a new revolution, and societies are in a prime position to be  at the forefront of this movement. Content enrichment is a vital component of this new wave of  publishing.

A key financial activity for societies, publishing typically provides over 60% of their revenue. Until  recently, it has been based on a print model for content development, and a subscription model  for sales of that content. While the majority of access to content is now online, PDF has been  remarkably tenacious as the primary format that people use for reading and sharing. As a result,  we’re still very much working in a page-based model, bringing with it a range of quirks and  idiosyncracies. For example, we want to enable users to read content on a range of devices, or  even determine their own preferred layout, but we are spending considerable resource reviewing and approving format and layout of the PDF. Under these circumstances, all of that work becomes redundant.

Market Pressures

The page-based model, and its associated challenges, are well documented; however, societies are facing new market pressures that are increasing the focus on their publishing business.

With online publishing the amount of content available is increasing at an exponential rate, which in turn is making it more important to make the content that you publish stand out, Simply having lots of content, or having it well-formatted, will no longer be enough.

Open Access is a major, and growing, publishing model. Scholarly publishing revenues have grown by 2.5%, while within that figure, Open Access revenues grew by 34% in the same period. This is already significantly changing the dynamic of the market, both in terms of how people are accessing content, but also the society’s relationship with the author. The author is now a customer.

Inherent in the Open Access model is the change in the revenue model for societies, and this will require careful management.

The Opportunity

The rapid development of these new forces on society publishing will require a range of new strategies to be adopted. Prominent among these is Content Enrichment. By no means is it the panacea or silver bullet, but a society that implements a comprehensive enrichment strategy will be well placed to develop in this new world we’re living in.

To clarify, by enrichment we’re referring to rich classification and entity identification in and around content, in such a way that it can utilized by delivery applications and users. For example, being able to understand that article A discusses the diagnosis of diabetes in children that have been living a healthy lifestyle, or that Fig. 3a illustrates compound A catalysing compound B, resulting in compound C, with a yield of ‘X’. Much of this knowledge is in the content already, just not captured in a structured format that can be utilized. It’s a simplification, but Content Enrichment can be thought of as making the implicit information explicit.

With this enrichment in place, societies will be able to realize 3 key objectives:

Increase relevance, discoverability, and usage

Content that is better described will enable discoverability through improved indexing (Search Engine Optimization) and linking (e.g. related articles, browsing by topic) and ultimately increased usage. That usage can be key to driving Impact Factor, and is becoming even more important with the growing adoption of article level metrics. Also, as we move from subscription models, usage is vital to support advertising based revenues.

Improved credibility

Credibility is key, not just from the arms race perspective (society ‘X’ is doing it, so we’d better do it), but also for securing authors, which is arguably more important. As authors become customers, society publishers will need to demonstrate more value to themby making their work easier to find and utilize. There are a number of ways that this can be achieved, but one strategy is to better capture their hard work represented by the data accompanying their articles.

Facilitate new value propositions

The future lies in being able to do more with content and its associated data. If content is enriched and well described, we open up new opportunities. Just one example is the ability to illustrate trends in research─i.e. which topics are becoming increasingly active, and which are becoming less so. With well-enriched content, these capabilities become easier to realize.

We are embarking on a journey where the basis by which we publish content, and the associated value model, is fundamentally changing. Moreover, it’s not yet clear what the new model is that we’re working towards. Consequently,we need to adopt a strategy that will add value to what we do today, and equip us to explore new business models and value constructs in the future. Implementing a content enrichment strategy is crucial to surviving and thriving in this new world.

    ben Hogan
Ben Hogan
   Regional Manager, Peer Review, Wiley

ever expanding world of peer review.PNGISMTE’s annual conferences are ideal spots for editors, managing editors, and publishing professionals to come together to share ideas and learn about what’s going on in the world of editorial management and peer review. This year’s US meeting, held from August 20-21st in Baltimore, was the society’s largest yet, with over 200 attendees. Each year, there are a number of recurring themes that cover the hot topics and issues affecting the work of editors, reviewers, and managing editors across the globe. Below are just a few of the highlights:

Ethics

A number of sessions centered around publication ethics, whether focused on authors, reviewers, editors, or publishers and staff. This year’s conference kicked off with a keynote from Jeffrey Beall. Dr. Beall is an associate professor and librarian at the University of Colorado who’s led a years-long quest against “predatory publishing,” a blanket term covering unethical publishers and authoring services. These services flood the academic marketplace, and are tracked by Dr. Beall in his blog. The keynote sparked a lot of debate about the growing number of ethical cases and retractions, as well as the future of open access publishing.

Dr. Beall was also part of a panel on how to combat predatory publishing practices. Don Samulack, of Editage, announced the beginning of a Coalition for Responsive Publication Resources as a possible cross-publishing solution meant to validate ethical publishers and vendors.

Another ethics session, led by Kathleen Lyons, from Nature Publishing Group, and Debra Parrish, from Parrish Law Offices, focused on coordinating a journal’s best practices with legal requirements. One of the key takeaways was the suggestion for editorial offices to be prudent with wording. From a legal perspective, “We strictly follow COPE guidelines” is much more restrictive than “We generally follow COPE guidelines” when it comes to particularly troubling cases. The session reminded us of the importance of being careful about exactly what makes it into public author guidelines.

Public relations and author marketing

There were also several sessions on the growing importance of public relations and author marketing. Alice Northover, from Oxford University Press, and Charlie Rapple, from Kudos, gave a fascinating presentation about the different tools Kudos provides to authors to help support themselves, as well as the role publishers and public relations play in getting articles out to the mainstream media and public. The first half of the talk centered around some of the tools publishers have to help market journals and authors, as well as some of the misconceptions around public relations, social media, and advertising. During the second half, Charlie spoke about Kudos and how it allows authors to share publications through e-mail and social media, enrich content by adding contextual links, and measure downloads, citations, and altmetrics.

Dr. Audrey Huang’s plenary address, entitled “Science Out of the Box” was a really helpful demonstration of how labs at Johns Hopkins University are using toys to explain complex medical and chemical concepts to a non-medical audience. These videos can raise public awareness and be a potential link for funders. During this year’s Speed Networking session, conference attendees were encouraged to create and record their own Out of the Box video to explain a peer review concept.

China

With the growing influx of Chinese scholars serving, as authors, editors, and reviewers, China is annually a large part of the ISMTE discussion. This year, Xianyong Yin, from UNC-Chapel Hill, and Yan Shuai, from Tsinghua University Press, shared insights on the publishing environment in China. The presenters spoke at length about the author and reviewer experience in China and how they interact with journals. One thing that really stood out here was that there are over 5,000 STM journals based out of China. About 80% of these aren’t published by traditional publishers, but by research institutes, universities, and societies. The presenters also spoke about a project led by CAST (China Association for Science and Technology) called PIIJ (Project for Enhancing International Impact of China STM Journals). PIIJ is a centralized initiative to increase the scope of English language journals based in China through better promotion of content and collaboration with international scholars, among other initiatives.

The prevalence of predatory publishing in China was a popular discussion point, following on from Dr. Beall’s keynote. As Chinese scholars are often the target of paper mills and other unethical practices, vigilance and increased awareness on the part of all peer review management professionals will help create consistency and allow Chinese scholars to continue to flourish in the academic publishing world.

Aside from these sessions, ISMTE also offers networking opportunities as well as an Exchange Forum where managing editors can put their burning questions to their peers. Of particular note, this year were questions about best practices for running iThenticate similarity reports and how often to run them, as well as thoughts on validating author and reviewer e-mail accounts. Responses were all over the map, but they did give attendees a lot to think about. These forums are always fascinating, and this year’s topics ranged from plagiarism checks to authenticating reviewers. For the past two years, ISMTE has partnered with COPE (Committee on Publication Ethics). COPE now offers a one-day seminar the day before the ISMTE conference. Because ethical issues are such a huge part of peer review management, it’s a natural fit and this year’s COPE seminar was an excellent addition to the week. Hopefully this partnership will continue at future conferences.

With the next US conference already set for August 2016 in Philadelphia, and the next COPE/ISMTE European conference scheduled for October 12-13, 2015, along with the first ever Asian ISMTE conference in April 2016, there are plenty of opportunities for peer review management professionals across the globe to see what the society has to offer. As someone who’s been an attendee of the meetings for the past five years, the learning and networking opportunities are invaluable.

Image Credit/Source:International Society of Managing & Technical Editors

Filter Blog

By date:
By tag: