ACS Reviewer Lab: an Online Course for Peer Review

In 2017, the American Chemical Society (ACS) launched ACS Reviewer Lab, a free online course designed to educate researchers about peer review. The course consists of six interactive modules, each of which can be completed in approximately 20-30 minutes depending on your level of familiarity with the concepts discussed. At the end of each module, a summary of discussion points (downloadable as a PDF) as well as videos are displayed to highlight the main points. The course covers the basics of peer review and also provides practical guidance on how to prepare for a review, assess the manuscript, and complete the review including tips for writing the actual report.

Overview of ACS Reviewer Lab

As a non-chemist Humanities major, I found the course helpful, informative, and well-designed. Alas, lending your time and expertise as a peer-reviewer need not be a daunting task! One of the things I liked most about the course was its broad applicability to those outside of chemistry. Intended to be multidisciplinary, I felt the course was very user-friendly, accessible, and not overly technical or filled with scientific jargon. (Given this, I would recommend it to my non-chemistry research friends).

One caveat is that once you've signed up, you must complete the course within one month. I felt this deadline was very reasonable and further, it limited my tendency to procrastinate to the last minute.

Another perk is that at any time in the future, you can login and access the course material, including module worksheets (or summaries), videos, and a glossary of terms. You can also redo the whole course. This could be helpful if you took the course now, but didn't get invited to review a paper until later.

How ACS Reviewer Lab Helped Me Professionally

A little about my motivation: as an early career editorial professional, I provide peer-review support to Journal Editorial Offices and work with Editors, Authors, and Reviewers. I have an advanced degree, but not in chemistry. A big part of my job is facilitating the peer-review process. I was curious if learning more about peer review could help me do my job better.

From where I sit, a good peer-review can be subjective, incredibly valuable, and sometimes hard to come by. Before I took the course, my questions included:

• What constitutes a ‘good' peer review?
• Are there standards of peer-review the research community enforces, albeit informally?
• What are the expectations when you accept an invitation to review?
• What exactly is the reviewer agreeing to provide?

And once I learned there was an online course available, I started asking these questions:

• How is peer-review taught?
• Is it taught?
• What does that look like?

From working closely with Editors, I can often feel the frustration when they receive a ‘bad' review. A poor report can hinder the entire process. (And when I'm using the words ‘poor' and ‘bad,' I do not mean negative or critical). A bad review is one that eats into the overall time it takes to get a decision back to the authors, which ultimately delays timely publication of really fascinating research.

Further, it can also be frustrating for authors to receive comments which offer little to no constructive feedback or practical advice on how to improve their manuscript. Given that peer review training can be inconsistent or lacking altogether, an online course seems like a great format for providing more education.

In sum, I felt that the course covered a lot of material without being too dense or boring. For me, the most helpful sections included the Introductory module, and the sections on ethical considerations and how to prepare for a review. I imagine if you're asked to review a lot, you would find value in the latter sections on how to assess impact and rate a paper's technical quality as well as the final module on writing a review.

I also think there are a lot of good tips that could benefit seasoned reviewers such as including a summary of the paper to let authors know you've read their work and recommendations on how to organize your critiques of the paper. After learning more about what the gold standard looks like, I have seen reviews that follow these recommendations.

So, did the course enable me to do my job better? After ACS Reviewer Lab, I have a much better understanding of what is valuable to authors. And if the report adds value to authors, it's inherently valuable to Editors.

For More Information

I'm not here to spoil the contents of the course further so if I've piqued your interest, visit ACS Reviewer Lab to get started by first creating an ACS account and ORCID. Once you've completed the course, you'll receive an online certificate signifying your accomplishment. Print it out. Hang it up. Email it to your boss or your Mom. Either way, you'll be better prepared to tackle your next reviewer assignment. Or if you're like me, you'll have more context into the incredibly valuable gift that is a solid and outstanding peer-review.