4 Critically Examining Established Peer-Review Practices
4 Critically Examining Established Peer-Review Practices
Introduction
In this module we will expand on the opportunities and challenges presented in Module 2: Opportunities and Challenges in Peer Review, as well as the work you did in Module 3: Bias and Power Structures in Peer Review. This module offers you more time to think critically about established peer-review norms and practices, as well as reflect on your own personal biases and how you might thoughtfully work with your awareness of them. When we think critically, we think about the players in peer review processes, why things are the way they are, and if they have to be that way.
A Note on Critical Examination
This module asks you to critically examine established peer-review practices. The module approaches critical examination as it is rooted in Critical Theory, a historical and philosophical approach that studies social and political contexts: “It must explain what is wrong with current social reality, identify the actors to change it, and provide both clear norms for criticism and achievable practical goals for social transformation” (para. 3).
By the end of this module you should be able to:
- Identify concepts and practices that reinforce bias and power structures in peer review
- Examine peer-review systems for bias and imbalances of power
- Identify established anti-bias practices
- Develop strategies for eliminating bias in peer-review systems
Activities and Exercises
Read and Reflect: Editorial and Ethical Policies (4:1)
Review the editorial and ethical policies for peer review at the American Journal of Public Health. (Scroll about halfway down the page to get to the peer review section.)
Read the Ethical Guidelines for Peer Reviewers from the Committee on Publication Ethics (COPE).
Take some time to consider and reflect on these questions:
- Does what you read from the American Journal of Public Health policies match the Ethical Guidelines outlined by COPE? How do they, and/or how do they not?
- What are the strengths and weaknesses of the COPE guidelines? Would you change anything about them?
Do and Reflect: Defining Rigor (4:2)
Use this prompt to free-write for five minutes: I know something is rigorously researched and of quality when…
Next, spend some time (~10 minutes) looking for a concrete definition of rigor or quality in academic research and publication in your field. You could look on the open web or via handbooks or encyclopedias at your library. Then, when you’ve found a definition that you like, write it down. Reflect on the following questions in a journal or aloud on a recorder:
- Did my personal definition match the one I found ?
- What concrete measures are used to evaluate rigor?
- Who defined it for my field?
- In their 1996 article, Ways of knowing, culture, communication and the pedagogies of the future, Paul Wildman and Sohail Inayatullah assert, “The search for rigour is also often the call for the elimination of difference” (p. 733-734). Based on wha