By Čarna Brković

Emergent Conversation 16

This essay is part of the series PoLAR Online Emergent Conversation on Peer Review as Intellectual Accompaniment

Beyond the Courtroom Model of Intellectual Exchange in Peer Review

Imagine a person writing a peer review. Perhaps you will picture a skeptical individual crushing scholarship just because they can. This is at least the popular image that circulates in social media groups such as “Reviewer 2 must be stopped!”: an angry person who inflicts pain because they do not (want to) understand an author’s argumentation. The sense of being misunderstood and wronged by a peer reviewer seems to be almost universally shared in all corners of academia. Even when acknowledging that the paper improved thanks to the peer reviewers, we can “remain somewhat bewildered by some of their comments” (Smith 2012, 61).

A different picture might emerge when we imagine a relationship between a supervisor and their doctoral student. These are people “linked by an umbilical cord,” I was told once by a scholar from the U.K. German language echoes the same idea: a PhD supervisor is called a “doctoral father” (Doktorvater) or a “doctoral mother” (Doktormutter), expressing an intellectual bond in the idiom of family intimacy. Just like any kinship relation, PhD supervision can go awry. But it is, in principle, supposed to be dialogical and nurturing, critical, safe for differences, future-oriented, enabling growth and maturation.

There is something structural here in how the relationship between thinking, hierarchy, and temporality is often imagined in the “Global North.” Many of us have been trained to express respect for our peers by thinking “against them”—by trying to poke holes in their arguments. This—let us call it a “courtroom” model of an intellectual exchange—can be stimulating. Learning how to separate an attack on the argument from an attack on the person and how to immerse yourself in the former can bring intellectual joy and also advance our arguments.

Yet “thinking against” is not the only way to pursue an intellectual exchange in academia. In the case of peer review for journal articles, it might not be the most helpful one either. When a peer reviewer engages in “thinking against,” this does not seem to bring intellectual pleasure. More often it seems to generate a sense of being wronged by the review. This is perhaps because “thinking against” is predicated upon an assumption of egalitarianism within a scientific community, while peer review is an exchange in which some participants have the power over others to decide whether to let their argument be published, affecting, even limiting, career trajectories and the visibility and accessibility of different voices and knowledges. For “thinking against” to bring intellectual enjoyment, the participants in an exchange should be equals in some fundamental sense; it should be possible to safely disregard socio-economic differences between them, including how experiences of sexism, racism, classism, coloniality, and precarity have shaped their lives and thinking. In practice, this is not the case, as Kikon’s contribution to this series illustrates. Scientific community is not a “deep horizontal comradeship” (Anderson 1991, 7); instead, it emerges out of a set of fragmented scholarly conversations that take place among overworked and/or precariously employed scholars, who find themselves in unequal hierarchical positions in the uneven and fractured spaces of international academia (Hodges 2014). This has direct repercussions for the criteria of what gets published:

A system of journals with prestige tiers enforced by extreme selectivity creates a review system where scientific soundness is a necessary but far from sufficient criteria for publication, meaning that fundamentally aesthetic and sociological factors ultimately determine what gets published and inform the content of our reviews (Esarey 2015, 2).

The pandemic increased stress on authors, peer reviewers, and journal editors, pushing many of us to face the gap between how we usually imagine intellectual exchange in academia and the shifting demands of the world we live in. Conversations on the politics and practices of peer review and journal editing mushroomed over the last few weeks, deliberating on various kinds of approaches to creating sustainable change in the peer review system.[1] The need to think about how to reorganize peer review so that it presents an opportunity for “thinking with” in the unequal, hierarchically organized world we live in is precisely at the focus of this PoLAR Online Emergent Conversation.

Thinking With?

Following the invitation of the PoLAR editors, Georgina Ramsay and Sindiso Mnisi, in November-December 2021, Jennifer Curtis and I invited anthropologists based in different countries to reflect upon how scholarly approaches to peer review might change if we see it as an ethical practice premised upon certain forms of relationality, as much as a mechanism of quality control. We asked the contributors: What structural conditions are needed to encourage peer reviewers to take the time and to try to “think with” an author? What organizational set-up could foster peer reviewers to imagine an author as a person worthy of having a conversation with them? How do our existing personal experiences as authors, peer reviewers, or editors speak to this set of questions? What normative ideal of peer review can be developed for the future?

The background assumption of this Emergent Conversation is that peer review serves both as a mechanism of quality control of scholarly work and as a form of a dialogue among unequally positioned scholars—a dialogue in writing, mediated through editors, and taking place with many delays, but a dialogue, nonetheless. This PoLAR Online Emergent Conversation proposes that we understand peer review as an ethical practice of considering and reorganizing relations between arguments and bodies of knowledge as well as among scholars. It is an invitation to reimagine peer review as a sort of a dialogue between unequally positioned scholars, as a conversation across the fractures and gaps in both contemporary neoliberal academia.

The “courtroom” model of “thinking against,” whereby the reviewers focus on finding flaws in the argument and poking holes in the article, is not the only possible approach. I would like to suggest that peer review can also present an opportunity for “thinking with,” whereby the reviewer attempts to step into the shoes of an author and to think together with them on how to develop their point and how to make it in a more convincing manner. In my understanding, “thinking with” asks a peer reviewer to engage in an imaginative identification with an author of a paper they read. “Imaginative identification” is an aspect of everyday ethics that involves “understanding, getting a feel for, learning vicariously and fictively to inhabit not only my own point of view, but other people’s points of view too” (Chappell 2019).

Approaching peer review as an opportunity to “think with” requires considering not just the present state of the article, but also its potential future(s). It is not just about finding mistakes and/or (not) vetoing the article for publication in its current form; it is also about making an interpretive effort to understand what the author tried to do and to help them to get the paper there—just like we aim to do with our students. Imaginative identification does not ask us to agree with an author when reviewing a paper, but to engage in “reparative” reading, to paraphrase Sedgwick (2003, 147). “Thinking with” means considering which conversations this paper could advance, if it had the right support, and then (re)directing the author there. This might include a range of things, from providing the right support on the spot, to simply suggesting a new body of literature, new authors, or even a different (kind of a) journal. The important thing is to provide actionable suggestions as Elizabeth C. Dunn argues in her text for this series. Just as “thinking against” is a practice we cultivate during our academic training, the same might happen with “thinking with.” It can also be practiced and improved over time, iteratively, as Clarke illustrates in his piece (see also Clarke 2019). To use naïve metaphors, scholarship does not need to be always “cutting edge;” it can perhaps sometimes support “stitching the world” too.

Decentering the Discipline

“Thinking with” as an approach to peer review is especially important as a support to the attempts to “decolonize” and “decenter” the discipline by diversifying anthropological voices that get published. Thinking with can help to prevent epistemic injustice from taking place during a peer review: the kind of injustice that wrongs someone specifically in their capacity as a knower (Fricker 2007). Epistemic injustice sometimes occurs when reviewers and/or editors engage in “epistemic positioning,” evaluating “knowledge claims based on the speaker’s stated or inferred identity” (Baćević 2021, 1), as illustrated by Dada Dacot’s piece in this collection. It may also take place when peer reviewers “think against” an author who links diverse bodies of knowledge when developing their argument. As Docot’s piece also shows, when writing autoethnographically, drawing on non-Western ontologies, an author might encounter a reviewer who wants them to change their approach and argument to conform to norms of relevance that the author never wanted to speak to in the first place. This suggests that Indigenous, international, or working-class anthropologists often have to invest additional “interpretive labor” (Graeber 2012) when writing articles for a high-profile English-speaking peer-reviewed journal (the kind of a journal that makes and breaks careers). By “interpretive labor” I refer to a continual work of putting yourself into another’s shoes, trying to understand what the world, and you in it, looks like from someone else’s perspective—that of a reviewer, for instance.

The need for this additional interpretive labor might be shaped by different degrees of “contextual distance” (Hodges 2022) to the topic of the article between the author and the reviewer. For instance, during peer review a “native” anthropologist of Brazil or Belarus might understand that they need to explain more of what they initially took for granted—because their readers in an English-speaking journal simply do not share the same background knowledge. Additional interpretive labor might be required from scholars who work in-between intellectual spaces with differing conversations that necessitate multiple translations. This includes anthropologists working with Indigenous, as well as with modernist but non-Western forms of knowledge:

For example, a paper written by a top MA student from Hungary was rejected twice by Western journal editors. In the first round of review they requested her to frame her results around a Western theoretical framework. In the second round they rejected her paper because it did not confirm or reject Western theories, and was therefore not important enough to be published. (Blagojević and Jair 2010, 347)

As Reeves in her contribution to this Emergent Conversation writes, peer reviewing in socio-cultural anthropology is not an “objective” process of verification of results but affective, relational labor of navigating obligations, negotiating gaps, and assessing “contribution” to an always-already skewed scholarly conversation. “Thinking with” during peer review can support Indigenous, international, or working-class anthropologists in understanding what pieces of diverse intellectual conversations happening in various corners of the world still need to be explained and/or translated for the analysis and the argument to make sense to the readers of the journal. The idea for this Emergent Conversation came out of a Facebook conversation, which started after I received a double anonymous peer review where one of three reviewers invested this kind of interpretive labor on my behalf. The reviewer did not doubt my empirical claims or push me to change the argument so as to make it seem relevant in a different frame of reference. Instead, they invested an effort to think with me about how to make the frame I chose speak to a broader set of readers. The need for this kind of immaterial labor, and its effects, is not being discussed often enough, or at least not openly enough. “Thinking with” when peer reviewing might help to share some of this labor more evenly. Sharing such labor is essential for the future directions, and progress, of intellectual conversations globally (Blagojević and Jair 2010; see also Saglam’s contribution to this Emergent Conversation).

It is difficult to say with precision how interpretive labor is predominantly distributed during a peer review. This is partly because what exactly happens during a peer review process remains largely unknown—except as gossip. Without details of the review process, it is impossible to know when an editorial demand for “epistemic humility” (see Appel’s contribution to this series) carries an important request to enter into a dialogue with the reviewers, and when it turns into a form of epistemic silencing. Editorship is a form of sovereignty, as Saida Hodžić argues in her text. Moving peer review into a domain of speakability—transforming the publishing process, including peer review, into a practice that we can discuss publicly, reason about collectively, and demand accountability from—is vital for cultural anthropology that sees itself as a decentered global discipline.

Structural Changes

Whether we engage in “thinking with” when peer reviewing is not just a matter of goodwill, as if we were living in a fairytale where “Reviewer 2” is a villain to vanquish. Structural conditions shape our intellectual exchanges and mold them into certain directions (and most of us have probably been the “evil” Reviewer 2 at times).

“Thinking with” asks for a not-insignificant amount of interpretive labor from a peer reviewer. This seems impossible to sustain as a principle outside of student mentorship, in the current organization of academic work, especially in the profit-oriented publishing industry. As it is, reviewers’ labor goes largely unacknowledged, undervalued, and unpaid, because it is understood as part of “academic service” models that were possibly sustainable only alongside tenured or tenure-track positions decades ago (and perhaps not even then). Editors have increasing difficulties finding the most suitable reviewers. In 2013, journal editors had to invite an average of 1.9 reviewers to get one completed review; by 2017, the number had risen to 2.4, according to a Publons report (2018, 45; see also the Bendix contribution to this series). The Covid-19 pandemic has turned this into an urgent issue. This is why we explicitly asked the contributors to consider what structural conditions must be met for peer review to be improved.

Various ideas emerged. First and foremost, contributors suggest that journal editors must create and enforce clear peer review guidelines, making explicit the demands, the expectations, and the recommendations placed upon a peer reviewer. These explicit guidelines must also outline the scope of editorial responsibility of both authors and reviewers. For an example, see PoLAR Peer Review Guidelines prepared by Jessica Greenberg and Jessica Winegar;[2] as well as Sandberg’s contribution to this series, in which she invites us to envision editing as care-work. Another example can be found in Public Philosophy Journal that explicitly “understands the practice of publishing, including the process of peer review, as a way of creating publics’ and has developed Community Collaborative Review (CCR) as a ‘a healthier, more effective way to review and develop work for publication.”[3]

Second, the labor of peer reviewers should be acknowledged and valued materially, rather than just symbolically. Peer review could count towards tenure, for instance. It could be compensated, in ways that do not further restrict access to publishing and reading scholarly knowledge, like some of the current initiatives to pay peer reviewers (Bohannon 2015).

Furthermore, anonymity could be sidelined—not fully removed, as in open peer review, but temporarily bracketed, with the help of editors. For instance, journals could lift anonymity once the editors reach a decision on whether to publish an article.[4] Maintaining double anonymity, but only temporarily, could help maintain existing mechanisms against bias and increase a sense of connection between the peer reviewer and an author (and thus emphasize reviewers’ responsibility for what they write during a peer review). This is a different model from an “open” peer review, in which the names of the authors and reviewers are known in advance.[5] Yet another model of sidelining anonymity is used by some journals in STEM, which simultaneously rely on “crowd-sourced” peer reviews and on double-anonymous standard peer reviews (Arvan et al. 2022). Crowd-sourced peer review is nowadays being tried out also in humanities/social sciences journals; Ian Cook (2022) offers a good example of how crowd-sourced peer review is being experimented with across disciplines.[6] Biagioli suggests radical approach of “post-publication review,” which is provocative and good to “think with.” However, considering it from the perspective of some academic peripheries (in which edited journals have been frequent and peer review has not become a standard) indicates that placing responsibility for publishing squarely into editorial hands easily becomes conducive of clientelism, hurtful for having challenging conversations, especially in humanities and social sciences.[7]

Fourth, and perhaps most importantly, we could try to thoroughly reorganize the current distribution of labor and profits in academia. Here, it would help to know more about the political economy of academic institutions internationally. In Germany, some positions of a journal editor are being paid like a proper academic job.[8] Paying editors (and authors and peer reviewers) a living wage for their labor would require a total reorganization of the publishing industry in academia—a seemingly impossible goal. Still, would it not make sense to have the profits generated by a scholarly journal invested in the long-term sustainability of that journal, by paying a living wage for the labor invested into its production? This could perhaps help to tackle the current overall lack of academic jobs, as well as to find more time that “thinking with” requires.

Finally, peer review can be used outside of academia too. Some scholars have begun seeking “community peer review,” whereby they go back to communities they studied, host public meetings to present their results, methods, and analysis (Liboiron 2018). In this model, community members can provide feedback, refute findings, and even refuse to allow the research to be published. It sounds like another procedure that could be profoundly beneficial for anthropological research, if only we had enough time for it.

Čarna Brković is Lecturer in European Ethnology and Cultural Anthropology at the University of Goettingen. As a scholar with Montenegrin passport and a PhD from the UK, she works towards transforming anthropology into a decentered global discipline.

Notes

[1] See, for instance: https://medicalhealthhumanities.com/2022/02/07/notes-toward-an-ethics-of-editorial-care/ , https://allegralaboratory.net/who-cares-peer-review-at-allegra/

[2] For example see: https://politicalandlegalanthroreview.files.wordpress.com/2020/08/polar-reviewer-guidelines.pdf (last). Accessed March 27, 2022

[3] https://publicphilosophyjournal.org/about-community-collaborative-review/

[4] I am grateful to Dr. Martin Fotta for sharing this useful idea with me.

[5] In one model of open peer review that I participated, the peer review and the different versions of the article were published alongside, allowing an interested reader to retrace the conversation between the reviewers and the author. See, for instance: https://saw.americananthro.org/pub/care-publicity-and-worker-politics/release/5

[6] https://allegralaboratory.net/category/stuff/call/?month=2001-03&print=pdf

[7] See also Sandberg’s contribution to this Emergent Conversation for a discussion of problems that have emerged in “a world without peer review.”

[8] It should be explicitly stated that Germany has plenty of its own specific problems with academic precarity and the distribution of labor in academia (see, e.g. https://ichbinhanna.wordpress.com/english-version/), perhaps even more than English-speaking academia does. Instead of looking up to any specific academic setting as a model when thinking about how to improve peer review, it would be helpful to understand different political economic structures in the background of different academic settings, with all the (dis)advantages they bring.

Works Cited

Anderson, Benedict. 1991. Imagined Communities: Reflections on the Origin and Spread of Nationalism. London: Verso.

Arvan, Marcus, Liam Kofi Bright, and Remco Heesen. 2022. “Jury Theorems for Peer Review.” The British Journal for the Philosophy of Science, Preprint. https://doi.org/10.1086/719117.

Baćević, Jana. 2021. “Epistemic Injustice and Epistemic Positioning: Towards an Intersectional Political Economy.” Current Sociology, (November 2021). https://doi.org/10.1177/00113921211057609.

Blagojević, Marina, and Gad Yair. 2010. “The Catch 22 Syndrome of Social Scientists in the Semiperiphery: Exploratory Sociological Observations.” Sociologija 52(4): 337–58.

Bohannon, John. 2015. “Editor Quits Journal over Pay-for-Expedited-Peer-Review Offer.” Science. March 27, 2015. Accessed March 27, 2022:  https://www.science.org/content/article/updated-editor-quits-journal-over-pay-expedited-peer-review-offer.

Chappell, Sophie Grace. 2019. “How to Be Somebody Else: Imaginative Identification and the Limits of Ethics, Part I.” The Junkyard of the Mind (blog). February 18, 2019. Accessed June 6, 2022:  https://junkyardofthemind.com/blog/2019/2/17/how-to-be-somebody-else-imaginative-identification-and-the-limits-of-ethics.

Clarke, John. 2019. Critical Dialogues. Thinking Together in Turbulent Times. London: Policy Press.

Cook, Ian M. 2022. Scholarly Podcasting: Why? What? How? London: Routledge.

Esarey, Justin. 2015. “Introduction to the Special Issue: Acceptance Rates and the Aesthetics of Peer Review.” The Political Methodologist. Newsletter of the Political Methodology Section American Political Science Association 23(1): 1–27.

Fricker, Miranda. 2007. Epistemic Injustice: Power and the Ethics of Knowing. Oxford:  Clarendon Press.

Graeber, David. 2012. “Dead Zones of the Imagination. On Violence, Bureaucracy, and Interpretive Labor.” HAU: Journal of Ethnographic Theory 2(2): 105–28.

Hodges, Andrew. 2014. “The Scientific Community: Creating a Language to Deal with the ‘Everyday Geopolitics’ of Neoliberal ‘Transition’ in Post-Socialist Serbia.” Anthropology Matters 15(1): 91–113.

Hodges, Andrew. 2022. Contextual Distance and Exposition in Ethnography and Science Fiction and Fantasy-writing. The Narrative Craft (blog). January 6, 2022. Accessed June 6, 2022:  https://thenarrativecraft.com/2022/01/06/exposition-ethnography-sff/?fbclid=IwAR0EXWhxhSPj9LwtjFS-lF2sZXDus0gfklotBo5vIWYxpoWWlmZdscnGUbI.

Kosofsky Sedgwick, Eve. 2003. Touching Feeling. Affect, Pedagogy, Performativity. Durham, NC:  Duke University Press.

Liboiron, Max, Zahara, Alex, Schoot, Ignace. 2018. Community Peer Review: A Method to Bring Consent and Self-Determination into the Sciences. Preprints 2018, 2018060104. doi: 10.20944/preprints201806.0104.v1).

Publons.  2018. 2018 Global State of Peer Review. Wellington, New Zealand and London: Publons and Clarivate Analytics. Accessed June 6, 2022:  https://publons.com/static/Publons-Global-State-Of-Peer-Review-2018.pdf.

Smith, Karl. 2012. “From Dividual and Individual Selves to Porous Subjects.” TAJA: The Australian Journal of Anthropology 23: 50-64.