By John Clarke

Emergent Conversation 16

This essay is part of the series PoLAR Online Emergent Conversation on Peer Review as Intellectual Accompaniment

Peer review remains one of the most frustratingly mysterious parts of academic life. For the most part it is profoundly non-dialogic, performed by “peers” who too often see themselves as guardians (of standards, of the discipline, of quality). Reviews are typically dispensed in the form of disembodied judgements. Having spent an academic lifetime encountering the process as both reviewer and reviewed, I have rarely experienced it as a relational encounter between the parties. And despite my age, I know that I am still not immune to being infuriated by the experience. Last year, I received a review on a proposal which I found upsetting (at different points turning what my proposal called conjunctural analysis into “conjuncture theory,” “conjunctional thinking,” and “conjectural analysis”) while turning ignorance into critique. I occupy a relatively privileged position, no longer having a career at stake. So, having worked with the editor before, I complained about the review’s inadequacies. We managed to resolve the problem—but at no point did this feel like a peer review.

That experience took me back to one of the most instructive days of my working life, when our entire department (postgraduate students and staff) gathered to discuss the practices and relationships of peer review. We were all invited to talk about our experiences (as reviewers, reviewed and, in some cases, editors) and to bring examples. Many genuinely horrible examples were shared, typically characterized by a tone that fell a long way short of collegial. Younger members of the department were shocked to hear that established people also got bad reviews—and that they got upset by them. Those who were involved in reviewing were pulled up short by the examples and were prompted to think again about modes of address, tone and style. Journal editors reflected on what their role was—and what it might become, not least in terms of thinking about who to ask to be reviewers (and who to avoid or not re-use), and what guidance might be given to reviewers.

In the course of the day, we devised three practical rules:

First, get someone else to read the review you have just received, even read it out loud to you. This came about after one (senior) person talked about kicking walls, swearing, and shouting, after one unpleasant review came in—meaning that they missed the very short sentence about the piece of work being accepted.

Second, we thought all reviewers should read their draft review and ask themselves about how they would feel if they received it. The temptation to be clever, profound, or just plain superior seemed to arrive all too easily in the process of writing.

Third, we thought people writing reviews should adopt what was known locally as the “Open University sandwich.” Staff responding to students’ written work (this was a distance teaching system, centred on written assignments) were expected to open their responses with a positive statement, even if this was only a thank you or congratulations for meeting the deadline. The comments were expected to be a mix of critical and expansive, written with the purpose of helping the student to improve next time. Finally, the sandwich was to be completed by a validating comment.

All three of these suggestions still feel relevant, partly because of their continued absence from much of the peer reviewing process. I don’t apologize for how banal they are because I think we start from a very low baseline. But that day also taught me something else: about how little—as members of the much vaunted “academic community”—we actually talk about this process and the relationships at stake in it.  The small proposals that we made only address part of the question about peer review: they are more about making the process more civil (and less painful) but do not address whether the process might be made more dialogical.

Peer review is structured in a profoundly non-conversational way. It takes place in a very different temporal framing from conversation. There is an object, usually in writing, which eventually receives a response in writing. Sometimes, the original author(s) are invited to respond to the review (in resubmitting journal articles) which may be taken into account in the subsequent review—and so on. This temporality has the tendency to fix or freeze things which might, conversationally, remain open for further reflection, amendment or clarification.

This structuring generates different issues that need to be addressed and they are ones that arise from the institutional arrangements of academic work that we inhabit. Some of the problems derive from the romantic haze that surrounds the image of the “peer” in peer review. Who do we imagine ourselves and our peers to be? How might we go about the business of addressing them? Some of the troubles arrive from forms of intensification: the pressure to publish and to publish quickly (in career terms); the increasing demands for (and on) peer reviewers, especially as the number of journals grows; the fragmenting and precarious academic job market and so on.

There are also continuing formations that have consequences: the systems of hierarchy and what Cook (2022) calls “prestige” in institutional and personal terms, and the disciplinary formations that shape what counts as knowledge. The latter also have their own structures of hierarchy: I recently found myself reviewing a research proposal by an economist on parental choices to have children. Despite employing the “Open University sandwich” as a format, I noted that the proposal appeared not to know anything about the vast literature written by historians, sociologists, policy scholars, and, yes, anthropologists who had important things to say about these processes and relationships. I hope I was civil but was troubled by the ways in which disciplinary boundaries (and hierarchies) are not conducive to producing useful knowledge.

In the face of such structural and cultural arrangements, is it possible to make the processes be made more dialogical and (potentially) productive? I was fortunate to be brought up (in academic terms) in what were remarkably collaborative and conversational styles of working but have come to realize that these cannot be romanticized and fetishized (as though their material conditions and relations do not make a difference). At the heart of these practices was a senior scholar with a deep-rooted commitment to fostering collective ways of working and thinking. In his remarkable book on Stuart Hall’s Voice, David Scott (2017) has described Hall’s conversational style as a practice of “clarification”:

Clarification is a way of approaching thinking—and learning—that aims to make us more aware of what we are thinking or doing … That is to say, clarification involves endlessly saying the next thing, never the last thing. Clarification therefore does not presume the possibility of resolution; on the contrary, there is no presumption of closure, only successive provisional resting points along the way where we gather our thoughts for further dialogic probing. (Scott 2017; 16, italics in original)

It may be that such a view of thinking together would tilt the balance too far towards “peer” and leave out the “reviewing” part of the process. But it does point towards some questions: about how dialogue works; how dialogue might be imagined as taking place over time and distance and in a non-spoken form; how we think of the purposes of reviewing; how to make sense of the relationships between dialogue and judgement; and how to think about the ways in which the “next thing” is envisaged—or implied—in the process of peer review.

The conversational style that Scott describes is one that invites thinking out loud and in a way that assumes dialogic relationships: that is, the listener and speaker will change roles, “taking turns” in socio-linguistic terms, to advance a common interest in the topic at hand. Peer review disrupts these possibilities in almost every way imaginable. There is no “common interest” in play: rather a distinction exists between the work and the commentary. However, the possibility of “clarification” might still exist, if the reviewer has an interest in advancing the project at hand rather than merely judging it. Treating reviewing as clarifying can be productive and there are devices through which it can be made “quasi-conversational”: offering alternatives to reinforce or deepen the analysis being offered; enriching the resources in play and so on. Good reviewers (in my experience) put their expertise at the service of the author(s) and there have been times when I have learned a lot from such comments.

Can such structural and cultural features be overcome? Not readily, given the imperatives that the review process strains to reconcile: the desire to be published (or funded); the desire to get the piece or proposal reviewed, the desire to get the review off the list of jobs, and so on. At its heart, though, peer reviewing operates in a space of tension between its “peer” aspirations and its commitment to evaluation, judgement or, more brutally, “gate-keeping.” Reviewers may try to juggle those two obligations in different ways but organizations seeking reviews prioritize the second, desiring reasoned gate-keeping rather than sponsoring peer-to-peer developmental processes. As a result, “judgment” is always in command and the drift to judgmentalism is always a risk. Journal publishers, funding agencies and others may reasonably insist that dialogue, development and related aims are properly the responsibility of others—academic departments, research teams and the like. But I think these organizations that exist to develop, circulate and advance “scientific work” in its largest sense also have responsibilities to promote good practice.

While it may be impossible to entirely dissolve the tensions between the evaluation objective and peer to peer modality, there are several fronts on which forms of good practice (or even better practice) might be developed. The following suggestions aim to build on the suggestions about reviewing etiquette developed by my OU colleagues. There the aim was to stress writing for civility, checking for tone and de-privatizing the experience of being reviewed.

More widely, though, would it be possible for academic units (departments, centres, teams) to institutionalize peer to peer reviewing as a normal practice of academic development? This might enable newer scholars to present themselves to journals and other bodies as having had experience and training. It might enable longer established staff to think again about how they approach reviewing—and being reviewed. Creating reviewing circles might help to develop skills, confidence and even pieces of work which might also create the conditions for sharing the experience of being reviewed.

Is it possible that reviewing organizations (journals, funding bodies, etc) might make their expectations of good practice in reviewing explicit—and act on them? For example, editors and commissioners might take a more active role in mediating the content and outcome of reviews. They could also create the possibility for those who have been reviewed to respond to the reviews—not instrumentally as part of a resubmission process—but as an integral part of a process of dialogic reviewing.

Finally, in keeping with Scott’s idea of dialogue as clarification, is it imaginable that reviewers might be explicitly invited to think with as well as against what they are reading? That would emphasize a role in helping to develop arguments and analyses rather than revealing their flaws. In academic life, it is both too easy and too common to think that the adjective “critically” only means thinking against.

And just in case this comes out sounding smug, let me end with a salutary tale. While teaching a graduate class at Central European University some years ago, I was struck by how ruthless participants’ critical reviews of our readings were. They had certainly learned very well how to “think against.” I asked that, in future, those introducing the readings should draw attention to at least one thing they had found valuable or productive. A few weeks later, I was explaining why I found one of the readings so frustrating when I became aware of a wave of quiet laughter. When I stopped and asked what was wrong, the bravest member of the group said: “Well, we were just wondering if you had found anything good to say about it?”

John Clarke is an Emeritus Professor in the Faculty of Arts and Social Sciences at the Open University and a Leverhulme Emeritus Fellow (2019-2022). His work has ranged between social policy and cultural studies and recent publications include: Making Policy Move: Towards a politics of translation and assemblage (with Dave Bainton, Noémi Lendvai and Paul Stubbs; Policy Press, 2015) and Critical Dialogues: Thinking Together in Turbulent Times, based on a series of conversations with people who have helped him to think (Policy Press, 2019). He is currently working on a book provisionally titled: The Battle for Britain: Crises, Covid and Culture Wars (Bristol University Press).

Works Cited

Cook, I.M. 2022 “Fuck Prestige.” In Opening up the University: Teaching and Learning with Refugees, edited by Céline Cantat, Ian M. Cook, and Prem Kumar Rajaram, 209-219. New York and Oxford: Berghahn Books.

Scott, D. 2017 Stuart Hall’s Voice. Durham, NC: Duke University Press.