By Dada Docot

Emergent Conversation 16

This essay is part of the series PoLAR Online Emergent Conversation on Peer Review as Intellectual Accompaniment

In 2015, I submitted a manuscript to a Tier 1 journal in anthropology with a single-blind peer review process which means that the author is known to the readers, but not the other way around. I received feedback from two reviewers. Reviewer 1 appreciated the “unconventional style” of my paper which began with an invitation to watch a short film. They wrote, “The author is a good writer, she makes her point clearly.” Reviewer 2 sent comments and an annotated copy of my manuscript. They described my writing as “way too soft” which they attributed to a “cultural and gender issue.” Additionally, they felt that my manuscript needed to be substantially edited into a form that would be more “suitable to, or conventional” for the “hard core journal” to which I had submitted my work. They remarked that my discussion presented an “interesting twist,” and recommended that I focus on a particular theme from my ethnography, and revise my paper following their outline.

Summarizing the reviews, the editor said that the reviewers found “many strengths” in the manuscript which was “precisely the sort of piece” that the journal was looking for. The editor wrote that the reviewers’ recommendations would help improve my manuscript which was “already quite strong.” They also recognized that some of Reviewer 2’s comments on my ethnography were “a bit problematic” so they would “leave to [my] judgment as to how to address this issue.” Encouraged by this first set of reviews, especially by the positive comments of Reviewer 1 and the editor’s note about having the choice to address Reviewer 2’s feedback selectively, I revised my manuscript. Two professional editors and many peers read the revision before resubmission.

In the second round of reviews, Reviewer 1 wrote a one-sentence acceptance on what they thought was a “good revision.” However, Reviewer 2 completely changed their tone, strongly opining that my paper was “not ethnography,” that it had “no method,” and “no discussion of social relations.” Unimpressed, Reviewer 2 found that a discussion “à la Goffman” would have been more fruitful. Additionally, they deemed that my use of concepts from Philippine personhood theory was an “extreme hyperbolic exaggeration.” On my discussion about my own gender and racialized positionality, in which my paper is grounded, Reviewer 2 wrote that my arguments seemed to have taken the route of discussing “everything to do with the semiotics of race identity.” They heavily criticized my grammar, the manuscript’s arguments, flow, and contribution to the field. There was not a single positive comment on the revision. Palpably angry, Reviewer 2 even wrote profanities such as “What the f…” and “What the heck” in the annotated copy. Reviewer 2 groaned on in his comments, needlessly nitpicking, mostly on my syntax and word choice: “I am barely able to continue to read;” “At this point I am ready to put this article away,” among others. Their highlights and comments hinted at the entire text being almost incoherent.

The editor wrote that they acknowledge that the reviews that I received were “mixed” and that they failed in their attempt to find a third reviewer to mediate between these two wildly different responses. Unfortunately, without a third review, Reviewer 2’s sexist, inconsistent, implicitly racist review of my work, which included a verbose critique of my paper’s structure, syntax, and word choice, overshadowed Reviewer 1’s succinct but brief acceptance. Unable to find a mediator, the editor sided with Reviewer 2, writing that I did not do enough revision even if I had turned the contents of my paper upside-down following Reviewer 2’s outline. In the rejection letter, the editor wrote that the “main weight” of the comments from the two reviewers appears to be “negative.”

I wrote a three-page response to the editor to acknowledge and accept the rejection, and also noted the wide disjuncture between the reviews, Reviewer 2’s biases and inconsistent expectations, the journal’s lack of communication in their attempt to search for a third reviewer, and my concern that a top journal in anthropology has allowed a reviewer to communicate with a colleague with disrespect. I provided detailed examples to substantiate my points. I did not demand for a third reviewer nor requested for the editor to reconsider their decision. Instead, I stated my reasons for wanting to participate in the debates in the journal. I briefly pointed out the editor’s recognition that Reviewer 2’s comments were inconsistent with the goals of my manuscript. Finally, I thanked the editor, the managing editor and the reviewers, and requested that my response letter be forwarded to both reviewers for their reference.

The editor responded and offered two options: 1) the manuscript could be sent to a third reviewer, or 2) I could choose to revise the manuscript incorporating the comments from the previous reviews, but this would be considered a new submission. I took some comments from Reviewer 2 about defining concepts, adding suggested citations, but reverted to the original flow of the manuscript with the non-conventional structure that Reviewer 1 had appreciated. My manuscript was treated as a new submission and sent to Reviewer 3.

Reflections

I write these reflections on this experience of the peer review process to think about some of the features of a potentially abusive and harmful process. I write as an untenured assistant professor, keeping my fingers crossed that this essay will not place upon me very heavy repercussions. In academia, we commonly say that peer review is a bastion of the scientific tradition. However, just like all traditions, as we study them in anthropology, the peer review process is riddled with enduring myths about what counts as “proper” scientific rhetoric and acceptable ontologies, and that to be successful, one must endure academic hazing.

Various disciplines have certain standards which gatekeepers expect new initiates to meet. Academic writing often demands certain scientific rhetoric. Reviewer 2 wrote that the “softness” that was so pervasive in my text “only points towards not understanding the scientific-disciplinary anthropological merits of the contribution versus a demonstration that nothing is certain.” They wrote that I must use “conventional” rhetoric that was “appropriate” to the venue which they noted “has [a] mission of science production of new knowledge.” Reviewer 2 felt that my chosen tone lies outside standard rhetoric, fortifying expectations about how academic writing must look and read like—hard, objective, and emotionally opaque. Sara Ahmed (2015) reminds us that the metaphor of “soft touch” implies a weak, feminized body which means that softness is not an aspiration (2).

Reviewer 2 wrote that my paper is missing a structure and suggested that my revision be structured in a “straightforward manner.” Meanwhile, Reviewer 1 commented on their appreciation for the paper’s organization “in the style of a conceptual art piece.” Reviewer 1 added: “I think it works.” Reviewer 2’s feedback alludes to their preferred masculinist rhetoric which is based on racial and sexist biases while also ignoring the long and growing genealogy of feminist and decolonizing works in anthropology which have challenged the ways of doing and writing ethnography. Confidentiality in this type of peer review is meant to give the reviewer space to provide feedback without pressure. However, a single-blind review process is founded on assumptions of objectivity, which Dianne K. Lewis (1973) writes developed the scientific canon of objectivity that supported a colonial system and that shores up the “traditional relationship between anthropology and its nonwhite subject matter” (581). Reviewer 2 suggested that I adopt the language of the “scientist as master of all.” This “scientist as master” was to elide softness as it read to them like “false humility,” and thus, they strongly recommended that I drop the use of “hope/want/seek” throughout the paper. One who wrote with “softness” like me, and who referred to the personal experience did not read like a “scientist.” It seemed that for Reviewer 2, a scientist is not an autoethnographer. Following my discussion on the emotional weight of anthropological auto-ethnography as opposed to participatory or observatory methods with observation, Reviewer 2 wrote that “there is nothing inherently critical” about using self-reflexive strategies in ethnography. However, have we not heard of so many stories about scholars whose self-reflexive works have been dismissed as mere products of navel-gazing and are therefore not research data? Lying at the heart of Reviewer 2’s supposed well-meaning recommendations to improve the tone of my arguments is a form of intellectualism that polices and punishes other forms of articulation and knowledge.

In my view, waving the banner of “hard core” science cloaks the underlying racism in the academy. I recall here Charles Leslie’s (1990) article in which he writes about the ideology underpinning the peer review process. He discusses an article that was published despite the author using eugenicist arguments in explaining differences in susceptibility to AIDS based on behavior, intellectual ability, and morality in different “races.” The fact is, Leslie argues, many academics who have published in the journal in which the eugenicist article was published belong to the positivist tradition which elicits persuasiveness through scientific vocabulary and hard data (903). Leslie argues that the peer review process often uses the pursuit of knowledge and freedom of expression to justify scientific racism. Scientific rhetoric plays a role in perpetuating the power imbalance that is deeply ingrained in the academy.

I was confused by all the academic scolding and the dismissal of my work that allegedly had “too much meanderings and precious style going on.” I thought Reviewer 2’s commenting style was so different from the reviews of my work that I had received until then from my graduate committees composed of feminists and scholars of indigeneity. During my revisions, I felt compelled to just hit CTRL+F to search for all instances of “hope” in my paper, to delete them one by one. After accomplishing this task, I told my friend laughing, “My paper has absolutely no hope now.”

Reviewer 2’s disparaging remarks, I realized, were rooted in something deeper than style of expression or language. Reviewer 2’s bitterness towards my work, was linked to their assumptions of what they called “cultural and gender issue.” In the annotated copy, Reviewer 2 wrote, “The ultimate message is a moralization of your position and scolding of other anthropologists who do not do the work that you do, and imagine, if they did then you would not have anything to say or to study because others would have already monopolized that topic.” Reviewer 2’s comments revealed their bias against non-Western ontologies, pushing my analysis in the direction of the anthropological mainstream. On my discussion of political art and ideology in Latin America, they wrote, “Makes me laugh!”

By submitting to the peer review process, we sign an invisible contract to be part of a community of scholars whose members will provide honest and helpful feedback. We let go of our ideas and hand them over to a stranger. Critiques of single-blind review note its exposure of the author’s identity to reviewers, which might lead reviewers to make assumptions based, for example, on the author’s non-Western name or gender. Furthermore, some authors who write self-reflexively or who research within their own community and family using autoethnographic methods may be easily identifiable. Thus, the review process becomes automatically single-blind for them. Reading the work of a scholar of color reflecting on her struggle with multiple forms of locatedness, exclusion, and colonial histories, Reviewer 2 saw only gibberish.

Overall, my initiation into the anthropological peer review process was a traumatizing one that could be called a form of academic hazing. It took too long, against the initial expectation. As Saida Hodžić writes in her essay in this collection, the truth is that it is the editor who decides on the fate of the manuscript and the reviews. Editors’ decisions make or break careers. As editors are highly respected members of the academic community, many authors may not bother to question their decision. In my case, the editor was swayed by the very long opinion of Reviewer 2 as opposed to the short positive one, which is telling of how the academy tends to nurture negativity rather than nourish potential.

Reviewer 2’s absolutist critique of my revision as not ethnographic at all should have been articulated at the very beginning, as I would not have attempted a revision in the face of a reviewer that saw no value in auto-ethnography, self-reflexivity, and the lived experience. I felt beaten into a form that I was reluctant to take. Trying to follow the suggestions from the first round of reviews resulted in a dramatically more negative response which shocked me and made me feel that the process is structured to keep people like me away. The editor forwarded Reviewer 2’s report without removing the profanities. I was a Ph.D. candidate when I submitted my manuscript. However, I felt very strongly at that time that Reviewer 2’s comments on my work and the editor’s decision did not seem just. I was saddened by the thought of others treading the same traumatizing path. Reviewers’ participation in the peer review, even if harmful, ends up as an item in their curriculum vitae’s “service to the profession.” It did not feel right to give up even if many peers advised me to take my manuscript elsewhere.

Richard Smith (2006), who was editor of the high-impact medical journal BMJ for 13 years, has a lot to say about the peer review process. He writes, “So we have little evidence on the effectiveness of peer review, but we have considerable evidence on its defects. In addition to being poor at detecting gross defects and almost useless for detecting fraud it is slow, expensive, profligate of academic time, highly subjective, something of a lottery, prone to bias, and easily abused” (179).

Under Smith’s editorship, the BMJ experimented with different ways to improve the peer review process, such as blinding reviewers to the identity of authors, reversing the single-blind peer review process, opening the peer review by publishing the names of reviewers on the journal’s website once an article is published, and formally training reviewers—but all of these experiments minimally affected the quality of reviews. Smith therefore concluded that the process is flawed and “full of easily identified defects with little evidence that it works” (182). Despite the evidence about the imperfections in the peer review, including the many possible avenues for abuse, dishonesty, and intellectual theft, Smith recognizes that the peer review will likely continue to hold weight in the academy because of the unwavering faith of the academy in its merit. He concludes his article, “How odd that science should be rooted in belief” (182).

Sulong! (Forward!)

Supporters of peer review would argue that it is an open space where everyone from any rank, place, and positionality could participate. However, the tools of our discipline continue to change. Many practices which were normative in the past, such as helicopter research or armchair anthropology now stand as examples of what should never be done again or must eventually change. The review process is flawed and problematic, and that, too, must be reimagined. Junior scholars are also often told they must publish in the discipline’s top journals to get tenure, and our publications in other spaces (interdisciplinary, open access, public-facing) count for less. The premium that our senior colleagues place on elite anthropology journals justifies and reinforces peer review traditions, despite their problematic practices.

Overall, it took 36 months for the journal to process my publication from submission to acceptance. I learned from this process that negative and unreasonable reviews of our work, as well as rejections, do not necessarily shut down the gates entirely. Persistence and insistence on fairness sometimes do indeed yield results, but advocating for oneself and for change can be exhausting and potentially disastrous to early-career scholars. On the one hand, it could be said that the peer review process worked in my case as the editor reacted positively to my response to the journal rejection by offering options. On the other hand, authors should not have to spend such a long time banging on the gates of academia, in an effort to be let in. Despite seeing my article go to press, the scars remain, and the experience has made me a supporter of open and experimental journals looking to change the process. My recent experience with Commoning Ethnography was nourishing. In a process that they call a “peer-engaged review,” the reviewers have the option to reveal their names, and turn the review process into mentorship, creating opportunities for open dialogues and peer-making. Anonymity allows for abuse, which editors do not always moderate successfully.

While I told myself to just carry on through every new twist and turn of the process leading to publication, I had not expected it to be so dispiriting in the end.

Dada Docot, a diasporic Filipino scholar, is an assistant professor of anthropology at Purdue University and a postdoctoral fellow at the Tokyo College, the University of Tokyo. Dada is currently working on her first book manuscript which is an ethnography about the effects of overseas migration in her hometown in the Philippines.

 

Works Cited

Ahmed, Sara. 2015. The Cultural Politics of Emotion. Edinburgh: Edinburgh University Press.

Leslie, Charles. 1990. “Scientific Racism: Reflections on Peer Review, Science and Ideology.” Social Science & Medicine 31(8):891–905. https://doi.org/10.1016/0277-9536(90)90025-N.

Lewis, Diane. 1973. “Anthropology and Colonialism.” Current Anthropology 14(5):  581–602. http://www.jstor.org/stable/2741037.

Smith, Richard. 2006. “Peer Review: A Flawed Process at the Heart of Science Journals.” Journal of the Royal Society of Medicine 99(4):  178-182.