“The Grievance Studies Affair” Project: Reconstructing and Assessing the Experimental Design

Recently, high media visibility was reached by an experiment that involved“hoaxlike deception” of journals within humanities and social sciences. Itsaim was to provide evidence of “inadequate” quality standards especiallywithin gender studies. The article discusses the project in the context ofboth previous systematic studies of peer reviewing and scientific hoaxes andanalyzes its possible empirical outcomes. Despite claims to the contrary,the highly political, both ethically and methodologically flawed “experiment”failed to provide the evidence it sought. The experiences can be summed upas follows: (1) journals with higher impact factors were more likely to rejectpapers submitted as part of the project; (2) the chances were better, if themanuscript was allegedly based on empirical data; (3) peer reviews can be animportant asset in the process of revising a manuscript; and (4) when the project authors, with academic education from neighboring disciplines,closely followed the reviewers’ advice, they were able to learn relativelyquickly what is needed for writing an acceptable article. The boundarybetween a seriously written paper and a “hoax” gradually became blurred.Finally (5), the way the project ended showed that in the long run, thescientific community will uncover fraudulent practices.