Notes

Anonymous Internet Vigilantes Are Taking Peer Review Into Their Own Hands by Online Security

Since 2012, the message board PubPeer has served as a sort of 4chan for science, allowing anyone to post anonymous comments on scientific studies. Originally intended as a forum for the discussion of methods and results, PubPeer has perhaps become best known as a clearinghouse for accusations of scientific error, fraud, and misconduct—forcing journals to issue corrections and retractions, damaging careers, and eventually embroiling the site in a court case in which it’s advised by Edward Snowden’s legal team at the American Civil Liberties Union.

In the view of its critics, PubPeer enables an unchecked stream of accusations with no accountability. But to its supporters, PubPeer is maybe the only consistently effective way to expose fraud and error in the current scientific system. It exists at a time of quiet crisis for science and science journals, when the community is concerned about an inability to replicate past results—the so-called “reproducibility crisis”—and the number of papers retracted is on the rise. The traditional system of peer review seems unable to address these problems.

“We started it because we wanted more detailed arguments about science, and we were really shocked at how many fundamental problems there are with papers, involving very questionable research practices and rather obvious misconduct,” said Brandon Stell, a neuroscientist at the Centre National de la Recherche Scientifique in Paris and the creator of PubPeer.

There’s certainly no denying its effect. According to Retraction Watch, a blog that monitors scientific corrections, errors, and fraud, at least three high-profile scientists in the past few months have had their studies retracted by journals after their data was questioned by anonymous commenters on PubPeer.

The most frightening words a researcher could read on PubPeer are 'There are concerns'

One of the scientists, Fazlul Sarkar, is currently suing several of the commenters. His lawyers argue the site must reveal the identities of the users that have done damage to Sarkar’s career, after he lost a tenured position at the University of Mississippi. PubPeer has refused to release the information. Both Google and Twitter have filed a court brief in support of the site, which is currently being defended pro-bono by lawyers from the ACLU.

It’s perhaps the most interesting case about internet privacy you've never heard of, and it all stems from a frustration among scientists with the shadowy politics of publishing and peer review.

At its base, PubPeer is a site that allows anyone to post comments on any scientific paper listed on the federally-funded PubMed database, either anonymously or under their own name. It’s functionally very simple, but the built-in anonymity makes it a safe outlet for scientists—especially young, early-career scientists—to discuss and criticize research without fear of repercussion. And that’s something they’re apparently eager to do: The site has logged over 55,000 mostly anonymous comments since its launch.

Back in October 2013, someone on the PubPeer site started threads for about 20 previously published papers on which Fazlul Sarkar, a cancer researcher then at Wayne State University in Michigan, was an author. The papers span over a decade and involve a variety of complex molecular signalling pathways involved in cancer. The issues raised by the comments, though, were relatively straightforward: They claimed that images in these studies appeared to have been changed, duplicated, and re-used across papers, suggesting that the experiments they appeared in may have never actually happened, or could have produced different results.

Stell noted that, in an effort to keep the discussion civil (and legal), PubPeer specifically requests that users do not accuse authors outright of misrepresentation or fraud. Comments are moderated in case they break these guidelines, so any discussion of such allegations tends to have a muted tone.

That doesn’t make this group of self-appointed watchdogs any less effective, though. The most frightening words a researcher could read on PubPeer are “There are concerns.”

Discussion over “concerns” surrounding Sarkar’s work expanded rapidly as it became clear the commenters had found a rich vein to mine: According to the NIH funding database and PubMed, Sarkar has received more than $12 million in NIH funding and authored over 500 research papers over his career. The community is nothing if not meticulous—PubPeer commenters have been known to pull up decades-old PhD theses looking for dirt—and a search of the message board shows that eventually 77 papers with Sarkar on the author list were presented for scrutiny. By checking the papers against each other patterns began to emerge; for example, one user claims a single set of images were duplicated up to 54 times in 13 papers, across three years.