TL; DR: PubPeer was born in 2002 after Brandon Stell, George Smith, and Richard Smith decided to do something about the conflicts of interest prevalent in leading academic and scientific journals. The trio understood that the process of correcting a published scientific paper is long and arduous with much at stake for a number of parties. On one hand, researchers are sometimes hesitant to admit errors when careers are on the line. Meanwhile, for-profit journals want to sell copies, not ignite controversy, and universities and research institutions seek to attract funding, not lose grant money. PubPeer aims to change this dynamic. We recently sat down with Brandon, who told us how PubPeer cultivates post-publication collaboration by removing the middleman and fostering community-based peer review.
In 1998, Andrew Wakefield published an infamous article in The Lancet, one of the world’s oldest and best-known medical journals. His paper claimed there was a link between the administration of vaccines and the appearance of autism. Even though researchers couldn’t reproduce Wakefield’s findings, anti-vaccination advocates used the article to convince parents not to vaccinate their children. The movement caused a reemergence of measles and other vaccine-preventable diseases, and The Lancet didn’t publish a retraction until 2004.
But, in 2017, when a similar article appeared in The Journal of Inorganic Biochemistry linking vaccines to autism, the time to correction was drastically reduced. Scientists who were part of the PubPeer community studied the paper and discovered that images and data had been manipulated. The result? The article was retracted roughly a month after it was published.
“I am proud of results like that,” said Brandon Stell, PubPeer’s Co-Founder and President. “The scientists who visit our site quickly provide feedback, and that means the scientific community as a whole is able to avoid major problems.”
Founded in 2012, PubPeer is a nonprofit that allows users to search for any published scientific paper by using a serial number or digital object identifier, and then comment on that paper. Used by a large community of scientists, it has become a widely discussed and increasingly important part of the peer-review process.
Conflicts of Interest and the Difficulty of Correcting Academic Articles
Brandon told us all scientists read papers and have critical discussion groups focusing on published work. And it’s very common for them to find errors — sometimes significant ones.
“That’s why we started PubPeer,” he said. “It was a way of saying publication shouldn’t be the end of the scientific process.”
Brandon noted the publication, university, and scientific grant systems have a number of conflicts of interest that prevent correction of the scientific record.
“Before PubPeer, scientists who discovered errors had a few options, but none of them usually resulted in any changes,” he said. “First, they could write an email to a paper’s authors and tell them, ‘I think you’ve made a mistake.’ But authors aren’t interested in ruining their reputations by admitting to errors, so those emails usually get a polite response and nothing more.”
Another option is to write to the journal that published the problematic paper. Occasionally, journals publish letters to the editor, but that can be an extremely slow and painful process. And Brandon told us it’s very rare that journals ever actually publish an admission of error.
“Their conflict of interest is that they’re creating a product, and they don’t want to admit it has any problems,” he said. “Because they’re for-profit, their goal is to sell as many journals as possible. Their primary interest is to publish high-profile research that will be reported on NPR or in The New York Times, et cetera.”
Scientists can also report errors to the institutions that host the lab where the studies were done. The problem, Brandon said, is that the institutes also have no interest in correcting mistakes.
“Their researchers bring in grant money, and the universities and research centers don’t want to risk that income,” he said.
From scientists seeking to protect their reputations to gigantic institutions safeguarding needed grant money, competing interests often stand in the way of revising published work. PubPeer makes it possible to correct mistakes without having to go through any traditional gatekeepers.
Improving Scientific Accuracy by Cutting Out the Middleman
If a scientist wants to say something about any paper from any discipline, they simply log on to PubPeer and make a comment.
“Rarely, a paper’s author will put their work on our site and say, ‘Hey, I just published this, and I’m looking for comments,’” Brandon said. “But usually it works the other way around.”
In physics and chemistry, researchers have an established pre-publication tradition of putting data online for review. Colleagues read and critique studies before they’re sent to journals. Hosted by pre-print servers, including Princeton University’s The Archive, this tradition is growing. Now, fields like biology are adopting the practice.
“In PubPeer, you can comment on preprint papers also,” Brandon said. “Whether it’s a study published in Nature or one that isn’t even in a journal, you can come to our site and point out anything you’ve noticed.”
On PubPeer, scientists who want to correct a paper’s mistakes don’t have to wait for a middleman to approve their critique.
“As soon as a scientist spots an issue, they can post that on our site,” Brandon said. “Then, anyone who is interested in that paper can be alerted to the problem.”
Some scientists are also critical of how publication in a prestigious journal can make mediocre articles or findings seem more important than they are. Conversely, important work and papers are sometimes overlooked because they’re published in lesser-known journals.
“PubPeer solves this problem by putting the focus on the article itself, not the journal where it’s published,” Brandon said. “If Nature publishes a subpar article, our community can point it out. And if the community raves about an article in a so-called lesser-tier journal, that could increase the paper’s profile.”
A Unique Community: Why Anonymous Comments Are Good for PubPeer
When PubPeer was launched, it wasn’t completely anonymous. Brandon told us the site allowed anonymity, but commenters still had to create accounts to participate, which meant they were theoretically identifying themselves to the PubPeer team.
“Some scientists were hesitant to do that possibly because they were worried that critiquing an article by a famous or powerful colleague might derail their careers,” he said.
Brandon and his team watched the comment sections on other sites devolve into wastelands of hateful memes, but they thought PubPeer might be different.
“They’re not people commenting on cat videos,” Brandon said. “They’re scientists commenting on data. Anyone who wants to make a convincing case is going to have to use solid, fact-based arguments in order to convince other scientists. So we made the change and allowed visitors to comment with complete anonymity.”
While anonymity comes with risks, Brandon feels that the risks are worth taking.
“It’s really important to get this movement started,” he said. “We want scientists to build a vast body of information, which we hope will eventually become at least as valuable as the journals where these articles were first published.”