From Library Babel Fish, October 10, 2013
We often judge information by the company it keeps. A story in The New York Times is more likely to be taken seriously than a news story that was published in a small town paper. A university press’s reputation is built by the strength of its list. It’s the principle underlying the “impact factor,” as flawed as that measure is for assessing the worth of any particular paper published in a “high IF” journal. An article in Science has a lot of clout because it’s published in Science. That journal is near the top when it comes to prestige. But it’s not perfect. It has come in for a lot of criticism for an article it published recently. Its author, John Bohannon, went to a great deal of trouble to establish that many journals today are accepting papers with little sign of peer review or with peer review so ineffectual that it fails to catch errors that should be obvious.
Bohannon has come in for quite a lot of criticism for his article, which was written while wearing his journalist hat, though he has a PhD in molecular biology to go with extensive credentials as a journalist. (Both Science and Nature combine well-respected scientific research with news and opinion pieces in an effort to keep scientists abreast of significant new research and current reporting and commentary on the world in which scientists work.) A major criticism is that he focused on open access journals exclusively and drew conclusions about them without establishing a control group. The problem was compounded when a news release from Science exaggerated the study’s meaning in a press release. Sal Robinson, at the Melville House book blog, adds a pointed critique that when Bohannon represented himself as a group of authors from an African country, running the text through Google Translate to make the English suitably substandard (part of Bohannon’s methodology), makes it seem the problem is at least in part the fault of third world authors getting in over their heads and partly the fault of publishers who take advantage of ill-prepared would-be scientists with sketchy credentials. Gunther Eysenbach, an open access journal editor who refused to pass the article along to reviewers, but whose refusal didn’t appear in the study’s data, wonders if the author left out inconvenient data. Michael Eisen, who describes himself as “a strong proponent of open science,” responded to the sting with a blog post about how the author of a badly-flawed paper fooled Science, which accepted it without detecting serious flaws. Eisen argues that the problem is with peer review itself, not with open access publishing. Blaming the problem on open access, he writes, “is like saying that the problem with the international finance system is that it enables Nigerian wire transfer scams.” Mike Taylor rounds up the critiques and adds some of his own at Sauropod Vertebra.
It is undeniably true that a lot of silly scam operations apparently are profiting from the way we