Those of us who regularly seek out information about scientific and medical subjects have learned there’s a hierarchy of reliability.
For example, the “facts” in a random blog item or a news release from a special-interest group are on average less trustworthy than, say, those from the Web site of the National Institutes of Health.
Likewise, the term “peer-reviewed journal”–which means articles have been reviewed anonymously by independent experts before being accepted for publication–elicits instant respect from many people.
But not from me, because I actually read many of those journals.
Peer Review Shortcomings
I’ve found it is not uncommon for articles that are egregiously, obviously flawed–most often because the basic rules of scientific research have been violated–to find their way into prominent scientific publications.
Some of the worst of these papers have conveyed false alarms about the safety of gene-spliced (“genetically modified”) plants that subsequently have been extensively reported in the popular press. In this way, misinformation is widely propagated and feeds the propagandizing of anti-technology activists.
Methodology Flaws Ignored
A case in point is a 2001 paper in the British journal Nature purporting to show genes from a pest-resistant, gene-spliced variety of corn had migrated into native corn plants in Mexico.
Colleagues of the authors had pointed out flaws in the methodology and results months before the article was submitted to the journal. The publication triggered criticism from major research groups, which Nature subsequently published.
Eventually the original paper was condemned by Nature‘s editor-in-chief: “Nature has concluded that the evidence available is not sufficient to justify the publication of the original paper.”
Since then, no other research group has been able to confirm the original results.
Environmental Benefits Ignored
In 2000, the American journal Science published an article in which the authors claimed to have evaluated the “ecological risks and benefits of genetically engineered plants.” En route to concluding they could draw no conclusion, the authors neglected the proven benefits of gene-spliced organisms, including enhanced yields, nutritional enhancements, less use of chemical pesticides, and greater use of no-till farming (and thus less soil erosion and runoff of chemicals and lower release of carbon dioxide to the environment).
At the same time, no detrimental effects of gene-spliced plants had been described–nor have any been identified since.
Politically Charged Discussion
Another egregious and exceedingly harmful example of apparent anti-biotechnology bias appeared in an article in the British medical journal The Lancet. It claimed to show adding to potatoes a gene that codes for a substance toxic to insects caused damage to the immune system of laboratory rats and stimulated abnormal cell division in their digestive tract.
But many research groups have shown the experiment’s research methodology was fundamentally flawed and no conclusions about the safety of biotech foods can be drawn from the research results. After an extensive review, the British Royal Society issued a statement detailing the ways in which the experiments were fatally flawed, concluding, “On the basis of this paper, it is wrong to conclude that there are human health concerns with the process of [gene-splicing] itself, or even with the particular genes inserted into these [gene-spliced] potatoes.”
The editors of the journal nonetheless remonstrated against their critics, arguing that in spite of the article’s admittedly deficient methodology–and over the strenuous objections of the paper’s referees–they published it to “make constructive progress in the debate between scientists, the media, and the general public” about a very politically charged issue.
Unleashing such a sham has proved to be anything but constructive, because the paper’s publication is frequently cited as presumptive validation of its spurious conclusions. The irresponsible rationalization of the editors makes a mockery of the peer review process.
Real World Ignored
The most recent example of the failure of editorial and peer review in scientific journals occurred in an article published in September 2007 in the Proceedings of the National Academy of Sciences (PNAS). The authors claimed to show pollen from gene-spliced corn harmed certain insects in a laboratory aquatic ecosystem.
Their conclusions are dubious, and their methodology was atrociously sloppy and inadequately described. More important, the researchers reported elsewhere that they had failed to find these same effects in studies in the field–which they neglected to reveal in the PNAS article.
This is a critical omission because laboratory studies are designed to mimic what happens in the real world. In other words, even if the laboratory studies had been performed correctly and carefully, positive results arguably would have been irrelevant because they don’t mimic what happens in the field.
Process Must Be Corrected
These and other failures of peer review corrupt the traditional process by which new scientific knowledge is obtained and reported. They inflict irreparable harm on the reporting and archiving of scientific developments for policymakers, the media, the public, and the scientific community.
Within weeks of the publication of the flawed PNAS article, for example, European Union environmental regulators cited it as justification for a ban on the sale of gene-spliced corn seeds.
Because science is (or is supposed to be) self-correcting–a thesis is put forth, tested, and ultimately revised on the basis of new data–misinformation conveyed to the scientific community distorts the entire process.
Journals should request reviews of research articles from bona fide experts who do not have a known bias toward the subject of the research. These reviewers should be encouraged to ask probing, detailed questions, and the authors of the submitted article should be required to answer them satisfactorily before a paper is accepted.
Had such measures been taken regarding the research articles described here, it is unlikely any of them would have been published in a prominent journal.
When editors lack the integrity and competence to undertake these measures, the scientific community should take them to the woodshed. If they want to retain society’s respect, they will.
Dr. Henry I. Miller ([email protected]) is a fellow at Stanford University’s Hoover Institution. He has held numerous posts at the National Institutes of Health and the Food and Drug Administration.
