Can We Trust Published Scientific Data?

Can we trust the data published in scientific journals?

The image of scientific objectivity and rationality suffered a notable setback in November, 2009. In what became known as the Climate-gate scandal, hackers released e-mails from the Climate Research Unit at England’s University of East Anglia.
The leaked e-mails reveal scientists from various academic institutions suppressing dissent from other scientists who have doubts regarding global warming, massaging research data to fit preconceived ideas about human-induced global warming, and manipulating the peer review process to keep skeptical views from being heard. Motivating factors seem to be politics and finance--the prospect of huge grants that might evaporate were human-induced global warming shown to be non-threatening.

     Stephen Meyer, in a recent article “Climategate Recalls Attacks on Darwin Doubters” cites similar cases where promoters of Darwinism use their influence to prevent the publication of articles critical of Darwinism.What gets published may thus not tell the whole story.

This naturally raises the question: How wide-spread is bias and fraud in science?

      David Shatz, in his 2004 book Peer Review: A Critical Inquiry concludes that reviewers are “biased toward papers that affirm their prior convictions…are biased against innovation and/or are poor judges of quality.” Reviewers also seem biased in favor of authors from prestigious institutions. Shatz describes a study in which “papers that had been published in journals by authors from prestigious institutions were retyped and resubmitted with a non-prestigious affiliation indicated for the author. Not only did referees mostly fail to recognize these previously published papers in their field, they recommended rejection.” In 2003 the British Royal Society studied the effects of peer review. The chairman of the investigating committee reported that peer review has been criticized for being used by the scientific establishment “to prevent unorthodox ideas, methods, and views, regardless of their merit, from being made public.”

          Epidemiologist John Ioannidis, in a paper (2005) entitled Why most published research findings are false finds that a randomly chosen scientific paper has less than a 50% chance of being true. Small sample sizes, poor study design, researcher bias, and selective reporting and other problems combine to make most research findings false. But even large, well-designed studies are not always right. Many papers may be accurate measures only of the prevailing bias among scientists.

        Bias is difficult to avoid. It may be quite unintentional. Consider the case of astronomer Walter Adams. In 1925 he tested Einstein's theory of relativity by measuring the red shift of the binary companion of Sirius, brightest star in the sky. Einstein's theory predicted a red shift of six parts in a hundred thousand; Adams found just such an effect. A triumph for relativity. However, in 1971, with updated estimates of the mass and radius of Sirius, it was found that the predicted red shift should have been much larger--28 parts in a hundred thousand. Later observations of the red shift did indeed measure this amount, showing that Adams' observations were flawed. He "saw" what he had expected to see.

      In short, our theoretical expectations can influence what we see. We tend to give undue weight to those observations that agree with our expectations and ignore or discard those that don't. Observational confirmation may sometimes be little more than wishful thinking.

       Sometimes, however, the bias may be more intentional. Even such scientific giants as Johannes Kepler, Galileo, Isaac Newton, Mendel, Pasteur, and Freud have been accused of deliberately fudging data or fabricating experiments to better support their theories (see Horace Freeland Judson (2003) The Great Betrayal: Fraud in Science).

       Consider this notorious example. A graph published by B.D. Santer et al (Nature 382 [4 July 1996]:39-46) seemed to show compelling evidence for global warming:

This looks quite convincing. The observed data were almost precisely what had been predicted by mathematical models. Yet, five months later, P.J. Michaels and P.C. Knappenberger (Nature 384 [12 Dec. 1996]: 522-523) published the following graph, showing exactly the same evidence but in wider context. This tells quite a different story.

In the last decade numerous cases of scientific fraud were discovered. For example, in 2001 Jan Henrik Schön of Bell Labs was found to have falsified data regarding molecular-scale transistors. That same year Victor Ninov ( Lawrence Berkeley National laboratory), who claimed to have discovered elements 111 and 112, was also found to have fabricated his results.

      In many cases, such as these, fraud can be revealed by trying to duplicate the alleged experiments. Often, however, this may be difficult to do. This is particularly the case when it involves very sophisticated and expensive equipment or unique artifacts. One of the areas of science that is most vulnerable is that of origins, particularly human origins.

     The most infamous of all scientific frauds was the Piltdown Man, unearthed in 1912 in a Sussex gravel pit. With its huge human-like braincase and ape-like jaw, Piltdown Man was heralded for 40 years as the missing link between humans and their primate ancestors. But in 1953 it was proven to be a forgery.

     Or consider the case of Shinichi Fujimura (of the Tohoku Paleolithic Institute in Japan) . In 1981 he discovered the oldest stoneware (40,000 years old) found in Japan. He investigated more than 150 prehistoric sites in that country. At his most recent excavation he claimed to find artifacts dating back more than 600,000 years: objects that appeared to show that Paleolithic man at this time was far more sophisticated than heretofore believed.. He rewrote the pre- history of Japan. He was unmasked in 2000, when a newspaper printed pictures of Fujimura digging holes and burying objects that he later dug up and announced as major finds. As a result, fundamental ideas about Japan's history were called into question, textbooks were rewritten, and more than 20 artifacts on display at the Tokyo National Museum removed.

     In 2005 Reiner Protsch von Zieten, distinguished anthropology professor at the university of Frankfurt, an alleged expert in radio-carbon dating, was found to have forged scientific facts for the past 30 years, including the dates of hundreds of ancient human specimens from Europe and America. He was caught when a colleague sent some of the dated fragments to Oxford University for a second opinion and found major discrepancies. Subsequent tests showed the ages of many of his skulls were off by tens of thousands of years. Many published radio-carbon dates thus turned out to be falsified. Consequently, secular anthropology has had to revise its picture of ancient European man between 40,000 and 10,000 BC.
             
     In sum, there is cause for some skepticism regarding the reliability of published scientific data . Data might well be distorted, fabricated or suppressed. Papers critical of the dominant paradigm might well be prevented from being published in mainline scientific journals. This is hardly surprising. After all, scientists are only human—fallen and fallible. They, too, are driven by various extra-scientific motivations, whether ideology, wealth or fame. It is thus important to double-check whether what was reported to have been observed is in fact accurate and complete.

     Of course, when it comes to interpreting the data, we enter into that stage of science were worldview presuppositions play an even more significant role.

*****