Typically, during grade school we are introduced to the scientific method with the following steps:
Hypothesis
Gather Data
Experiment – test hypothesis
Reject or accept hypothesis
However, there is one extra step that is very important and that is replication of the original research result. Replication of results never gets the headlines, or wins a Nobel, but without it, science cannot advance. This is a vital, if often overlooked, aspect of the scientific process that guards against fraudulent work and/or erroneous findings. One such example of the replication process nullifying an original research result is the vaccination/autism link. In the late 1990’s, a research paper was published indicating a link between vaccinations and autism. Those results could not be replicated independently and were later discovered to be fraudulent.
Another controversial science paper was published around the same time as the fraudulent vaccine/autism study. That being, the hockey stick graph indicating a rapid rise in global surface temperatures during the 20th century. Unlike the vaccine/autism link, the hockey stick result was replicated independently. Also, an investigation of fraud charges against the author, Michael Mann, cleared him of any wrongdoing. Hence, the vaccine/autism link was fraudulent science that is not valid, but the hockey stick graph is a valid result confirmed independently and cleared of fraudulent charges.
In astronomy, the replication process recently nullified a discovery that attracted quite a bit of publicity. In 2014, a team of astronomers announced the detection of gravity waves in the cosmic background radiation left over from the Big Bang. This had huge implications, as this would have confirmed the inflationary model of the Big Bang as well as gravity being transmitted via waves as predicted by Einstein in 1916. However, the replication process determined the signal the team detected in the form of polarized light was actually caused by dust in the Milky Way.
It happens, another case was the exoplanet Gliese 581g. When discovered, it was thought to be a habitable planet and garnered quite a bit of press coverage. However, subsequent observations determined the signal received was caused by hyperactivity of the star itself and not an exoplanet at all. Nor is this restricted to the natural sciences. In economics, there was a notable failure to replicate a research result that had significant policy implications during the height of the worst economic downturn since the Great Depression. As you can tell, a failure to replicate does not necessarily mean the original work was fraudulent, sometimes it is caused by a breakdown in methodology or misunderstanding of cause and effect.
All this can be very confusing to students and can lead to disillusionment with science unless the process is understood properly. We cannot be experts at everything, so how do we know if a scientific report is trustworthy or not? How do we know if that latest nutrition study the press is hyping will pan out in the long run? I tell my students to see if the results have been verified independently. For example, the link between lung cancer and smoking has been replicated by many studies and thus, can be trusted to be quality info. That is the hallmark of good science.
The lesson here is, never jump on an initial finding (no matter how interesting) as a conclusive result. Best to wait for replication of the original study as confirmation of those results. Failure to do that with the autism/vaccine link has caused a significant increase in measles cases the past year. And as we have discovered, once a concept gets lodged in a mindset, it can be very difficult to dislodge it.
So, another bit of advice on how to do good science. If we develop a “rooting” interest in a scientific result as we do at a sporting event or in politics, we have already gone off the rails are far as the scientific method goes. Nature will not bend to our wishes. We can only employ the scientific method to understand it better.
*Image at top of post is replication of Mann’s hockey stick graph courtesy NOAA.