This reminds me of the story behind the "discovery" of cold fusion in the 80s. Sure, Pons & Fleischman may have thought they measured something, but in the end the whole effect could be attributed to careless experimental design and procedure.
Cold fusion is a bit more complicated than that. The early experiments were certainly flawed, and there were way to many people announcing results for publicity before they had checked their work.
After a few years, it was no longer socially acceptable for respected scientists to work on cold fusion. It didn't matter anymore how good your reputation was as a researcher, or how carefully designed your experiment was--if your research was in cold fusion, it would get largely ignored.
Unfortunately, this happened around the time a couple groups of respected researchers with good experiment design were getting interesting results pointing to real new phenomena, and perhaps explaining why prior results had been so erratic.
David Goodstein around this time wrote a great article [1] looking back at cold fusion, which included a discussion of these later results, and how socially science had reached a state where they could not be considered. Here are the last few paragraphs.
All of this was much less important than the fact
that Cold Fusion experiments, if they gave positive
results at all, gave them only sporadically and
unpredictably. When Bednorz and Mueller announced
the discovery of high-temperature superconductivity
in 1986, no one carped about control experiments,
because, once the recipe was known, any competent
scientist could make a sample and test it and it
would work immediately. If, at their press
conference, Pons and Fleischmann had given a
dependable recipe for producing excess heat, they
very likely would be Nobel Prizewinners now (as
Bednorz and Mueller are) rather than social outcasts
from the community of scientists. The essential key
to the return of Cold Fusion to scientific
respectability is to find the missing ingredient
that would make the recipe work every time.
Experiments done in the U.S. and in Japan, and
reported at the Maui meeting indicate that the
missing ingredient may have been found. In all the
various Cold Fusion experiments, the first step is
to load deuterium into the body of metallic
palladium. The issue is how much deuterium gets into
the metal. The ratio of the number of atoms of
deuterium in the metal to the number of atoms of
palladium is called x. It turns out, by means of
electrolysis, or by putting the metal in deuterium
gas, that it is rather easy to get x up to the range
of about 0.6 or 0.7. That is already a startlingly
high figure. If there are almost as many deuterium
atoms as palladium atoms in the material, the
density of deuterium (a form of hydrogen) is
essentially equal to that of liquid hydrogen rocket
fuel, which can ordinarily exist only at extreme low
temperatures. In other words, palladium (and certain
other metals including titanium) soak up almost
unbelievable amounts of hydrogen or deuterium if
given the chance. This is far from a new discovery.
However, according to the experiments reported at
Maui, x=0.6 or 0.7 is not enough to produce Cold
Fusion. Both the American and Japanese groups showed
data indicating there is a sharp threshold at
x=0.85. Below that value (which can only be reached
with great difficulty and under favorable
circumstances) excess heat is never observed. But,
once x gets above that value, excess heat is
essentially always observed, according to the
reports presented at Maui, and recounted by Franco
Scaramuzzi in his seminar at the University of Rome.
The audience at Rome, certainly the senior
professors who were present, listened politely, but
they did not hear what Franco was saying (that much
became clear from the questions that were asked at
the end of the seminar, and comments that were made
afterward). If they went away with any lasting
impression at all, it was just the sad realization
that a fine scientist like Franco had not yet given
up his obsession with Cold Fusion. They cannot be
blamed. Any other audience of mainstream scientists
would have reacted exactly the same way. If Cold
Fusion ever gains back the scientific respectability
that was squandered in March and April of 1989, it
will be the result of a long, difficult battle that
has barely begun.
Recently, I told this story in a Philosophy course
we teach at Caltech called "Ethics of Research." The
first question, when I finished my tale, was, do I
believe in Cold Fusion? The answer is, no.
Certainly, I believe quite firmly the theoretical
arguments that say Cold Fusion is impossible. On the
other hand, however, I believe equally firmly in the
integrity and competence of Franco Scaramuzzi and
his group of co-workers at Frascati. I was disturbed
when I saw that Franco had gotten caught in the web
of science-by-news conference in April 1989
(although I was truly pleased that he finally got
the long overdue recognition his agency ENEA owed
him), and I was even more distressed when I learned
that Franco and his group had observed excess heat
(the "bad kind" of Cold Fusion). However, I have
looked at their cells, and looked at their data, and
it's all pretty impressive. The Japanese experiment
showing that heat nearly always results when x is
greater than 0.85 looks even more impressive on
paper. It seems a particularly elegant, well
designed experiment, at least to the untutored eye
of a physicist (what do I know about
electrochemistry?) What all these experiments really
need is critical examination by accomplished rivals
intent on proving them wrong. That is part of the
normal functioning of science. Unfortunately, in
this area, science is not functioning normally.
There is nobody out there listening.
I suppose that, if nuclear fusion really does take
place whenever x is greater than 0.85 in palladium,
the world of conventional science will eventually be
forced to take notice. If not, then the whole story
I have told you is nothing but a curious footnote to
a bizarre and ugly episode in the history of
science. Either way, I think the story illuminates
the inner dynamics of the scientific enterprise in a
way that few other stories have done. For that
reason alone, it may be worth telling.
That was taken very seriously at the time. I went to a well-attended talk at Stanford by a Stanford physicist who was trying to replicate the Pons/Fleischman experiment. When they first set up the apparatus, they had radiation detectors with alarms, in case it started generating a dangerous level of neutrons. After some initial tests where not much happened, they moved the radiation alarm gear back a bit.
Total measured neutron output, best case, appeared to be twice background. That's small enough that people moving around (people are mostly water) was enough to cause such variation. So they moved the experiment to a "neutron cube", a big box built of lead bricks where few background neutrons penetrated. Then there was no significant difference in neutron levels between "off" and "on".
They also tried heat generation measurement. Because you have to put power into the Pons/Fleischman apparatus to get anything to happen (it's not self-sustaining) it's a difficult experiment to do accurately. It requires water jackets, insulation, and lots of measurements. They were unable to detect any heat output above measurement noise.
See this paper: http://pus.sagepub.com/content/5/2/121.short (unfortunately behind a paywall)