Originally posted by DeepThought
I've been trying to think of a way to respond to your excellent post. One thing I've seen in medical research papers are the words "marginal significance". They've done a trial and the drug's failed (one even sees the words "the patient failed the treatment" on occasion) but it's right on the borderline so they call it "marginal significance" when it s ...[text shortened]... while ago which is why I've seen these things. It put me off going to the doctors for life! 😉
Yes indeed, and I can do you one better; I've seen "near-marginal significance." Explain that one to me. To be fair, those studies are usually a case of "well, we have to publish it for documentation, disclosure and funding purposes, but the drug probably won't make it to market so it's back to the drawing board in the lab."
I have felt the escalating need recently to defend science from the "perpetual skeptics." (i.e. pseudoscientists) Skepticism is an important skill, however its purpose is to fill in logical gaps/flaws in reasoning, to refine the conclusion. A perpetual skeptic, regardless of what the science demonstrates, will always have a reason to reject the conclusions. They don't take the time to comprehensively review the existing literature before they make definitive statements; rather, they cherry pick certain information and ignore other, solid lines of evidence. They don't think about the experiments. They exploit uncertainty to say that the science overall isn't settled.
Blame it on education systems teaching bad critical thinking skills and the internet where anyone can have a soundboard. Because scientists deal with probabilities and not truths, the lack of any statistical correlation between vaccinations and autism can't definitively rule out arguments that "vaccines might cause some harm". Unfortunately that leaves enough of an open window for people who want to believe that vaccines cause autism.
I think the "reproducibility crisis" fits into this. While it is a legitimate problem, it has been understood for a long time. A very small minority of the unreproducible studies are fraudulent, but that seems to be the implication from media reports. Usually its as simple as an overlooked variable or a bad reagent or a misplaced decimal. But it leaves open that window for the perpetual skeptic to point their finger at the "shameful, biased, selfish scientists".
Again, science is hard. It takes a long time. We just published a story that took 3.5 years to collect data. Repeating all those experiments would be insanity; however, we are following up on it, validating the results in other model systems and hopefully other labs are doing the same. If 5 years goes by with no follow up publications, it was a dead end.