Old, but good article.
Old, but good article.
Historians of science are always quick to point out that Ronald Fisher, the UK statistician who invented the p-value, never intended it to be the final word on scientific evidence. To Fisher, “statistical significance” meant the hypothesis is worthy of a follow-up investigation.
https://www.vox.com/science-and-health/2017/7/31/16021654/p-values-statistical-significance-redefine-0005
https://www.vox.com/science-and-health/2017/7/31/16021654/p-values-statistical-significance-redefine-0005
Historians of science are always quick to point out that Ronald Fisher, the UK statistician who invented the p-value, never intended it to be the final word on scientific evidence. To Fisher, “statistical significance” meant the hypothesis is worthy of a follow-up investigation.
https://www.vox.com/science-and-health/2017/7/31/16021654/p-values-statistical-significance-redefine-0005
https://www.vox.com/science-and-health/2017/7/31/16021654/p-values-statistical-significance-redefine-0005
Nobody Damn few people understand what a p-value really is.
ReplyDeleteI'm going to retreat from being out on the limb and claim that I probably don't understand it as well as I should (although definitely better than some who should also know.)
Some of the best teaching on statistics in research I ever had made a strong point about Fisher and his intention in developing the p-value. Much better than people going on and on about null hypotheses and theoretical constructs that become tortured and convoluted.
ReplyDeleteThe article makes a great point about the mythological gatekeeping power with which the p-value has been imbued and its effect on publishing, not to mention the entire culture.
I do agree about the mythical power of the p-value, especially the magical cut-off of 0.05.
ReplyDeleteI mean, yes, there has to be a line somewhere because without it we'd have drifted back to 0.1 or 0.2 in computer science and 0.75 in social science. But some people can't even have a rational conversation about it without it becoming a dogma war.
To build off of John's comment, the p-value cutoff does have important mythical power - among non-scientists who matter for the real-world exercise - and funding - of science. Journalists, politicians, non-scientist foundation officials, non-scientist wealthy philathropists, non-scientist administrators, etc.
ReplyDeleteActual scientists - in all their forms, including engineers and physicians - already know the difference between a result that has a p=0.049 and a p < 0.001. When we evaluate presented data, we know which results we should take with a major grain of salt and which results we can stake more heavily on. But...
... a result you can get under "the line" - wherever that line is - is one you can have an excuse to put out a press release on. One you can spin to a non-scientist philanthropic funding agency. One you can spin to a wealthy benefactor who might not get the difference. Moving the mythical line from p=0.05 to (as suggested) p=0.005 suddenly takes that maneuvering room away. And that's a very big part of why the mythical line matters.
Now, whether that's a net good thing or a bad one is a whole different question, with legitimate arguments on both sides.
ObXKCD: xkcd.com - xkcd: Significant
ReplyDeleteP-hacking getting so common it’s now a known concept is a damn shame.
I have to say my experience has been that a surprising number of people do not know, they really don't. And I watch people getting taught to look for p-values as if it were something objectively pure all the time. If you get the magic number, they stop thinking. Aside from the real world issues of funding and news releases, which are driven by that kind of teaching which perpetuates the p. Change the culture of the p and you will change the message that the money responds to. Well, that part of it, anyway.
ReplyDelete