Questions Americans Desire Answered: Should The 5% Convention For Statistical Significance Last Dramatically Lower?

Can you lot ask heed the clamor?
From The Conversable Economist:

For the uninitiated, the thought of "statistical significance" may look drier than desert sand. But it's how enquiry inward the social sciences in addition to medicine decides what findings are worth paying attending to as plausible true--or not. For that reason, it matters quite a bit. Here, I'll sketch a quick overview for beginners of what statistical significance means, in addition to why at that topographic point is argument amid statisticians in addition to researchers over what enquiry results should live on regarded as meaningful or new.

To attain to a greater extent than or less intuition , reckon an experiment to determine whether a money is as balanced, or whether it is weighted toward coming upwardly "heads." You toss the money once, in addition to it comes upwardly heads. Does this trial prove, inward a statistical sense, that the money is unfair? Obviously not. Even a  fair money volition come upwardly up heads one-half the time, afterwards all. 

You toss the money again, in addition to it comes upwardly "heads" again. Do 2 heads inward a row test that the money is unfair? Not really. After all, if you lot toss a fair money twice inward a row, at that topographic point are iv possibilities: HH, HT, TH, TT. Thus, 2 heads volition plow over off one-fourth of the fourth dimension amongst a fair coin, merely yesteryear chance.

What well-nigh iii heads inward a row? Or iv or 5 or half-dozen or more? You tin never completely dominion out the possibility that a string of heads, fifty-fifty a long string of heads, could plow over off alone yesteryear chance. But as you lot learn to a greater extent than in addition to to a greater extent than heads inward a row, a finding that is all heads, or generally heads, becomes increasingly unlikely. At to a greater extent than or less point, it becomes real unlikely indeed.  

Thus, a researcher must brand a decision. At what betoken are the results sufficiently unlikely to convey happened yesteryear chance, thus that nosotros tin declare that the results are meaningful?  The conventional answer is that if the observed trial had a 5% probability or less of happening yesteryear chance, thus it is judged to live on "statistically significant." Of course, real-world questions of whether a sure as shooting intervention inward a schoolhouse volition heighten examination scores, or whether a sure as shooting drug volition aid process a medical condition, are a lot to a greater extent than complicated to analyze than money flips. Thus, thus practical researchers pass a lot of fourth dimension trying to figure out whether a given trial is "statistically significant" or not.

Several questions arise here.

1) Why 5%? Why non 10%? Or 1%? The brusk answer is "tradition." Influenza A virus subtype H5N1 duad of twelvemonth ago, the American Statistical Association set together a panel to reconsider the 5% standard. The

Ronald L. Wasserstein in addition to Nicole A. Lazar wrote a brusk article :"The ASA's Statement on p-Values: Context, Process, in addition to Purpose," in  The American Statistician  (2016, 70:2, pp. 129-132.) (A p-value is an algebraic way of referring to the measure for statistical significance.) They started amongst this anecdote:

"In Feb 2014, George Cobb, Professor Emeritus of Mathematics and Statistics at Mount Holyoke College, posed these questions to an ASA give-and-take forum:

Q:Why create thus many colleges in addition to grad schools learn p = 0.05?
A: Because that’s all the same what the scientific community in addition to mag editors use.
Q:Why create thus many people all the same purpose p = 0.05?
A: Because that’s what they were taught inward college or grad school.

Cobb’s draw of piece of occupation organisation was a long-worrisome circularity inward the sociology of scientific discipline based on the purpose of brilliant lines such as p<0 .05:="" because="" blockquote="" do="" e="" it="" nbsp="" s="" teach.="" teach="" we="" what=""> But that said, there's nil magic well-nigh the 5% threshold. It's fairly mutual for academic papers to study the results that are statistically signification using a threshold of 10%, or 1%. Confidence inward a statistical trial isn't a binary, yes-or-no situation, but rather a continuum....MORE