Learning to Love Uncertainty (Why We Need to Stop Using “Not Statistically Significant)

I want to talk to you about why you should stop saying “not statistically significant” based on sample size alone.

The term “not statistically significant” should only be applied to a hypothesis, not a sample size, and even then it’s an arbitrary line we’ve drawn to lump results into a false dichotomy of “certain” or “uncertain”, instead of talking about what level of uncertainty we are actually dealing with.

“Statistical significance” relates to how strongly the data you have can disprove your hypothesis.

Let’s say we have a survey about how much people like the finale episode of the Sopranos on a scale from 1–10. One of the hypotheses we’re trying to disprove is that men liked the episode more than women. We sample some people out of the…