Don’t Let the P in P Value Stand for Privilege

by Heather Krause | Jul 29, 2022 | Analysis

If we’re going to use P values as our measure of what’s a meaningful difference versus a difference due to random chance, we need to recognize that P values are highly dependent on sample size.

This dependency leads to a situation where we call a problem experienced by a large group “real” while we dismiss the very same problem experienced by smaller groups as “chance”.

Let’s say that our problem is systematic underpayment of women employees compared to men.

Is the company systematically paying women less than their male counterparts, or did it just happen to come out like that in the swirling semi-random soup of the human resources process (incoming salaries, education, experience, hiring practices, interviews, raise requests, job performance, etc.)?

In this jurisdiction, not only is gender a protected class for pay equity, race is too, so when we run the numbers, let’s break it out by race as well:

The difference in pay between employees who are men and employees who are white women is $900 per year with a p-value of .02. That’s the kind of p-value that your average stats professor salivates over. Oh boy, that’s not random chance at all! This is strong evidence of a systematic underpayment…