Identity Sorting Dials: Certainty

by Heather Krause | Analysis, Communication & Distribution, Data Collection & Sourcing |

The Identity Sorting Dials are a really useful tool to start (again, I’ll say start) thinking about the interplay between human considerations like fit (“can I see myself in this category”) and ease (“how difficult is it for me to interact with this data process”) and technical priorities like certainty (what kind of confidence interval do our categories lead to) and specificness (what is the resolution of the data we need).

The certainty dial refers to the statistical certainty offered by the sample size of each of our groupings. Sample size isn’t the only thing that affects our point estimates and confidence intervals (important measures of how certain we are that a statistical estimate reflects reality) but it is a major factor and is largely determined by the groups we choose to put people’s data into. Really small sub-groups tend to lead to really low levels of certainty.

Let’s say we want to collect data about a person’s disability status because we want to examine gaps in how our services work for people across an ability spectrum. There are many different recommendations from excellent organizations about how to do this and what categories to use. The choice of categories here is going to have a strong impact on how much certainty we are…