Is the “Interviewer Effect” Skewing Your Survey Results?

  • Determining if social programs and projects are working as intended
  • Setting policy and making decisions
  • Influencing human lives

So What Can We Do?

  • Check who was asking the questions. What are the enumerators’ demographics? Were they women or men? How does their income related to respondents’ income? Are there differences in ethnicity, race, social status, or education? (A recent EU survey on violence against women used only female interviewers, to eliminate that variable. This, however, is not common.
  • Check how interviewers were assigned to administer surveys. Were they randomly assigned to survey respondents? The vast majority of the time, this is not the case.
  • Adjust for the interviewer effect in your analysis. If interviewers were randomly assigned to respondents, this is unnecessary. If not, the next best situation is one in which enumerators were balanced by characteristics between treatment and control groups. If the data has already been collected and isn’t balanced, you’ll need to use a statistical method to estimate the effects of the interviewer on your questions and adjust for that in models. (You can test for effects by dividing data into subsections and using a machine learning algorithm. You can also use matching techniques like propensity score matching. Or you can build mixed-effects models with an interaction effect for the interviewer characteristics.)

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Heather Krause

Heather Krause

Data scientist & statistician (one of only 150 accredited PStats worldwide). Providing data science services grounded in an equity lens. https://weallcount.com