The many ways to calculate adverse impactPosted: December 1, 2005 | Author: Jamie Madigan | Filed under: Uncategorized | 8 Comments »
Yesterday I attended a pretty good workshop put on by the Personnel Testing Council of Southern California in which Dennis Doverspike talked about assessing adverse impact –when a test or other hiring system discriminates against one group more than another. (He also spoke on hiring based on a public service work ethic, which I’ll probably write about next week).
Adverse impact analyses had always been pretty straight forward to me. I was certainly aware that other methods existed, but I had always used the “Four-Fifths or 80% Rule” to determine the presence of a hiring system’s adverse impact against minorities or women. Quoth the Uniform Guidelines on Employee Selection Procedures:
A selection rate for any race, sex, or ethnic group which is less than four-fifths (4/5) (or eighty percent) of the rate for the group with the highest rate will generally be regarded by the Federal enforcement agencies as evidence of adverse impact, while a greater than four-fifths rate will generally not be regarded by the Federal enforcement agencies as evidence of adverse impact.
So here’s an example:
In this example 64 males took a test and 16 passed while 17 women took the test and 3 passed. So the passing rates were 20% for males and 15% for females. Is the 5% difference enough to signal adverse impact?
The answer is yes: 15 / 20 = 75% or three quarters. The Four-Fifths rule says that if it’s less than 80% (i.e., four-fifths) then you’ve got evidence of adverse impact. Pretty cut and dry, right?
Well, as the PTC-SC workshop point out, no. There’s also language in the Uniform Guidelines that allows for most rigorous statistical tests like Chi Square or Fisher’s Exact Test, and there’s a history of court cases that use other quasi-statistical rules of thumb, like saying that pass rate for the protected group must be within 1.97 standard deviations of the dominant group’s passing rate. And the thing is that depending on the distribution of your data, one method may yield a red flag while another may not. There are also different assumptions about what’s the population of interest –is it all the people who applied for the job or is it all the people in your labor market who could have applied. And don’t even get me started about setting different levels of alpha (i.e., accepting a 5% or 10% or 1% chance of saying there’s a difference between the groups when there’s not). Seriously, don’t. We’ll be here all day.
Dr. Doverspike’s presentation provided a long list of helpful formulas and procedures, but the thread that ran through them all: There’s more than one way to skin a cat and then not hire it based on discriminatory hiring practices against skinless cats. In other words, the Four-Fifths rule isn’t the final word and whether your hiring procedure has adverse impact may depend as much on your data as your lawyer.
In the end, though, it’s almost all a moot point. My own rule of thumb would be this: Unless you’re actively trying to increase the diversity of your workforce, assume you have adverse impact and move on to looking at validity and utility. If you use your favorite method and find out that you don’t have adverse impact, assume that some other lawyer or expert witness could come along and uncover some just by slicing your data differently or making a couple of assumptions differently. If you want to maximize the usefulness of your test, you should be more worried about whether or not it’s valid and what kind of utility you’re getting out of it.