There’s a pretty good article in a recent issue of Personnel Psychology by Christopher Berry, Paul Sackett, and Shelly Weismann called “A Review of Recent Developments in Integrity Test Research.” It’s a readable, relatively non technical read that you may want to check out if you use or think you may use integrity tests. Or if you want the short and skinny of it, there’s currently a write-up on the front page of SIOP.org. But I can’t figure out how to direct link to it so it may be gone by the time you read this.
You know integrity tests. They’re those sneaky paper and pencil tests that have questions like this:
Q1 – Have you ever stolen anything?
a. What? No way, man.
c. Nope, never, nope.
d. Why, I never! Look, you’ve made me drop my monacle!
Q2 – No, seriously dude, have you ever stolen anything?
a. Oh, sure. All the time.
b. Yeah. I’m stealing this pencil right now!
c. No. Never. But tell me: Where do you keep the petty cash?
d. Well, just some office supplies. And millions in pension benefits.
Well, maybe not quite like that unless we’re talking about the world’s worst overt integrity test. The Berry, Sackett, and Weismann article gives you a much clearer picture of where things stand with integrity testing. Perhaps the section most fascinating to me was the one dealing with how item-level analysis of integrity tests revealed that they may be tapping something outside of the Big 5 personality taxonomy. Because anything that stretches us beyond that holy pentagram of personality research is potentially a good thing.
And integrity tests do appear to work, despite all the grumbling about them and people who say that folks will just lie when completing them. I’ve talked to a few people about this kind of thing, and beyond the typical biases surrounding personality testing there just seems to be this miasma surrounding integrity testing in particular. Maybe they just don’t want applicants to think they’re being treated like criminals, or maybe people just refuse to believe the test validities aren’t affected much by faking (which they typically aren’t.). Who knows. But it’s very practical article that should probably be kept on hand for when it’s needed.
Here, I want to share two quotes with you, neither of which make much sense to me. First, from the newsletter of a major management consulting firm:
Enterprise risk management is the proactive execution of a senior-management sponsored, entity-wide assessment and response to the collective risks that impact an organization’s ability to maximize stake holder value. While many acknowledge that developing a leadership pool is critical for business, few see the lack of leadership or lack of succession planning as a business risk.
What? Huh? I can barely make it past the first sentence the communication barrier is so high. It’s like it’s being spoken by some guy on the Moon who’s speaking French for some reason. And I don’t speak French. I only speak a little Spanish, so my standing there and shouting “¿¡Adonde esta la biblioteca!?” over and over again isn’t going to get us anywhere. Seriously, I have no idea what it is that the author is talking about there. It’s business speak gone out of control.
But wait, let’s look at something from the scientist part of the scientist-practitioner spectrum. Here’s an excerpt from something published in the latest Journal of Applied Psychology:
Table 3 shows the correlations and descriptive statistics for the study variables. There were positive correlations between (a) supervisor-rated task performance and (b) extroversion (r = .15, p < .05) and positive moods (r = .16, p < .05). A preliminary step for HLM began with fitting a null model to estimate the total systematic variance in the dependent variable; this resulted in an intraclass correlation coefficient (ICCI) of .21.
Okay, so I happen to speak whatever language this is written in, but I imagine a lot of people who are interested in that study’s general topic (the effects of being happy on job performance) don’t. Or they only speak a little, so they could only haltingly ask “Please which way to the alligator station please?” in response. For most people, the message is equally ineffable.
Before I go on, I want to emphasize that I’m not singling any of these particular authors out. They’re obviously very good at what they do. Better than me, certainly. And yes, the quotes are devoid of context which makes them seem more dense than they really are, but I got a point to make. And that point is that addressing the practitioner/scientist divide in I-O Psychology is a hot topic right now, and a lot of people are talking about how to make academic journals more accessible to lay people by using less esoteric language and communicating the research within a more realistic context. This is a good discussion to have, and I think that there’s a lot of low hanging fruit to be captured if those who publish in the academic press can just unstraddle their preoccupation with certain styles and traditions. My part of the upcoming “Good Science Good Practice” column in TIP deals a lot with this.
But what really struck me about that first quote from the management consulting firm is that this argument can be applied to both sides of the spectrum. Granted this is partially marketing speak, but I’ve seen a lot of brochures, websites, and even white papers that use similar language at the expense of clarity and effectiveness. When I read stuff like that I have to either just ignore it or look for the contact information of a person I can actually talk to. Perhaps some of those crying for academics to change their tone should consider changing their own.
The new movie The Golden Compass is set in a world where part of people’s souls (called, perhaps oddly, “demons”) live outside their body in animal form. The animal that one’s demon imitates is supposed to say something about your personality –more subservient people may have dog demons, while scholarly ones may have owls, and Nichole Kidmans have golden monkeys.
To help promote the movie, the official website has set up a quiz to help you determine what kind of demon YOU would have if you lived in this magical world. The site is in awful full flash animation, so I can’t directly link to the quiz but if you go to the site and choose the “demons” tab you’ll get to a page where you can choose “meet your demon.”
The quiz consists of items that look an awful lot like what you might find on personality tests used in pre-employment testing:
At parties you prefer not to talk to people you don’t know.
It doesn’t take much to get you fired up.
You usually get your own way.
And so forth. Test-takers are asked to respond to each statement using a standard Likert-type scale of Strongly Disagree to Strongly Agree.
I took the quiz and my demon is a bunny! And apparently bunnies are “modest, assertive, shy, passive, and responsible.” I’m …not quite sure what to make of that. The quiz is obviously more like a horoscope and less like any kind of scientific test, but I can’t help wondering how it was put together and what kind of model it used. There are many different demons that you can get by taking the test different ways so some thought must have gone into it. I wonder if they consulted any kind of real expert? I’m guessing not, but it’s interesting to speculate.
As a testing professional, I’m of two minds with something like this. On the one hand, it’s fun and kind of cool to see that something like personality testing is getting mainstream enough to be used in this way. On the other hand, it’s fairly disheartening to see it treated like a cheap marketing gimmick, and one that’s associated with magic and parlor tricks. But then again, we bunnies tend to take things like that seriously.