Internal Stakeholders’ Reactions to Testing


Okay, if there are any grad students out there looking for a thesis/dissertation idea, I’m going to give you a freebie here. In the recent issue of SIOP’s new journal, Organizational Psychology: Perspectives and Practice Scott Highhouse has a nifty little article entitled “Stubborn Reliance on Intuition and Subjectivity in Employee Selection.” This article could easily also been titled “What? No! Why Are the Hiring Managers Doing That? Make Them Stop!” because it basically looks at what he sees as two of the root causes for organizational decision-makers to reject or circumvent scientifically derived selection systems like employment testing.

First is the belief that it should be possible to explain 100% (or close to it) of the variance in human behavior within an organizational context. Someone holding this belief may scoff at your puny validities — 11% of variance explained? Pshaw! Humans are just squishy machines, right? We should be able to predict their performance perfectly. Your expectancy tables and realistic discussions of false positives are powerless in the face of this belief.

The second common reason for objecting to selection systems is the belief that experience makes people better at figuring someone out. This comes through intuition, hunches, reading between the lines, and other nebulous decision-making. Your test results may not mean much if the interviewers like the cut of the candidate’s gib. Just the fact that the guy actually brought in a gib to show them how he had cut it won them over.

Anyway, Highhouse’s article discusses the origins of these troublesome beliefs, and several of the follow-up articles in the same article discuss how to combat them. This, though, made me realize that there’s a whole nascent line of research that’s just waiting to be expanded: stakeholder reactions to selection systems.

Think about it. There’s a great and thriving body of research on applicant reactions to testing, drug screens, and other selection systems. I should know –it was the topic of my Ph.D. dissertation. We know how to study this kind of thing, so why hasn’t anyone turned their attention to building and testing theories about the reactions of other stakeholders, like hiring managers and other decision makers?

We could do it. Heck, you could use the applicant reactions literature as a template to get started. Do hiring managers dislike aptitude tests because of their lack of face validity or because they rob them of control over the decision making process? What kinds of biases and kinks of the human mind come into play when trying to understand probability and utility in a selectoin testing context? Are hiring managers more likely to support testing if they get to interview candidates before or after testing? If we knew more about what kinds of test characteristics drive what kinds of reactions among internal stakeholders, we testing professionals would be better equipped to address, assuage, and prevent those concerns without sacrificing the validity and utility of our tools.

So, there you go. Somebody get on that. If I can find time, I certainly will.

A Primer on Testing Issues


I got a piece of advertising-slash-content in my inbox the other day that actually turned out to be noteworthy. And so I shall make note of it. A company called Peopleclick (a name we can only assume they settled on after rejecting “Peopledrag” and “Peoplerightclick”) put together a white paper entitled “Questions the Government is Asking about your Employment Tests: Do you have the Answers?” You can get it by performing clicking motions here. They’ll ask you to input contact information, but if you’re so disposed you don’t have to make it accurate.

This isn’t going to score you any continuing education credits, but the white paper actually does make for a decent primer on adverse impact, validity, and federal requirements on documentation of all the above. It’s something that you might give to hiring managers who want a little more information about the regulations and legal landmines around testing. Or as a take-away from a meeting on the same kind of topic. I often have meetings with clients who say “I want to do testing!” and while I’m always (well, almost always) happy to hear this sentiment, sometimes a little eduction and setting the scene is necessary. This paper tells them just enough to let them know that they need the help of an expert in things like this.

My only slight complaint is that there is relatively little consideration given to more cutting edge validation techniques, such as job component validity, validity generalization, and validity transportation. This is probably because not only do those techniques quickly bog down in jargon, statistics, and other detailed considerations, but also documents like the Uniform Guideslines and even SIOP’s Standards are a bit out of date in those areas, not to mention case law. Still, that’s a whole different debate.

Book Review: 150 Things You Need to Know

100 Things

100 Things

100 Things Your Need to Know: Best People Practices for Managers & HR and 50 More Things You Need to Know: The Science Behind Best People Practices for Manager & HR Professionals (whew!), are curious and different from most books that I’ve seen on similar topics. As you might guess from the titles, they contain 150 chapters between them, covering sup-toics like selection, Human Resources law, leadership, HR metrics, corporate culture, training, recruiting, HR technology systems, compensation, benefits, motivation, organizational development, job design, teams, performance management, surveys, and more.

Each of these 150 chapters is dedicated to a single “fact,” which is framed as a multiple-choice question at the beginning on the opening page. Do applicants have preferences among various selection techniques? Is there still a bias against African Americans in the workplace? Do people differ in how they learn from experience? How many points should your survey question response scales have on them? How skilled are managers, typically, at being good coaches?

You’re supposed to try and answer the question without peeking, and there’s even places to keep track of your answers so that you can get “scores” for the books that reflects your knowledge of these 100 and 50 things. (Me, I always just peeked.)

After the opening question in each chapter, the correct answer is given, along with a 1-5 ranking of how solid the current state of the research is on this answer, from suggestive to absolutely sure. Then there’s a discussion of the factoid, then citations of research that back up the claim, then a discussion of what it means to HR practitioners, then finally a bibliography for further research. This all happens in the space of 3-5 pages each, so it’s nice and easy to digest. I would typically read a chapter or two over lunch at work or when I needed to take a little break but still wanted to feel like I was doing something work related.

What I like about these books is that they are very research oriented, with each of the 150 assertions backed up by scientific research, usually taken from refereed journals in various branches of psychology and management. It’s not, in short, arm chair punditry or bland platitudes. And while I found myself disagreeing with their reading of the current literature on some topics –such as the importance of emotional intelligence for job performance or the nature of employee engagement as a construct distinct from others– they were mostly spot on from what I could tell.

My only substantial complaint about the books are that I wish the authors had organized all the similar chapters together. I would have liked, for example, to have read through a chunk of chapters all dealing with leadership development all at once, rather than having those same chapters sprinkled randomly throughout the book. Still, with the use of a good index and some skimming, these are going to make pretty good reference books, especially with the bibliographies in each section serving as jumping off points for more in-depth reading.

Do you like these reviews? Check out my profile on

Since I don’t have a Twitter feed…

Say you’ve got a nice little Master’s thesis on the importance of handshakes in job interview evaluations and you want to “dress it up” for submission to a refereed journal. Step 1: replace all instances of “handshake” with the phrase “tactile nonverbal communication.”