|
|
posted: August 19, 2017
tl;dr: People who spent years with a candidate know him/her better than you do based on an interview...
The hiring process in technology is flawed in multiple ways. In my small attempt to help improve it, I am giving away some of my secrets. I’ve already written up The best technical interview question ever. Here’s another secret, on the importance of references.
It’s always made sense to me that reference checks are a potentially valuable way to learn about candidates and make hiring decisions. In an interview you only get to speak to the candidate for an hour or so; why wouldn’t you want to speak to multiple references who have each worked with the candidate for an extended period of time? In effect the reference check is similar to most companies’ favorite way of filling positions: hiring someone who is a known quantity to one or more people already in the company, either via the grapevine or professional networks or employee referral. In the case of the reference check the references do not work inside the hiring company, but as long as they are reasonably truthful in their assessments, they can have valuable opinions as to the candidate is a fit or not.
Years ago I read a scientific study on the hiring process, which measured the correlation factor between various assessments that can be made on a candidate before he/she is hired, and how he/she actually performed on the job after a period of years. Some of the assessment factors were: how well the candidate interviewed; how strong the candidate’s references were; how high the candidate’s SAT scores were; grades in college; etc. Simplistically, each candidate was rated on each assessment factor before they were hired; then the candidates’ overall on-the-job performance was rated years later; then the correlation factors were computed. One would expect that the candidates who interviewed the best would be the best on-the-job performers, right?
In this particular study, the number one most positively correlated factor, however, was not how well the candidate interviewed. It was how strong the references were. In fact, how well candidates interviewed was only slightly positively correlated with on-the-job performance. SAT scores had a similar correlation factor to interviews; in other words, instead of interviewing candidates, you could make similarly good hiring decisions just by asking candidates to submit their SAT scores. This study appeared years ago, and I have since seen other studies claiming that structured interviews are the most valuable assessment tool, or observing the candidates solve problems. So the debate rages on.
Some hiring managers dismiss or de-emphasize references because they assume that every candidate can find three people who will say the candidate walks on water. That’s not my experience at all; I find it pretty easy to get references to open up and provide some valuable insights into the candidate as well as comparisons between the candidate and other people with whom they have worked. Most big companies have policies that are supposed to ban their employees from providing references, because the big company doesn’t want to face a lawsuit over comments made about a departed employee by a current employee. These policies are widely ignored: I have never had a reference tell me he/she couldn’t talk about a candidate due to a company policy.
I’ve done hundreds of reference checks, and I can vouch that reference checks are not all the same. Most references are "good", but there are degrees of "good" that can differentiate the candidates. The most memorable reference check in my career so far was for an Engineering manager that I was considering hiring. He had already informed his boss at his current (large) company that he was going to leave at some point because he was unhappy with the company’s direction. His boss did not want to lose him, yet he agreed to provide a reference as a personal favor. When I spoke to the boss on the phone, the reality of soon losing one of his best employees hit home, and he started crying as he provided a glowing reference. That hire did indeed turn out very well for me and my company.
There are multiple ways for hiring managers to mess up the reference check process. The first way, of course, is not to perform reference checks at all; this eliminates a valuable source of input on a candidate. The second way is for the hiring manager to outsource the reference check process to someone else, such as someone in Human Resources (HR) or a recruiter. HR people don’t fully know what the hiring manager is looking for; their reference checks tend to be generic. Recruiters have a conflict of interest: they want their candidate to get hired so that they get their commission. Another way to screw up the reference check process, which just happened to me: I was asked to fill out an online supposedly “confidential” webform to provide a reference on someone I know. No webform is going to take the place of a personal conversation.
I do have a set of questions I like to pose to references, to try to get them to open up about the candidate. Perhaps I will post those at some point.