Interviewing for Evidence

One area where companies struggle is recruiting good people. I see enormous variability in quality and style of interviewing. This is something I care about, having spent nearly a decade working with some amazing recruiters at a global technology consultancy, and variously as an employee, contractor and independent consultant in many organisations.

There are many aspects to identifying and hiring great candidates, not least attracting the right people in the first place, and casting your net wide enough and creatively enough that you see beyond the usual suspects. A great piece of advice I received early on is ‘Do not hire yourself!’

Regardless of who ends up in the pipeline, one consistent thread throughout the interviews I have observed or experienced is the lack of consistency.

This is not about the type of interview; it is as true of deep technical interviews or the ones where a candidate might give a presentation, as it is ‘cultural’ or more nebulous ones. I do not want to get into the merits or otherwise of different types of interview, that is a windmill to tilt at another day.

I want to describe a model I was taught about 20 years ago, that I have applied and passed on faithfully and successfully ever since. I call this model interviewing for evidence.

The case for evidence ¶

When you interview a candidate, you are making a mutual buying decision. You want to know whether to make them an offer, and you want them to have enough information to decide whether this role/team/company is right for them.

In many cases, hiring decisions are subjective. The candidate ‘seemed smart’, ‘impressed me’, ‘knew what they were talking about’, and other observations that are hard to quantify or validate.

This has a number of issues. First, it makes hiring inconsistent, which means we don’t have adequate controls on the quality of people joining the company. It means we might be passing on candidates who would be good hires, but who didn’t present themselves in a way that met the (subjective and variable) bar.

Even with multiple people and checkpoints involved, a subjective process is open to challenge—and potentially litigation—by unsuccessful candidates. ‘They just didn’t impress us’ will not cut it with an employment lawyer.

The key here is evidence: things you observed, things you heard, things the candidate did or said. The important distinction is that these are factual rather than inferred. ‘The candidate answered a different question’ is a fact. ‘The candidate seemed nervous’ is an assumption. What did you observe that led you to believe they were nervous? Write that down instead.

Four types of evidence ¶

There are four primary types of evidence I look for in an interview, which are, in order of general importance, experiential, hypothetical, opinion, and credential.

I credit this model to John Hundreiser, known as ‘h’, by way of Suzi Edwards-Alexander, two recruiting legends I have been fortunate enough to work with.

Experiential evidence ¶

Experiential evidence is about things the candidate has done. These questions tend to take the form:

• ‘Tell me about a time when…’
• ‘Can you give me an example of…?’
• ‘Then what did you do?’
• ‘How did you respond?’

A good experiential question is open, which gives the candidate a chance to share their experience, while being specific enough to prevent the candidate wandering off on tangents, or at least to allow the interviewer to interject if they do.

Hypothetical evidence ¶

Hypothetical evidence is about what the candidate thinks they would do in a given situation. Hypothetical questions tend to take the form:

• ‘What would you do if…?’
• ‘If you could redo that project/team/situation, what would you do differently?’
• ‘What else could you have done? How do you think that would have turned out?’
• ‘How might you…?’

As an interviewer you cannot prove or disprove a hypothetical answer, but it is still useful in several respects:

• It enables the candidate to give an answer where they have no direct experience of a situation.
• It gives the interviewer an insight into how the candidate approaches problems, how they think, what they do and do not consider, whether they ask for help, how they respond to advice, feedback, clarifying questions.
• It establishes a context in which the interviewer can explore a topic.
• It allows the interviewer to cross-reference hypothetical answers with experiential ones to check whether the candidate is consistent in their understanding of their own behaviour.
• It allows the candidate to revisit a situation and discuss options and alternatives, demonstrating self-reflection and growth.

Credential evidence ¶

This is the most straightforward of the four evidence types. You are asking what the candidate is qualified to do. Credential evidence might be college records, professional certifications in technologies or methods (CCNA, CSM, CIPD, coaching qualifications, etc.).

While this is largely a bookkeeping exercise, it may be critical for some roles, especially those involving legal, compliance, HR, security, or similar regulated or more formal roles.

Opinion evidence ¶

Opinion evidence is asking the candidate what they think about things. Opinion questions take the form:

• ‘What is your opinion on…?’
• ‘What do you think about …?’

Opinion evidence is useful not only to gain an insight into the candidate’s values and beliefs, but to see how they react when their opinions are challenged. Do they have ‘strong opinions weakly held’; do they dig their heels in in the face of compelling contradictory evidence; do they sway with the wind to try to please the interviewer, or hold their ground and counter? Is their tone respectful, scornful, frustrated, condescending, encouraging?

You can dig into the source or origin of this opinion, to see whether the candidate has researched or read into (or experienced!) the topic. I am always delighted when a candidate says they do not have an opinion on a topic. That shows both humility and vulnerability in what is often a stressful situation.

You can learn a lot about someone by asking their opinion on a subject.

The evidence-based interview ¶

So how does this translate into an interview setting? Remember, the goal of an interview is for both parties to gain enough information to make a buying decision. The candidate should know enough about the company, the role, and ideally the team, to know whether they want to proceed. The interviewer should have enough evidence to make an informed and defensible decision about whether to proceed, or where the gaps are for further investigation.

When I go into an interview, I have either a page marked into quarters, or four index cards, headed with each of the evidence types. As I am interviewing, I take notes into one of these quarters. This means I can glance down and see where I am light on any type of evidence, and guide the conversation in that direction.

I tend to prioritise experience over hypothetical over opinion over credential, but this is an unscientific position I have come to after lots of interviewing! Different roles may prefer different weightings. More junior roles or internships, for example, will lean more heavily on hypothesis because the candidate does not yet have the experience. Adapt to the context you are in.

In any case, I like to finish an interview with a reasonable amount of evidence in each of the four areas. This allows me to make evidence-based observations after the interview, like ’the candidate offered three alternatives, and described the trade-offs, when I asked how else they might have stored the data’ rather than ’the candidate seemed really switched on about data.’

After an interview, I like to write up my notes as bullets based on the four sections, with some kind of summary notes. This is the most lightweight method I have found (I am generally terrible at following up, so this at least forces me to provide something that colleagues can use!), and it acts as a memento in case someone asks me to go into more detail about a note.

Paired interviewing ¶

Adjacent to the topic of evidence-based interviewing, I am a fan of interviewing in pairs. While one of you is speaking, the other is observing the dynamic between the interviewer and candidate, and can make interventions where necessary.

Taking turns asking questions gives each interviewer time to gather their thoughts, and maybe recap the recent conversation in a way that can get lost when you only write up your notes at the end.

Some people advocate for a ‘good cop / bad cop’ model where one interviewer puts the candidate under artificial pressure, say. I prefer people to be their authentic selves in an interview rather than playing games with the candidate, unless you are clear about the rules: ‘OK, I am going to put you in a pressurised context now because sometimes you will need to make decisions in a hurry and with limited information.’

The best possible chance ¶

My golden rule is to try to create an environment that gives the candidate the best opportunity they could have. The stakes are already high enough; I do not want them having to second-guess my motives as well. I do not manipulate and lie to my colleagues, so why would I do that with a potential future colleague?

So, seek evidence, be authentic, and travel in pairs, and you too could have a body (of evidence) like mine. And be kind to your candidates, they deserve it.

Oh, and hit up ‘h’ (USA) or Suzi (UK) if you want help with any of this. Completely unsolicited recommendation with no interest to declare. They are just both extraordinarily good at this.