Predictive Hiring: How to Get Started?

I delivered a pair of virtual learning events on May 20th-21st 2020 during the Canadian Innovation Week, which attracted 177 participants (67 for the French session and 110 for the English session):

What follows is adapted from a series of blog posts that complemented both virtual events.

1.  The Project’s Backstory

“Hiring involves making a public commitment to a strategic business goal. Hiring requires that you determine what actually drives business impact. Hiring requires that you determine the profile of the person best qualified to do the work. Hiring requires that you accurately assess skills.” (Bob Corlett)

What is Predictive Hiring?

Staffing and recruitment are too often disconnected from talent and performance management:

  • We don't know what the highest performing employees have in common;
  • This prevents us from identifying the qualifications that lead to superior performance;
  • Since we do not know what those qualifications are, there is no way to tell if our selection tools and methods are effective and accurate predictors of performance.

Predictive hiring is meant to inform and improve personnel decisions by transforming data into insights, more specifically by identifying:

  • The qualifications that correlate with and/or lead to superior job performance;
  • The assessment methods and tools that are the best predictors of performance.

The expected benefits of predictive hiring are to:

  • Replace guesswork and gut feelings in staffing and talent development with evidence-based decision-making;
  • Increase efficiency in staffing by focusing on the qualifications that actually matter for performance and the candidates who are most likely to succeed in the role;
  • Increase relevance and impact of learning interventions, by targeting capabilities and attributes that can meaningfully enhance performance;
  • Improve organizational performance and client satisfaction;
  • Link HR data to business outcomes.

Project documents

Relevant project background documents include:

2.  Crash Course in Predictive Hiring

“This is the Curse of Knowledge. Once we know something, we find it hard to imagine what it was like not to know it. Our knowledge has “cursed” us. And it becomes difficult for us to share our knowledge with others, because we can’t readily re-create our listeners’ state of mind.” (Chip and Dan Heath, “Made to Stick”)

Create Your Own Learning Program

Predictive hiring is completely new to you? Don’t worry! I have curated what I think are the top 1% of resources on the topic so that you can learn in two days what would otherwise take months. Just pick and choose based on your time availability and preferred media.

If You Only Have Half-an-Hour : Challenge Your Assumptions about Hiring

Podcasts

If You Have Half-a-Day: Discover the Art of the Possible (or What Google Already Knows!)

Total required time: 3.5 hours approximately

Book Chapters

Online Guides

Videos

Podcasts

If You Have One Day: Replace Guesswork and Gut Feelings with Science and Empirical Evidence

Total required time: Up to 7.5 hours

Research Papers

Academic Articles

If You Have Two Days: Broaden Your Horizons

Videos

Total Watch time: 7.5 hours

For a short introduction to HR Analytics:

For examples of predictive analytics applied to hiring and performance:

Most insightful takeaways on predictive hiring and data in the PS:

And Just for Fun: Moneyball

Most relevant scenes from the movie Moneyball (10 minutes):

3.  Crawl Before You Walk

“People use data sub-optimally. When they have the opportunity to collect new data, they don't. And when they do get new data, they don't use it in a consistent way or an optimal way.”(Luba Petersen, on What is data blindness?)

Choose the Approach That Makes the Most Sense for You

In light of what we learned through our research and the resources highlighted above, we determine we had three basic options to experiment with predictive hiring in order to solve the problems we identified at the outset of the project:

a) Start from scratch and take a forward only approach. While it would allow designing the experiments from an end to end for the best results, this approach would also require:

  • A substantial investment of time and money upfront;
  • A high volume of recruitment or hiring anticipated in the near future;
  • A significant amount of time elapsing before yielding results (likely more than a year…).

b) Test with your current workforce the selection tools you plan on using for recruitment. This approach would have the benefit of benchmarking the selections tools against the actual performance of employees before running an experiment such as the one described in the previous option. The downsides of this approach are that it likely represents:

  • A very significant investment of money to administer the selection tools you want to pilot;

  • A considerable investment of time to administer the selection tools to current employees for benchmarking purposes;

  • The disruption of operations due to the employees having to take time away from their normal work to subject themselves to the selection tools for the experiment.

c) Examine the historical data in your possession to draw the link between the qualifications and selection tools that have been used in the past, and the performance of the people who were hired once they are in the job. While this approach may require an initial investment of time and effort to collect scattered or fragmented sets of data (of unknown quality), it comes at a fraction of the cost of the alternative options, doesn’t require developing or purchasing any selection tools, only creates minimal disruption for operations (if any at all) and the data to analyze may in some cases be gathered in a matter of days or weeks, as opposed to months or years. Most importantly, this pre-experimental design approach allows building a foundation of knowledge before crafting any sort of experiment, and may even inform future hiring and recruitment processes based on the empirical evidence.

We recommend starting predictive hiring with Option C, as it is the least expensive, the fastest to yield findings, and the one presenting the lowest risk in case of inclusive results or failure. Moreover, if you discover in the process that the data you currently have is of poor quality, you will now understand its limitations and have useful information to correct the situation moving forward and make the necessary changes that will allow you to start collecting better data for future use.

Identify the Biggest (or Most Painful) Problem(s) You Can Solve for the Business

A simple way to tackle this is to look at the composition and distribution of your workforce by jobs (or groups and classifications), and ask yourself questions such as “Which jobs represent…

…the largest size in our workforce?

…the biggest volume of hiring - internal and external?

…the highest turnover (volume or rate)?

…the most important feeder group(s) for other jobs?

…the most significant amount of time and/or dollars spent on staffing


Other elements you may want to consider:

  • For which jobs are hiring managers most dissatisfied with the quality of hire?
  • Which jobs deliver on key business outcomes or provide service to Canadians?
  • Which jobs would most urgently benefit from improvements in how they are performed (for instance, as measured through client satisfaction)?

The more widespread and acute problem for the business, the better the chances of building a strong case to collecting and study the data that would inform predictive hiring.

Once you have narrowed down your options to a few jobs, you can refine your questions, for instance:

  • If we want to improve time-to-staff by reducing the number of qualifications we put on job posters, which are the few qualifications we should keep (because evidence shows they are correlated with superior job performance)?
  • If in order to meet the demands of virtual recruitment (such as in post-Covid-19 world...) we had to adapt all the selection tools we currently administer in paper form and in-person, which one(s) should we prioritize first (because we know these selection tools are the best predictors of job performance)?
  • Which competencies and skills have the most impact on client satisfaction?
  • Etc.

Knowing which questions you want to answer will help you determine which data to go hunting for and what to do with the data once you have collected it.

Start with the End in Mind: Performance Data

Consider the data that could be gathered and linked to individuals through a unique identifier:

  • Task-related performance vs. contextual performance (i.e. ability to successfully interact with others): Task-related performance data is probably more objective, more reliable, and often easier to quantify – all good reasons to start there. Examples could be the volume of calls of a call center agent, or the satisfaction rate of the clients they have assisted.
  • Quantity and quality information: Which metrics are available? For example, the productivity of translators can be measured in the number of words they translate per minute, while the quality of translation is evaluated by their team leaders against criteria such as accuracy, readability, adaptation, correction, presentation, etc. Both can be considered.
  • Calibrated vs. non-calibrated performance evaluations: Are there any standards applied by all supervisors when they review the performance of their direct reports to ensure the performance appraisals are done in a consistent manner across the organization? When idiosyncrasies in ratings are minimized or eliminated, the resulting data is better. For this reason, calibrated performance evaluations can be considered.
  • Snapshot data vs. longitudinal performance data: career development and apprenticeship program can be an interesting source of data, as several data points can be collected over time (sometimes years).

Assessment and Selection Data

For the purpose of predictive hiring, you’ll need quantifiable data; “pass/fail” ratings won’t be sufficient because you need range in the data to measure correlations. Here are some examples of data that would lend itself well to predictive analysis, as long as you can link it to individuals through a unique identifier:

  • Test scores
  • Ratings based on well-defined rubrics, for instance with specific behavioral indicators
  • Individual assessment ratings coming from multiple assessors
  • Rankings

When a Small Step Is a Giant Leap

While predictive hiring is something relatively new, we should not feel intimidated at the idea of experimenting with it because:

  • You don’t need that much data;
  • You don’t need perfect data either;
  • You don’t need the data for all the qualifications you find on job posters; just start with the data that meet the criteria listed in this blog post.

If you want to find out what the data you have access to can do for you, the first thing to do is to gather it.

4.  Wins, Challenges and Lessons Learned

"Sometimes we look at digital transformation and data maturity from a purely technical perspective; we don't look at it from the organizational change management component." (Ian Stewart)

I initially started working on predictive hiring thinking it would be a cutting-edge innovation project leveraging advanced people analytics. I’m now inclined to think that this was instead a change management initiative.

Early Wins

After just a few months of researching the topic, I already had developed a proof-of-concept based on a real data set that we could show around. The data visualizations were effective to show what a product may look like.

It was still too conceptual to get us immediate access to more substantial data sets, but promising enough to be selected for the second cohort of Experimentation Works. This meant that the project would receive support for running rigorous experiments in the form of expert advice, central agency guidance, and ongoing training and learning events, throughout 2020.

This in turn was enough to convince a few strategic partners to endorse the project and play a role in its governance. By this point the project was being designed for credibility.

Furthermore, despite my desire to co-design the project with partners and adopt an agile method to project management – which I thought would be best suited for an experimentation project such as this one – I realized that none of these suggestions sufficiently eased the uncertainty that accompanied a project located at the intersection of data analytics, innovation, and human resources management.

Challenges

In light of conversations I had with colleagues who have much more experience than I with data projects, here are some obstacles you should expect in proceeding with the implementation of predictive hiring:

1. The challenge of overcoming basic (and often false) assumptions about the data or its collection (i.e. thinking that what we’re currently doing works fine), combined with a status quo bias where change is perceived as a loss.

2. The multifaceted challenge of “data blindness”:

    • Failure and missed opportunities to collect data
    • Inconsistent collection of data
    • Missed opportunities to use the data we collect
    • Failure to act on data we have
    • Failure to realize that the lack of data or poor quality data is a valuable insight in itself

3. The challenge of creating something that will first add some work to the plate of people before it can take any away.

4. The challenge of doing something that has not yet demonstrated its value to the business.

5. The challenge of sustaining momentum through a storm of other competing priorities and urgencies the organization simply can’t ignore.

Lessons Learned

As I wrap up the initial phases of the project, I find myself struggling with one basic question: Who owns the problems I’m trying to solve?

Consider my original problem statement:

"Staffing and recruitment are too often disconnected from talent and performance management:

  • We don't know what the highest performing employees have in common;
  • This prevents us from identifying the qualifications that lead to superior performance;
  • Since we do not know what those qualifications are, there is no way to tell if our selection tools and methods are effective and accurate predictors of performance.”

 I wonder indeed:

  • Whose problem is it: the CEO, HR, hiring managers, etc?
  • Who has the most to gain from solving the problem?
  • Who has access to the data that could provide insights on how to solve the problem?
  • Who is best positioned to make it happen?

Once I gain clarity on the who, I should be able to get closer to the why and align the predictive hiring with a greater purpose, a mission, a vision.


Popular Posts