Natural language processing (a subfield of Machine Learning) to accelerate the recruitment process and correct diversity & inclusion issues
See how you can enhance candidate experience, detect diversity & inclusion issues, automate steps and improve the overall effectiveness of your recruitment process through natural language processing
Following the publication of my article The Potential for Machine Learning in HR: 3 Business Application Examples, a few reached out with questions on the different machine learning techniques cited. In response, I’d like to take a deeper dive in one of the mentioned techniques: natural language processing (NLP) and explain how it can be leveraged in a company’s HR recruitment process. I will start by explaining what NLP is, why it is relevant in the recruitment process and walk you through how companies can use it. I will conclude with a note of caution based on a recently lived example at Amazon.
What is natural language processing (NLP) ?
Natural language processing (NLP) is a subfield of linguistics, computer science and machine learning that allows the processing and analysis of large amounts of text data (commonly referred to as “unstructured data” in data science). There are countless ways to use NLP, here are a few examples:
· Convert audio to text (speech to text) and vice-versa (text to speech)
· Identify trends in public opinions (sentiment analysis)
· Identify the key topics discussed in a text (topic segmentation)
· Summarize large amounts of text (automatic/text summarization)
· Answer open-ended and yes/no questions (question answering)
· Translate a document (machine translation)
The applications of NLP are endless, and only becoming increasingly powerful in combination with other machine learning techniques. If ever you are interested in getting an idea of how advance algorithms have become, I recommend you read up on GPT-3 from OpenAi and BERT from Google.
Why use NLP in the recruitment process ?
Not only is recruitment usually the heaviest HR activity within a company, but it also handles the greatest amounts of text data (CV, cover letters, forms, interview grids, etc.). There are many opportunities to automate some process steps, as well as ways to improve the candidate experience and detect diversity and inclusion issues, which all have a significant weight on a company’s employer brand.
When reading the following examples, keep in mind you do not need a new software or technology to implement these solutions; algorithms can be coded directly into a company’s environment.
1 — Free your job descriptions from bias
There may be unintentional bias inserted in your current job descriptions. NLP can be used to review existing job descriptions and detect wording that may be favoring a type of subgroup of the population. For example, engineering job descriptions may have a more “masculine” or “aggressive” connotation, unintentionally discouraging women applicants. NLP can identify these biases and even recommend replacement vocabulary.
2 — Provide a better candidate experience, while gaining more complete information
NLP can be used to interact with candidates and support them in their application process, in the form of a chatbot or virtual assistant. An NLP algorithm can process the information submitted by a candidate (e.g.: cover letter, CV), pre-fill any fields in the application software used by the company, and ask the candidate any remaining unanswered questions or clarification on information provided. For example, if there is a gap between employment periods, the virtual assistant/chatbot could request more information to the candidate. This use case for NLP ensures all relevant applicant data is captured and offer a greater candidate experience and good first impression of the company.
3 — Automate and de-bias candidate pre-screening
Automate the review: NLP can be used to review candidate applications and assess how much their profile matches the job requirements, and then sort them in order of relevance. This can be a big win in terms of efficiency in the pre-screening sub-process.
De-bias the process: In cases where few CVs are received for a specific position, recruiters usually prefer to personally review all applications. NLP can be used to remove names or replace any wording that could potentially hint at the individual’s characteristics (e.g.: gender is reflected in the grammar of some languages, such as French) to ensure personal bias does not factor into the process.
4 — Conducting interviews
Although NLP could be used to create a bot that conducts the candidate interview, it may not be part of the candidate experience you are aiming for. However, some recruiters have found it helpful to leverage an NLP based bot to listen in to interviews and pick up on themes and sentiments shared by the candidate, to better assess their profile.
NLP could even be leveraged as a training tool to identify if any bias vocabulary was used to conduct the interview and provide corrective action.
Caution when using machine learning
Beyond NLP, there are many ways to leverage other machine learning techniques to support and enrich the recruitment process to better assess the market talent pool, evaluate the effectiveness of your sourcing channels and compare yourself against your competitors.
Something to keep in mind: the world isn’t perfect and neither is your data. Machine learning algorithms are mostly trained with historical data. Bias reflected in your data puts you at risk of repeating bias, depending on how there are used. A good example that illustrates this is the case of Amazon back in 2018:
Amazon was accidentally favoring male applicants to female, based on their machine learning algorithm used to pre-screen applications. The algorithm was built to favor applicants that compared better against the profile of existing employees. Since their employees are predominantly men, the algorithm was favoring male attributes, especially those captured in the vocabulary used and their writing style.
From this case, it is important to remember that as powerful as machine learning can be, it must be implemented cautiously to avoid accidental discrimination and enable a more effective and efficient recruitment process.
