How AI can help the recruitment process—without hurting it

This fall, LinkedIn rolled out its first AI agent for recruiters: Hiring Assistant. The new product, powered by OpenAI’s GPT, automates a range of tasks that would normally eat up a recruiter’s time, such as crafting job descriptions, sourcing candidates and handling outreach. Using LinkedIn’s vast trove of user data, the tool prioritizes skills over traditional filters, which typically sort candidates based on factors like location or alma mater.

Hiring Assistant is the latest entry into the vast and varied field of AI designed for recruitment: There are tools from Microsoft, Indeed, Google, IBM, and many others. And there’s a demand for it: A recent IBM survey found that human resources and talent acquisition needs account for 19% of use cases driving AI adoption. Like many of its peers, LinkedIn is aware of the potential biases of its new tool and says it will work to mitigate them. But will it be enough?

“[HR Assistant] is a great idea, but we need to have transparency and need to know what skills or keywords in the job description the tool is inferring upon,” says Hilke Schellmann, journalist and author of The Algorithm: How AI Decides Who Gets Hired, Monitored, Promoted and Fired and Why We Need to Fight Back Now. “We have seen too many misfires in this kind of technology.”

As the usage of AI-driven recruitment tools grows, state and city legislatures are beginning to take notice. New York City now requires companies to disclose the performance of AI hiring systems and conduct bias audits. In California, a new law protects against discrimination based on intersectional identities (although it doesn’t specify AI). And the US Department of Labor has created a framework to help employers promote inclusive hiring as the use of these tools grows.

Kyra Wilson, a doctoral student at the University of Washington’s Information School, is interested in investigating how AI hiring tools might discriminate across various occupations and social groups. Recently, she led a study that involved looking at 554 resumes and 571 job descriptions, with names altered to represent different genders and races. "We wanted to see whether these tools could unfairly disadvantage certain candidates," Wilson says.

The researchers tested three open-source LLMs from Salesforce, Contextual AI and Mistral AI. What they found was striking: despite controlling for qualifications like experience and education, the models still disproportionately favored candidates with white-associated names 85% of the time, and those with female-associated names only 11% of the time. And they found that the models didn’t just replicate existing societal biases—they also introduced new patterns.

“The models we used were not fine-tuned on any domain-specific datasets, so we observed that overall societal biases favoring white and male people also started occurring in positions which are not typically associated with these groups,” Wilson says. “Using these models at scale then could have the potential to change societal patterns of employment in negative ways.”

Biases linked to intersectionality (in this case, overlaps between race and gender) also surfaced in the results, particularly for Black men, who were disadvantaged in up to 100% of cases. “Intersectionality was an important part of our investigation because it's a better representation of how people are discriminated against in real life,” Wilson said. “People don't perceive characteristics like gender and race in isolation, and so studying them in isolation doesn't necessarily provide a full picture of the true societal impacts of these systems.”

While Wilson’s research only investigated identities that were signaled by names, she noted that in the real world, people might signal their identities through awards they've received, places they've lived and even the words that they use in their resumes. All of these factors could play a role in how AI evaluates them, and because a lot of them are also relevant to distinguishing strong candidates, they can't easily be removed during review (the way names can be) without getting rid of important information.

“Learning more about how these factors can signal intersecting identities and whether that plays a role in AI evaluation is an important next step for researchers and model developers,” Wilson says.

Data, after all, is the foundation on which these AI models are built. And according to IBM Senior Research Scientist Moninder Singh, it’s where most biases—whether implicit or explicit, historical or societal—are introduced. The most effective way to mitigate bias in AI tools of any kind is to address these issues early during the LLM training stage (and, if applicable, during subsequent fine tuning).

Singh explains that for most organizations building AI-based tools, such as those used by recruiters, addressing bias isn’t always feasible at the foundational level. Few businesses have the resources to train their own LLMs, so they typically rely on pre-trained models like OpenAI’s GPT or Google’s PaLM and fine tune them for specific use cases. However, this fine tuning can only go so far, Singh says. In practice, bias mitigation often happens at the data level, with companies tailoring the LLMs to their specific datasets, which in turn are shaped by the data they have access to.

“Despite employing best practices and fine tuning with potentially huge amounts of data that is relevant to the specific task, such as hiring, biases are still going to show up when the systems are applied in real life,” Singh says.

At the output level, Singh explains, companies can implement a range of strategies to detect and mitigate biases as they arise in real time. For instance, AI hiring tools might generate a shortlist of candidates, and companies can evaluate those recommendations for fairness, which is required under New York City’s new law. If a bias is detected—say, one group is consistently ranked lower than others—developers can adjust the model by either refining the training data or using post-processing techniques to reweight the recommendations.

Post-processing methods can also be used to adjust scores or rankings to make them fairer without negatively impacting the overall performance of the system, Singh explains. Tools like IBM’s AI Fairness 360, an open-source toolkit for bias detection and mitigation, provide a suite of techniques for doing just that. IBM is also working on bias detection through models like the Granite Guardian 3.0, which is fine tuned to identify bias risks in AI-generated content.

These models can be used to evaluate outputs, such as resume rankings, by generating explanations for decisions and checking if bias indicators appear in those explanations. Similarly, IBM’s watsonx.governance toolkit enables the governance of generative models, including bias detection, deployed on the watsonx platform. And IBM’s SocialStigmaQA benchmark tests LLMs for biases related to stigmas that are often overlooked in traditional bias tests but that can be critical in sensitive applications like hiring, such as those around mental health or drug use.

“Despite the best efforts by the developer of an AI-based system, such as a hiring tool, to eliminate bias, it is important to note that it cannot possibly address situations specific to every end user, especially if that end user also does not take enough care to not magnify or introduce biases at that level,” Singh says. “An end user of a hiring tool must similarly bring in diversity at every step.”