IÉSEG Insights » Resource Center » Big Data & AI » When recruitment algorithms reinforce the biases companies are trying to reduce

When recruitment algorithms reinforce the biases companies are trying to reduce

11/12/2026

4 min

As the European Union implements legislation on artificial intelligence, the use of AI in corporate recruitment processes is increasingly drawing attention. Research shows that algorithmic biases do not always stem from the algorithms themselves, but from human decisions “embedded” in the data used to train them.

In late 2025, the French anti-discrimination authority (la Défenseure des Droits) raised concerns about the job posting system of a major social media platform. Some job listings were shown more often to men, while others were shown more frequently to women. This case illustrates an issue that has now become central for businesses and policymakers: how can AI systems produce unfair results even without having been explicitly programmed to discriminate?

This question has become even more important with the gradual implementation of the European AI Regulation, which classifies systems used in recruitment and employment as high-risk applications. The message is clear for companies that use AI to select, rank, or recommend candidates… Removing sensitive variables such as gender or ethnic origin is not enough to fulfill their responsibilities on fairness.

Data is not ‘neutral’

A common misconception is that bias arises only when an algorithm directly uses protected characteristics or data sources. However, the problem is often more subtle. AI systems learn from historical data, which itself stems from human decisions that may already reflect inequalities.

Online job platforms and recruitment tools increasingly rely on ranking systems. These generally prioritize indicators such as past performance, the number of assignments completed, or accumulated experience.

From a business perspective, this approach seems perfectly rational, as it reduces recruitment costs, makes it easier to identify relevant candidates, and values experience. But it also carries a risk. These metrics assume that past results reflect individual merit alone. Yet they may also be the result of unequal access to opportunities.

A dangerous feedback loop?

This can create a feedback loop. A person who enjoys a slight advantage early in their career gains more experience. This experience improves their ranking in automated systems, which increases their visibility and opens new opportunities. Over time, this initial advantage grows stronger.

For companies that use AI-based recruitment tools, this means that a ranking presented as “objective” or “performance-based” may reproduce and amplify preexisting inequalities.

From human inequalities to algorithmic biases

Our research on online freelancing platforms shows that these mechanisms can affect multiple groups simultaneously. On average, women accumulate fewer completed assignments than men with comparable characteristics, such as age, education level, hourly rate, ratings received, and profile verifications. This difference then translates into lower visibility in the rankings. The gaps can be even more pronounced when multiple factors combine: women from ethnic minorities, for example, are often penalized more than would be predicted by the effect of each of these characteristics taken in isolation.

Similarly, young professionals often have less experience to highlight and may be systematically disadvantaged in the rankings, despite having comparable skills. The result is a “visibility gap” that does not necessarily reflect the candidates’ actual abilities.

One of the key findings is that these results can occur even in the absence of any explicit demographic information. Biases then manifest through indirect variables, such as the number of assignments already completed. This indicator, which reflects past hiring decisions, appears neutral but is closely linked to structural disadvantages.

Opening the black box

For organizations deploying AI in recruitment, talent management, or human resources, several lessons emerge. First, removing sensitive data is not enough. A system can continue to reproduce inequalities if it learns from historical results that are themselves biased. One might wonder whether completely removing sensitive data from freelance profiles would resolve the initial human bias.

Second, optimization criteria deserve special attention. Tools designed to find the “best candidates” risk favoring those who have already benefited from greater visibility or privileged access to opportunities.

Finally, feedback loops must be viewed as a genuine governance challenge. When a system influences candidates’ visibility, and that visibility shapes the data used in the future, biases can become self-perpetuating.

A major challenge for AI governance

The implementation of European regulations marks a shift in perspective. The question is no longer merely whether an algorithm uses sensitive data, but also whether it produces discriminatory effects through indirect mechanisms.

The stakes extend far beyond recruitment. Today, algorithms also influence credit decisions, insurance pricing, personalized recommendations, and content moderation. In all these areas, understanding the origin of the data is becoming just as important as the technical design of the models.

This article is a translated version of an article originally published in French on the Conversation France by Professors Matthijs Meire and Kristof Coussement from IÉSEG.