Miguel Afonso Caetano<p>"In an experiment involving 22 leading LLMs and 70 popular professions, each model was systematically given a job description along with a pair of profession-matched CVs (one including a male first name, and the other a female first name) and asked to select the more suitable candidate for the job. Each CV pair was presented twice, with names swapped to ensure that any observed preferences in candidate selection stemmed from gendered names cues. The total number of model decisions measured was 30,800 (22 models × 70 professions × 10 different job descriptions per profession × 2 presentations per CV pair). The following figure illustrates the essence of the experiment.</p><p>Despite identical professional qualifications across genders, all LLMs consistently favored female-named candidates when selecting the most qualified candidate for the job. Female candidates were selected in 56.9% of cases, compared to 43.1% for male candidates (two-proportion z-test = 33.99, p < 10⁻252 ). The observed effect size was small to medium (Cohen’s h = 0.28; odds=1.32, 95% CI [1.29, 1.35]). In the figures below, asterisks (*) indicate statistically significant results (p < 0.05) from two-proportion z-tests conducted on each individual model, with significance levels adjusted for multiple comparisons using the Benjamin-Hochberg False Discovery Rate correction."</p><p><a href="https://davidrozado.substack.com/p/the-strange-behavior-of-llms-in-hiring" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">davidrozado.substack.com/p/the</span><span class="invisible">-strange-behavior-of-llms-in-hiring</span></a></p><p><a href="https://tldr.nettime.org/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AI</span></a> <a href="https://tldr.nettime.org/tags/GenerativeAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>GenerativeAI</span></a> <a href="https://tldr.nettime.org/tags/LLMs" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LLMs</span></a> <a href="https://tldr.nettime.org/tags/GenderBias" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>GenderBias</span></a> <a href="https://tldr.nettime.org/tags/AIDiscrimination" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AIDiscrimination</span></a> <a href="https://tldr.nettime.org/tags/AIBias" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AIBias</span></a> <a href="https://tldr.nettime.org/tags/HR" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>HR</span></a></p>