12 AI and Legal Ethics 12 AI and Legal Ethics

12.4 AI in the Workplace 12.4 AI in the Workplace

12.4.1 Case Study: Algorithmic Hiring Systems and Compliance Risks 12.4.1 Case Study: Algorithmic Hiring Systems and Compliance Risks

 

Background: TechStart

TechStart is a 10-year-old technology company based in Chicago that develops software for retailers. They have around 200 employees and recently received $40 million in VC funding, enabling rapid growth. However, TechStart has struggled with workforce diversity:

  • 18% of their engineering staff are women

  • 6% are underrepresented minorities

  • Less than 3% have self-identified as having disabilities

These figures fall short of both national averages in the tech sector and the demographics of Chicago itself. Nationally, women make up approximately 21–22% of software developers, and Black and Hispanic individuals together account for about 15–17% of the tech workforce. In contrast, approximately 51% of Chicago residents identify as female and over 60% identify as members of racial or ethnic minority groups. TechStart leadership has acknowledged this gap and committed to exploring tools that might help expand their talent pipeline.

The Hiring Tool

To expand their pipeline and mitigate known human biases, TechStart adopted an AI hiring tool from TalentTech Inc. The tool evaluates resumes, video interviews, and behavioral cues. It was trained on general industry data and TechStart’s own historical hiring outcomes.

Compliance Audit Findings (18 Months Later)

A third-party audit conducted after 18 months of use identified several patterns in how the AI tool scored and evaluated job candidates:

  • Female engineers consistently scored lower than male counterparts with equivalent qualifications. This was most pronounced in video interview assessments and behavioral scoring.

  • Black candidates scored lower on average than candidates from other racial or ethnic groups.

  • Women over the age of 40 were disproportionately flagged as “low engagement” during facial and behavioral analysis.

  • Neurodiverse candidates scored significantly lower in facial/body language assessments, particularly when assessed for eye contact and fidgeting.

  • Overall hiring outcomes reflected these disparities: 20% of engineering hires were women; 8% were underrepresented minorities.

The vendor had trained the model in part on TechStart’s past hires, potentially embedding historical bias into the algorithm. At the time of the audit, TechStart had not conducted an independent validation of the tool’s fairness or accuracy, nor had they implemented a process for accommodations related to algorithmic assessments.

These outcomes raise potential legal exposure under federal and state employment and disability laws, especially in jurisdictions with developing or active algorithmic fairness legislation.

Legal & Policy Context

The readings assigned this week will provide the legal background you need to complete this case study. You do not need to conduct additional research, but you may cite outside sources if you wish. These materials will also support your upcoming AI Adoption Evaluation Assessment (Part 2).

You may also wish to consider that while Chicago does not currently have a law like New York City’s Local Law 144, city officials have publicly discussed the possibility of enacting similar legislation.. You do not need to conduct additional research; rely on the concepts and authorities covered in your assigned materials for this module.