12 AI and Legal Ethics 12 AI and Legal Ethics
12.1. PRELIMINARY GUIDELINES ON NEW JERSEY LAWYERS' USE OF ARTIFICIAL INTELLIGENCE
12.2. PENNSYLVANIA BAR ASSOCIATION COMMITTEE ON LEGAL ETHICS AND PROFESSIONAL RESPONSIBILITY and PHILADELPHIA BAR ASSOCIATION PROFESSIONAL GUIDANCE COMMITTEE JOINT FORMAL OPINION 2024-200 ETHICAL ISSUES REGARDING THE USE OF ARTIFICIAL INTELLIGENCE
12.3. ABA Formal Opinion 512: Generative Artificial Intelligence Tools
12.4 AI in the Workplace 12.4 AI in the Workplace
12.4.1 Case Study: Algorithmic Hiring Systems and Compliance Risks 12.4.1 Case Study: Algorithmic Hiring Systems and Compliance Risks
Background: TechStart
TechStart is a 10-year-old technology company based in Chicago that develops software for retailers. They have around 200 employees and recently received $40 million in VC funding, enabling rapid growth. However, TechStart has struggled with workforce diversity:
-
18% of their engineering staff are women
-
6% are underrepresented minorities
-
Less than 3% have self-identified as having disabilities
These figures fall short of both national averages in the tech sector and the demographics of Chicago itself. Nationally, women make up approximately 21–22% of software developers, and Black and Hispanic individuals together account for about 15–17% of the tech workforce. In contrast, approximately 51% of Chicago residents identify as female and over 60% identify as members of racial or ethnic minority groups. TechStart leadership has acknowledged this gap and committed to exploring tools that might help expand their talent pipeline.
The Hiring Tool
To expand their pipeline and mitigate known human biases, TechStart adopted an AI hiring tool from TalentTech Inc. The tool evaluates resumes, video interviews, and behavioral cues. It was trained on general industry data and TechStart’s own historical hiring outcomes.
Compliance Audit Findings (18 Months Later)
A third-party audit conducted after 18 months of use identified several patterns in how the AI tool scored and evaluated job candidates:
-
Female engineers consistently scored lower than male counterparts with equivalent qualifications. This was most pronounced in video interview assessments and behavioral scoring.
-
Black candidates scored lower on average than candidates from other racial or ethnic groups.
-
Women over the age of 40 were disproportionately flagged as “low engagement” during facial and behavioral analysis.
-
Neurodiverse candidates scored significantly lower in facial/body language assessments, particularly when assessed for eye contact and fidgeting.
-
Overall hiring outcomes reflected these disparities: 20% of engineering hires were women; 8% were underrepresented minorities.
The vendor had trained the model in part on TechStart’s past hires, potentially embedding historical bias into the algorithm. At the time of the audit, TechStart had not conducted an independent validation of the tool’s fairness or accuracy, nor had they implemented a process for accommodations related to algorithmic assessments.
These outcomes raise potential legal exposure under federal and state employment and disability laws, especially in jurisdictions with developing or active algorithmic fairness legislation.
Legal & Policy Context
The readings assigned this week will provide the legal background you need to complete this case study. You do not need to conduct additional research, but you may cite outside sources if you wish. These materials will also support your upcoming AI Adoption Evaluation Assessment (Part 2).
You may also wish to consider that while Chicago does not currently have a law like New York City’s Local Law 144, city officials have publicly discussed the possibility of enacting similar legislation.. You do not need to conduct additional research; rely on the concepts and authorities covered in your assigned materials for this module.