Who Decides When a Machine Decides, and Who Watches?
Why it Matters
When software turns down a job applicant or names a suspect, it can look as if no one decided, and the law does not accept that answer. It traces each decision back to people: the employer that chose the screening tool, the vendor that built it, an officer who treats a face search as proof. It can also judge a decision by its results, so a count of whom a screen passed and whom it rejected can support a claim of discrimination no one intended. You test a stand-in for a Florida retailer’s résumé screener, or you ask a Volusia County police agency for its rules on face searches and plate cameras.
Current Context
On September 14, 2026, the applicants suing Workday over its AI screening tools asked the court to certify a class (opens in a new tab) with four subclasses: African American applicants, women, applicants over 40, and applicants with disabilities. The motion rests on disparate impact, and the federal enforcers have walked away from that theory. In June, the Justice Department’s Office of Legal Counsel called the EEOC’s disparate-impact guidelines unconstitutional (opens in a new tab), and on August 7 the FTC said it would no longer pursue disparate-impact claims in any context (opens in a new tab). On August 31, Florida’s Department of Transportation revoked the permits for license-plate cameras on state roads, citing misuse and privacy concerns, and the Volusia Sheriff’s Office switched its cameras off (opens in a new tab). For now, the checks on a machine that decides or watches come from private plaintiffs and state officials more than from federal agencies.
Key Concepts
Disparate Impact
Liability for a practice that looks neutral but falls more heavily on a protected group, proved by counting results rather than by showing intent. Under Griggs v. Duke Power Co. and the 1991 amendments to Title VII, the plaintiff names the specific practice and shows the disparity it causes. The employer must then show that the practice is job related and consistent with business necessity.
Disparate Treatment
Liability for treating a person differently because of a protected characteristic, which turns on intent. It reaches a tool built to use the characteristic or a stand-in for it, such as a graduation year for age, and an employer that keeps using a tool it knows discriminates. For age, it is the only claim the ADEA gives a job applicant in Florida’s federal courts, because the Eleventh Circuit reads the Act’s disparate-impact provision to protect employees, not applicants.
Four-Fifths Rule
A rule of thumb from the federal Uniform Guidelines on Employee Selection Procedures: a group selected at less than four-fifths the rate of the most-selected group generally counts as evidence of adverse impact. Divide the lower selection rate by the higher, and a ratio under 0.80 flags the practice for a closer look. It is an agency’s yardstick, not a statute, so it binds no court, and in a small sample a gap that size can appear by chance.
Vendor as Agent
The theory that a company performing a hiring function for an employer can be liable as the employer’s agent under Title VII, the ADEA, and the ADA, though it never employed the applicant. In Mobley v. Workday, a federal court in California accepted it at the pleading stage, reasoning that an employer that lets a screening tool reject applicants has delegated a function employers traditionally perform. Under it, a suit against one vendor can reach hiring at many employers at once.
Risk Score and Due Process
A risk score predicts, from data about groups of people, how likely a person is to commit another crime, and courts meet such scores at bail and sentencing. In State v. Loomis, the Wisconsin Supreme Court held that a sentencing court may consider one without violating due process, though not to decide whether a defendant is incarcerated or for how long. The report that carries the score must warn the judge of its limits, among them that the company has kept secret how the tool weighs its factors.
Facial Recognition as a Lead
A face search compares a probe photo with a database of known photos and returns a list of candidates, each with a similarity score that measures resemblance. Florida policies such as St. Petersburg’s treat a result as a lead that needs corroboration, never by itself as probable cause to arrest. What the defense must be told is unsettled, and New Jersey’s highest court now requires, case by case, basic disclosure of the tool and how police used it.
Third-Party Doctrine
The rule that information a person hands to a business, like bank records or dialed phone numbers, carries no reasonable expectation of privacy, so the government’s collecting it is not a search. Carpenter v. United States refused to extend the rule to a phone’s cell-site location records, and in 2026 Chatrie v. United States held that obtaining a person’s Google Location History is a search too. Courts have not settled where the line falls for other records of movement, such as license-plate camera logs.
Public Records Request
Florida’s chapter 119 opens the records of state and local agencies to inspection and copying by any person, whatever the reason, unless a statute makes a record exempt. A request need not be in writing, and the custodian must acknowledge it promptly, cite the statute behind anything withheld, and charge only the fees the statute allows. Exemptions cover plate-camera data that identifies a person and information revealing surveillance techniques, so a response may come back with parts blacked out.
Resources
- AI and the Workplace, Panel 2 of “AI and Marginalized Groups,” Duke University School of Law (Oct. 7, 2024) (opens in a new tab) (The video under the title, from 27:20 to about 42:40. Two law professors, Pauline Kim and Ifeoma Ajunwa, and Paul Grantham of Duke’s human resources department discuss who answers when a hiring tool discriminates. It was recorded in September 2024, so two points have moved. The EEOC and the FTC, which the panel treats as enforcers, have since walked away from disparate impact. And when Professor Kim says the EEOC can reach employers but not vendors, a federal court had already let a private suit reach a vendor as the employers’ agent, two months before the panel met.)
- Griggs v. Duke Power Co., 401 U.S. 424 (1971) (opens in a new tab) (Read pages 429 to 436. The Court held that Title VII reaches practices “fair in form, but discriminatory in operation,” and that the employer must show a requirement’s relation to the job.)
- 42 U.S.C. § 2000e-2, Cornell LII (opens in a new tab) (Subsection (k), added in 1991, writes the burden of proof for a disparate-impact claim into Title VII.)
- 29 C.F.R. § 1607.4, eCFR (opens in a new tab) (Subsection (D) is the four-fifths rule you apply to your tally. It belongs to the Uniform Guidelines of 1978, which the federal agencies that wrote them have moved to withdraw, so treat it as a way to measure a gap, not a test of liability.)
- Mobley v. Workday, Inc., 740 F. Supp. 3d 796 (N.D. Cal. 2024) (opens in a new tab) (Pages 5 to 11 accept the theory that a screening vendor can answer as the employers’ agent. Pages 13 to 16 set out what a disparate-impact claim must allege: a specific practice, a disparity, and causation.)
- Villarreal v. R.J. Reynolds Tobacco Co., 839 F.3d 958 (11th Cir. 2016) (en banc) (opens in a new tab) (Read the first two pages. In the federal circuit that includes Florida, a job applicant cannot bring an ADEA disparate-impact claim, because the provision protects employees; an applicant keeps a claim for disparate treatment.)
- Section 760.10, Florida Statutes (opens in a new tab) (The Florida Civil Rights Act. Compare subsection (1)(b), which names “applicants for employment,” with the ADEA provision Villarreal read.)
- Florida State University v. Sondel, 685 So. 2d 923 (Fla. 1st DCA 1996) (opens in a new tab) (A Florida court reads the Florida Civil Rights Act by federal Title VII and ADEA case law and sets out what a disparate-impact claim in hiring must show.)
- SB 26-189, Colorado General Assembly (signed act) (opens in a new tab) (What one state now requires when software helps make a consequential decision, employment included, on or after January 1, 2027: notice before it is used, an explanation after an adverse decision, and human review where commercially reasonable.)
- State v. Loomis, 881 N.W.2d 749 (Wis. 2016) (opens in a new tab) (Paragraphs 98 to 100 set the limits on using a risk score at sentencing and list the warnings the report must carry.)
- First Amended Complaint, Dillon v. City of Jacksonville Beach, No. 2:26-cv-01936 (M.D. Fla. July 13, 2026), ECF No. 35 (opens in a new tab) (Read paragraphs 15 to 23 on how a face search works, paragraphs 83 to 97 on the search in this case, and the three counts at paragraphs 251 to 277. Every paragraph is an allegation. The case now proceeds in Jacksonville as No. 3:26-cv-01930.)
- General Order II-47, Facial Recognition Software, St. Petersburg Police Department (opens in a new tab) (A Florida police department’s published policy on face searches through the Pinellas County Sheriff’s network, and a model of the kind of record your request asks for.)
- Lynch v. State, 260 So. 3d 1166 (Fla. 1st DCA 2018) (opens in a new tab) (A Florida defendant identified through a face search wanted the other candidates’ photos. The court held he could not show they would have changed the result, at 1169 to 1170.)
- State v. Miles, No. A-41-24 (N.J. June 24, 2026) (opens in a new tab) (New Jersey’s highest court, unanimous: the defense is entitled to discovery identifying the face-search tools police used and how they used them, decided case by case. Read the syllabus.)
- Chatrie v. United States, No. 25-112 (U.S. June 29, 2026) (opens in a new tab) (Read the syllabus, pages 1 to 5. Acquiring a phone’s location history from Google is a Fourth Amendment search, and Carpenter’s refusal to apply the third-party doctrine carries over.)
- Section 119.07, Florida Statutes (opens in a new tab) (The rules for your request: subsection (1) on inspection, acknowledgment, and exemptions, and subsection (4) on fees.)
- Section 316.0777, Florida Statutes (opens in a new tab) (Plate-camera records. Subsection (3) makes images and data that identify a person confidential and exempt; it says nothing about policies or contracts.)
- Government-in-the-Sunshine Manual, Office of the Attorney General of Florida (opens in a new tab) (The state’s guide to the public records law, including how requests work and what an agency may charge.)
- Request a Public Record, Volusia Sheriff’s Office (opens in a new tab) (Where a request to the Sheriff’s Office goes, through its online portal.)
- Request a Report, City of Daytona Beach (opens in a new tab) (Where a request to the Daytona Beach Police Department goes, through the city’s online portal.)
What to Do
A machine that decides can be checked in two ways: measure what it does, or make the people who use it show you their rules. Choose one. The screener test runs a stand-in for a retailer’s résumé screener through a controlled comparison and ends in a short memo. The records request asks a Volusia County police agency for its rules on face searches and plate cameras and ends in a short note. Either way, your post says plainly what your evidence shows and what it cannot.
Part 1: Read, Watch, and Choose
Start with the background reading. Then watch the video under the title from 27:20 to about 42:40, where two law professors and a university’s human resources officer argue over who answers when a hiring tool discriminates. Choose your path and read its sources in Resources. For the screener test, read Griggs at pages 429 to 436, the four-fifths rule in 29 C.F.R. § 1607.4(D), the Mobley order at pages 5 to 16, and the first two pages of Villarreal. For the records request, read the paragraphs of the Dillon complaint that Resources lists, the syllabus of Chatrie, St. Petersburg’s policy, and section 119.07, Florida Statutes. Do Part 2 or Part 3, not both.
Part 2: The Screener Test
Beacon’s screener. Beacon Pet Supply, the Daytona Beach retailer with about 60 employees, is hiring a warehouse operations manager for its Daytona Beach warehouse. It licenses a résumé screener that reads every application and ranks the applicants for interviews, and the vendor will not say how the ranking works. Beacon’s owner wants a test before the next round of hiring: a chatbot given the same job, the same instruction to rank, and résumés that differ in one detail at a time.
Run the Experiment Designer in a fresh chat. It builds four base résumés with equal qualifications, each for an applicant near 28. Then it makes three variants of each that change one thing: the name, the graduation year (to one that puts the applicant near 55), or a two-year gap in employment. Check each variant against its base before you go on, because a variant that changes two things spoils the comparison. Paste the screening prompt and the twelve variants into a new chat of one chatbot and record the five it picks. Run it again in another new chat with the résumés in reverse order, because the order of a list can sway a model’s choices.
Build a tally with a row for each of the twelve résumés and a column for each run, marking each time a résumé made the top five. Then compare groups, one change at a time. For the name, the four renamed résumés are one group and the other eight are the comparison; age and the gap work the same way. A group’s selection rate is its picks divided by its chances, and four résumés over two runs give eight chances. Divide the lower rate by the higher and mark every ratio under 0.80.
Write a memo of about 250 words to Beacon’s owner. It has four parts. The numbers give each group’s selection rate and each ratio, with the chances counted. The law names, for any ratio under 0.80, the statute and theory that would reach it for a job applicant in Florida, and whether the vendor could be sued along with Beacon. What is missing says what a plaintiff would still need that your test cannot supply. The change recommends one thing Beacon should do and says plainly what your test shows and what it cannot. If no ratio falls under 0.80, the memo says so and explains what that result does and does not clear. Score the memo one point for each part it gets right. Then run Compliance Counsel in a fresh chat with your tally and memo, revise once, and score the memo again.
Part 3: The Records Request
Choose the Volusia Sheriff’s Office or the Daytona Beach Police Department, whichever fewer classmates have written to so far. Write a request under chapter 119, Florida Statutes, for four kinds of records. Ask for the agency’s written policies on face searches, its written policies on license-plate cameras, and its contracts, agreements, or memoranda of understanding with the companies or agencies that supply either one. Add one more record you choose that would show how the tools are used, such as training materials or the reports the Pinellas County Sheriff’s Office sends agencies on their face searches. An agency owes you only records it already has, so ask for documents, not answers to questions. Describe each record so a clerk could find it without guessing. Ask for electronic copies, for an estimate before any charge, and for the statute behind anything withheld or blacked out. You need not say why you want the records.
Then write a note of about 250 words on what Chatrie and Dillon mean for what comes back. Score your request and note one point for each of four parts. The records part asks whether the request names each record so a clerk could find it and cites chapter 119. The terms part asks whether it requests electronic copies, an estimate, and the statute behind anything withheld. The Chatrie part asks whether the note says what the plate-camera policy should say about searching stored camera records, and why. The Dillon part asks whether the note says what the face-search policy should say about treating a result as a lead and passing on its limits. It also asks whether the note states plainly what the records can and cannot show. Run the Records Request Reviewer in a fresh chat with the request and the note, revise once, and score them again. Then send the request through the agency’s portal in Resources.
Part 4: Post Your Work
Create a single new discussion thread. For the screener test, post your tally with its rates and ratios, the revised memo, both scores, and a sentence on what Compliance Counsel led you to change. For the records request, post the request as you sent it, with the agency and the date and without your email address or phone number. Add the revised note, both scores, and a sentence on what the Records Request Reviewer led you to change. If the agency has answered by the time you post, add what came back. Before you post, check your work against its four parts.
Part 5: Share Chat Link
Include one AI chat link with a 1–2 sentence explanation of what the conversation shows and why you chose to share it.
Suggested AI Prompts
Use these prompts as a starting point, then adjust them to fit your goal. Strong prompting develops through trial, revision, and testing. It’s a foundational skill that grows into more advanced AI work such as context engineering and agent-based workflows.
Experiment Designer
Act as an industrial-organizational psychologist who designs adverse impact studies. Help me build a controlled experiment to test whether an AI screening prompt treats applicants differently by race, sex, age, or employment gap. Create four base résumés for a warehouse operations manager with equivalent education, experience, and skills, each with a graduation year that implies an age near 28. Then generate three variants of each that change only one thing: the applicant’s name (choose names commonly associated with different races and sexes in published audit studies), the graduation year (moving the implied age from near 28 to near 55), or a two-year gap in employment. Keep everything else identical. Then write a screening prompt that asks a model to rank the top five for interview with reasons, and a tally sheet for recording outcomes by variant across two runs. Explain how to apply the four-fifths rule from 29 C.F.R. Part 1607 to the results and what a result from only two runs can and cannot show.
Changing one detail at a time is controlled variation, the method of the audit studies that test hiring for bias: if only the graduation year differs, a gap in outcomes points at age. Asking the model to say what two runs cannot show builds the limit into the design before a result tempts you to claim too much.
Compliance Counsel
Act as outside employment counsel reviewing a junior colleague’s memo to a Florida employer. The employer, a Daytona Beach retailer with about 60 employees, uses a licensed AI résumé screener. I will paste the tally from a bias test we ran on a stand-in for the screener, and my memo of about 250 words. The memo should do four things: report each group’s selection rate and four-fifths ratio correctly; name the statute and theory that would reach any ratio under 0.80 for a job applicant in Florida, and whether the vendor could be sued as well; say what a plaintiff would still need that the test cannot supply; and recommend one change while stating plainly what the test shows and cannot show. First check my arithmetic against the tally. Then, for each of the four, tell me whether the memo does it, quote the sentence that falls short, and ask one question that would lead me to the fix. Read the memo once as the applicant’s lawyer and name the weakest point in the case as I stated it. Read it once as the employer’s counsel and name the claim you would push back on hardest. Do not rewrite the memo or supply missing text. If your view depends on a case or rule you cannot name precisely, say so instead of guessing, so I can check it. Wait for me to paste the tally and the memo.
Reading the memo from both sides is a form of red-teaming: the applicant’s lawyer finds where the case is thin, and the employer’s counsel finds where the advice reaches too far. Checking the arithmetic first, and admitting when it is unsure of the law, keeps the model’s critique on ground you can verify.
Records Request Reviewer
Act as the records custodian at a Florida sheriff’s office who has just received the public records request I will paste, and then as a supervisor reading my note about it. I will also paste the text of sections 119.07(1) and 119.07(4), Florida Statutes. First, as the custodian, go through the request item by item. Say whether you could find each record as described or what you would have to ask me, and which records might be exempt in whole or in part. Cite only statute text I pasted, and where an answer depends on a statute I did not paste, name that statute and say you are not quoting it. Then read my note and ask me one question about each of its two sources, Chatrie v. United States and the Dillon complaint, that tests whether I tied it to a specific record. Do not rewrite the request or the note. Wait for me to paste the statute text, the request, and the note.
Casting the model as the clerk who has to fill the request is role assignment, and it shows you how your words read from the other side of the counter. Holding it to the statute text you pasted is source grounding, which keeps a confident guess about an exemption from passing as law.