Who Decides When a Machine Decides, and Who Watches?

A decision wearing a number

A score is a decision wearing a number. A hiring tool rates an applicant “low.” A face search comes back as a “93% match.” A sentencing report places a defendant in a high-risk group. Each looks like a measurement, and each decides something about a person. Someone chose to build the tool, someone chose to use it, and someone read the number and acted on it. So the law has two questions: which of them answers for the result, and how can anyone outside tell whether the result was fair?

This reading argues that the law has two good ways in. The first is to count what a machine does without asking what anyone meant. The second is to make the people who use a machine show their rules and their records. Those are the two paths in the activity: a controlled test of a stand-in for a résumé screener, or a public records request to a police agency. The argument runs in four steps: the count, the vendor, the match, and the records.

Counting what no one intended

Start in 1971, long before software. Duke Power Company required a high school diploma for work in its better-paid departments. It later let workers without one transfer if they passed two general aptitude tests. Neither test was built to measure the ability to do any job at the plant. White employees hired before the rules kept doing those jobs well. In North Carolina at the time, 34 percent of white men had finished high school, against 12 percent of Black men. In one EEOC decision, a battery of tests that included the same two passed 58 percent of white test takers and 6 percent of Black ones. In Griggs v. Duke Power Co. (opens in a new tab), the Supreme Court held that Title VII reaches practices “fair in form, but discriminatory in operation.” An employer’s good intentions, the Court added, do not save a requirement that screens out one group and has nothing to do with the job.

That is disparate impact. The plaintiff does not prove that anyone meant harm. The plaintiff names a specific practice and shows the gap it produces, and then the employer must show that the practice is job related and consistent with business necessity. Congress wrote that burden into Title VII in 1991, in subsection (k) of section 2000e-2 (opens in a new tab). An algorithm suits this theory unusually well. It has no intent to prove, but it leaves results that can be counted.

The count needs a yardstick, and the usual one is the four-fifths rule. Suppose a screen picks the older applicants 2 times in 8 chances, a rate of 25 percent, and everyone else 8 times in 16, a rate of 50 percent. Divide the lower rate by the higher and you get 0.50, far under 0.80, so the rule flags it. In other words, the older applicants got half the looks. But with only eight chances, one more pick would raise their rate by twelve and a half points. That is why the Supreme Court, in Ricci v. DeStefano (opens in a new tab), repeated an earlier description of the rule as a “rule of thumb for the courts” (at 587). It treated a gap like this as only a threshold showing. A flagged ratio starts a case. It does not win one.

Age adds a Florida wrinkle. The Supreme Court recognized in 2005 that the Age Discrimination in Employment Act allows disparate-impact claims, though more narrowly than Title VII. But in Villarreal v. R.J. Reynolds Tobacco Co. (opens in a new tab), the Eleventh Circuit, whose decisions bind the federal courts in Florida, held that the Act’s disparate-impact provision protects employees and not applicants. A Florida applicant screened out because of age keeps a claim for disparate treatment, which requires showing that age drove the decision. The court noted that statistics can still help show it. Florida’s own Civil Rights Act words its parallel provision differently: section 760.10(1)(b) (opens in a new tab), Florida Statutes, names “applicants for employment.” Florida courts read that act by federal case law. What the difference in wording means for an applicant’s age claim is a question your memo can take up.

The vendor in the middle

Most employers do not build their screening software. They license it, and the vendor’s tool reads every application before a person sees any of them. Title VII, the ADEA, and the ADA make an employer answer for discrimination, and each defines “employer” to include the employer’s agent. In Mobley v. Workday, an applicant sued the vendor instead of the employers. The federal court in San Francisco held, at the pleading stage, that a company can answer as an agent when an employer hands it a traditional employer function (the order (opens in a new tab), pages 9 to 11). Rejecting applicants at the screening stage is one of those functions. Treating software differently from a human screener, the court wrote, “would potentially gut anti-discrimination laws in the modern era.”

The court drew a line too. Suppose an employer used spreadsheet software to sort applicants by birthdate and filtered out everyone over 40. The spreadsheet’s maker would not be an agent, because the spreadsheet takes no part in deciding whom to hire. The employer would still answer. The vendor as agent theory reaches the tool that decides, not every tool a decision passes through. The same order let the disparate-impact claims go forward and dismissed the claims of intentional discrimination, and the court later conditionally certified a nationwide collective of applicants over 40. It also read the ADEA to let applicants bring disparate-impact claims, the opposite of Villarreal. So where an applicant sues can decide whether the claim exists at all.

The federal agencies that once brought cases like this have stepped back. An executive order in April 2025 told agencies to deprioritize disparate-impact liability, and the EEOC took down its guidance on AI in hiring that year. That leaves private plaintiffs and the states. Colorado’s SB 26-189 (opens in a new tab) takes a different route from counting. The act reaches consequential decisions, hiring included, made on or after January 1, 2027. A business using covered software for one must give notice before using it and a plain explanation after an adverse decision. It must also offer a chance to correct wrong data and, where commercially reasonable, a human review.

A match is a lead

The police side of the argument turns on the same kind of number. A face search takes a probe photo, reduces the face to a set of measurements, and compares it with the photos in a database. It returns a list of candidates, each with a similarity score. The complaint in Dillon v. City of Jacksonville Beach (opens in a new tab) explains the catch: a similarity score measures how close two sets of measurements are. A 93 percent score does not mean a 93 percent chance that the candidate is the person in the photo (paragraph 20). That is a plaintiff’s account, but Florida police policies make the same point. St. Petersburg’s general order on face searches (opens in a new tab) calls a result a potential investigative lead and says it does not, by itself, give probable cause to arrest. That is facial recognition as a lead.

Dillon alleges what happens when a lead is treated as an answer. A Fort Myers man was arrested for a crime in Jacksonville Beach, a city he says he had never been to. The complaint says a Jacksonville sheriff’s sergeant ran security-camera photos through a face search, and the investigating officer’s notes record that the photos “revealed a 93% match” to him (paragraph 86). According to the complaint, the sergeant gave no warning that the result was only a lead and did not say how many other candidates the search returned (paragraphs 88 and 89). The warrant affidavit repeated the 93 percent figure. It left out a plate-camera search showing that his cars were not in the county around that time. The charges were dropped. His suit claims a Fourth Amendment violation against two officers and faults the policies of the city and the sheriff. None of it has been proved yet.

What must a defendant be told about a face search? So far the answer depends on the court and the stage of the case. In Lynch v. State (opens in a new tab), a Florida defendant identified with the help of a face search asked for the photos of the other candidates it returned. The First District held that he could not show those photos would have changed the result, because he could not show they looked like him (at 1169 to 1170). In 2026, New Jersey’s highest court went further in State v. Miles (opens in a new tab). The defense there was entitled to learn which face-search tools police used and how they used them, though the court decided the scope case by case.

The courtroom has its own scores. In State v. Loomis (opens in a new tab), a Wisconsin judge considered a commercial risk score at sentencing, and the defendant argued that due process forbade it. The Wisconsin Supreme Court allowed the score, with limits. It may not decide whether a defendant goes to prison or for how long, and the judge must explain the other reasons for the sentence (paragraphs 98 and 99). The report must also carry written warnings (paragraph 100). Among them: the company keeps secret how the tool weighs its factors, the score describes groups rather than the person, and studies have questioned whether it rates minority defendants as riskier. In other words, the court let the number in but made it arrive with a label saying what it cannot show. That is risk score and due process. It treats the score as a decision, which is this reading’s claim: a judge may weigh it but may not hand the sentence over to it.

The records that watch, and the records you can see

The second way to check a machine is to see its records and its rules, and two bodies of law decide who gets to. The Fourth Amendment governs what the government may collect about you. Under the third-party doctrine, information you hand to a business carries no reasonable expectation of privacy, so the government’s getting it from the business is not a search. Carpenter v. United States (opens in a new tab) refused to apply that rule to the location records a phone company keeps. Florida’s Supreme Court had gone part of the way four years earlier. In Tracey v. State (opens in a new tab), it held that tracking a phone’s location signals in real time was a search that required probable cause. In 2026, Chatrie v. United States (opens in a new tab) carried Carpenter to Google’s Location History, which records a phone’s position every couple of minutes. A user does not truly share that history, the Court reasoned, just by turning on a feature the phone keeps asking for, so acquiring it was a search.

A network of license-plate cameras builds a record of movement too, one car and one pole at a time. Whether searching its stored records is a search is a question the courts have not settled. That is one reason an agency’s written rules for searching them matter.

Chapter 119 governs what you may see about the government. Any person may inspect and copy an agency’s records, for any reason, unless a statute makes a record exempt. Section 119.07 (opens in a new tab) sets the rules for a public records request. Some exemptions reach surveillance. Images and data from a plate camera that identify a person are confidential under section 316.0777 (opens in a new tab), and information revealing surveillance techniques is exempt as well. Neither exemption names a policy manual or a contract, so an agency that withholds part of one must cite the statute that covers that part.

Both paths in the activity end the same way: with a plain statement of what your evidence shows and what it cannot. If you test the screener, you start with the Experiment Designer and a set of résumés that differ in one detail at a time. If you write the request, you start with the paragraphs of the Dillon complaint on how a face search works. Either way, you are asking the question behind every score. Who decided?