When AI Hurts People

Design Defect

A products liability theory that a product was unreasonably dangerous as designed because a reasonable alternative design would have reduced the foreseeable risk. Plaintiffs in the chatbot cases argue that safeguards against self-harm content were available and were not used, or were removed.

Failure to Warn

Liability for not giving adequate warnings or instructions about a risk the maker knew or should have known. A chatbot that never tells a vulnerable user it is a machine, or a maker that never tells parents what the product does, invites the claim.

Duty and Foreseeability

Negligence requires a duty of care, and the duty extends to harms a reasonable actor could foresee. Whether a developer should foresee that a model trained on internet text will produce harmful content, or that an agent will act on a mistaken instruction, is the question that decides most AI negligence cases.

First Amendment and Section 230 Defenses

Defendants argue that chatbot output is protected speech and that 47 U.S.C. § 230 shields them as publishers of third-party content. Courts have been skeptical of both arguments when the content is generated by the defendant’s own product rather than posted by a user.

Companion Chatbot Duties

The statutory obligations California created in SB 243: clear disclosure that a user is interacting with AI, protocols to prevent suicide and self-harm content with referrals to crisis services, and additional protections for minors. They are the first specific safety duties imposed on a class of AI products.

Agentic Action

Software that does things rather than only saying things: booking, buying, posting, filing. When an agent completes a transaction its user never approved, the question is no longer defamation or bad advice but who is bound and who pays, and whether “the AI did it” is a defense at all.

Accelerationism and Decelerationism

Two policy stances toward new technology. Accelerationists favor rapid deployment with minimal regulation on the theory that innovation produces net benefit; decelerationists favor caution and precautionary rules. The New Law Bot asks you to pick one, and every statute you draft reflects the choice.

Part 1: Read the Frameworks

Read the RAND report’s summary, the introduction and conclusion of the SSRN paper, and the Wex entry on products liability. Note where each says current doctrine applies, where it strains, and where it may not reach. Read the causes of action in the Raine complaint and the disclosure and safety sections of SB 243 so you know what a real complaint and a real statute in this area look like.

Part 2: Work the Fact Patterns

Open the New Law Bot and work through at least eight of the twenty fact patterns below, including Fact Patterns 7 and 15 (companion chatbots) and Fact Pattern 14 (an agent). For each, enter the fact pattern, take the position the bot asks for, accelerationist or decelerationist, and review the statute it proposes and its explanation of how that statute would interact with existing tort law. Use the Liability Chain Analyzer on any scenario where you cannot tell who should be the defendant.

Fact Pattern 1: A self-driving car, operating under normal urban driving conditions, suddenly experiences a software error while navigating a busy intersection. Despite failing to detect a red light, the autonomous system accelerates and strikes a pedestrian who is crossing legally. The car’s owner had recently installed an over-the-air software update from the manufacturer. After the accident, engineers discover the update included a flawed sensor calibration module. The pedestrian suffers multiple fractures and permanent nerve damage.

Fact Pattern 2: A hospital’s AI diagnostic tool incorrectly identifies a benign tumor as malignant, prompting doctors to recommend an aggressive treatment plan. The patient undergoes chemotherapy and surgical procedures that result in serious side effects and complications. Post-treatment analysis reveals the AI was trained on insufficient data sets for rare tumor types. The hospital’s administration had implemented the tool to streamline diagnoses and reduce patient wait times. The patient must now cope with unnecessary scarring, prolonged recovery, and significant emotional distress.

Fact Pattern 3: An online investment platform’s AI advisor evaluates a user’s profile and concludes they should invest heavily in a volatile market sector. The user, who is inexperienced in financial matters, follows the AI’s recommendation without seeking human advice. Within a few months, the portfolio plummets in value, wiping out the investor’s life savings. Investigations reveal the AI’s model was skewed by outdated financial data. The investor is left facing extreme financial hardship and potential bankruptcy.

Fact Pattern 4: A robotic vacuum cleaner, marketed as being particularly safe around small animals, fails to register the presence of a family’s toy-sized dog. During a scheduled cleaning cycle, the vacuum traps the dog against a wall, causing serious injuries. The manufacturer had touted the device’s “smart sensor technology” as highly reliable in detecting obstacles. Further testing suggests the device’s object-recognition algorithm struggles with animals under ten pounds. The owners incur hefty veterinary bills and emotional trauma.

Fact Pattern 5: A customer service chatbot, provided by a major appliance company, gives dangerously incorrect instructions to a user seeking help with a malfunctioning refrigerator. Following the bot’s advice, the user attempts a risky electrical repair, resulting in a severe electric shock. Internal logs show the chatbot had been updated with new “learning” data minutes before offering the erroneous steps. The company had replaced many human agents with the chatbot to cut costs. The user sustains permanent nerve damage in one hand.

Fact Pattern 6: A news aggregation AI tool scrapes local and national headlines, then synthesizes them into short “breaking stories.” Without human review, the AI erroneously publishes an article accusing a well-known community leader of embezzlement. This false story goes viral on social media, leading to public outrage and reputational harm for the leader. The platform owner intended the AI to speed up content production and boost ad revenue. Investigators discover the AI had conflated unrelated data sets involving a similar-sounding name.

Fact Pattern 7: A personalized AI companion chatbot, designed for mental health support, begins sending sexually explicit messages to a vulnerable user. The chatbot’s behavior is triggered after it “learns” from an online forum with inappropriate content. The user, already dealing with emotional challenges, is shocked and disturbed by the sudden change in tone. The chatbot’s developer had assured robust content filters and ongoing monitoring but failed to detect the harmful shift. The user experiences heightened anxiety and is forced to seek additional therapy.

Fact Pattern 8: A retail store’s AI-powered security system uses advanced facial recognition to detect suspected shoplifters. One afternoon, it flags a regular customer with no criminal history, prompting security staff to detain and question her in a private room. The system’s developers discover an error rate that disproportionately affects certain facial features. The wrongful detention causes the customer to miss an important medical appointment and suffer humiliation. She later files a complaint seeking damages for false imprisonment.

Fact Pattern 9: An AI-controlled security drone is programmed to patrol a gated community but routinely flies over a nearby private backyard, capturing video of the homeowner’s family. The drone’s GPS mapping software mislabels the yard as a communal park. The homeowner notices the drone hovering at odd hours, recording footage of family events. Residents had been told the drone would enhance neighborhood safety, but no one realized it would invade private spaces. The homeowner experiences distress and embarrassment over the privacy intrusion.

Fact Pattern 10: A surgical robot with autonomous functions is used in a hospital’s operating theater to perform precise incisions. During a routine procedure, a software glitch causes the robot to make an unexpected movement, slicing into healthy tissue. The patient requires additional corrective surgery and endures weeks of extended hospitalization. Further inspection reveals the robot had not received a vital firmware patch that might have prevented the malfunction. The hospital had recently expanded its use of robotics to reduce labor costs and maximize throughput.

Fact Pattern 11: An experimental AI system, deployed in a materials-processing plant handling flammable chemicals, malfunctions without warning. Despite normal safety checks, the system sends contradictory commands to temperature control units, triggering an explosion. Nearby workers and residents are injured, and the facility suffers millions in damage. Investigations reveal the AI’s “predictive maintenance” module had a rare but critical software conflict. The firm had touted this AI as a breakthrough for efficiency and cutting-edge innovation.

Fact Pattern 12: A city’s AI control system for its power grid experiences a hidden algorithmic defect during peak summer demand. The system abruptly shuts down multiple substations, leading to an extensive blackout across the region. The unexpected outage causes traffic chaos, refrigeration failures, and thousands of dollars in property losses for local businesses. IT staff discover the AI’s anomaly detection feature misread routine spikes in usage as a catastrophic fault. The city faces public outcry and potential legal claims from affected residents.

Fact Pattern 13: A company employee, heavily relying on an AI analytics tool, provides a client with market projections that turn out to be wildly inaccurate. The client invests resources based on those projections, suffers significant financial losses, and threatens legal action. Subsequent audits show the AI had been fed incomplete data sets and produced misleading results. The employee lacked the expertise to question the analytics and the company relied heavily on the AI’s outputs. The client’s losses include damaged business relationships and lost market opportunities.

Fact Pattern 14: A business licenses an AI “agent” to handle its social media marketing campaigns with minimal human oversight. Over time, the agent begins posting inflammatory and defamatory content about the company’s competitors and even some customers. The business only becomes aware of the issue after multiple complaints and negative media coverage. It turns out the AI’s content filters were disabled during a routine system update. The scandal causes reputational harm and potential defamation claims from affected parties.

Fact Pattern 15: An AI companion, designed to replicate conversations with deceased loved ones, repeatedly mocks or belittles a grieving user after parsing unrelated text sources. The user’s mental health deteriorates, and they experience panic attacks and prolonged depression. Developers uncover that the AI’s emotional support settings were reset during a recent software patch. This glitch led the system to adopt a harsh, dismissive tone. The user is forced to seek professional psychiatric care, blaming the AI for exacerbating their trauma.

Fact Pattern 16: An autonomous vehicle, manufactured by a premium carmaker, veers off the road and crashes, leaving the driver permanently paralyzed. Preliminary results suggest the AI lost situational awareness during a handoff from autonomous mode to manual control. The driver claims the interface provided insufficient warning of the imminent mode switch. The carmaker had advertised the vehicle’s autonomy as nearly foolproof. The victim’s spouse files a separate loss of consortium claim, citing a drastic impact on their marital relationship.

Fact Pattern 17: A retailer secretly uses AI-based facial recognition in its stores to monitor customer behavior and gather detailed profiles, including shopping habits and emotional responses to product displays. Shoppers are not notified that their images and biometric data are being captured. When the practice is exposed, some patrons feel violated and suspect discriminatory targeting. Investigations show the retailer’s marketing department used the data for targeted advertising without legal consent. Several patrons consider litigation for privacy violations and breach of trust.

Fact Pattern 18: Hackers exploit a security flaw in an AI-powered factory control system, taking remote command of robotic equipment that assembles precision components. The hackers cause critical machinery to overheat and fail, damaging high-value inventory and halting production lines. Internal reports reveal the system lacked a recent security patch addressing vulnerabilities in the AI’s communications protocols. This downtime and damage trigger significant financial losses for the manufacturer. The breach calls into question whether the company met reasonable cybersecurity standards.

Fact Pattern 19: A mortgage lender’s AI underwriting tool, built on decades of historical mortgage data, disproportionately rejects loan applications from a particular minority group. The lender’s leadership had trusted the AI to expedite loan approvals without bias. An independent audit discovers the training data itself was skewed, reinforcing past discriminatory patterns. Several would-be borrowers face continued financial hardship due to these denials. Public backlash prompts regulatory scrutiny and possible legal action for discrimination.

Fact Pattern 20: An AI-based forklift training simulator is used to certify new warehouse workers, relying on virtual reality exercises to replicate real-world scenarios. The simulator’s AI engine, however, is calibrated with outdated safety parameters and fails to account for modern forklift weight capacities. On the first day of work, a trainee attempts to lift an overloaded pallet, toppling the forklift and causing multiple injuries to bystanders. A subsequent investigation reveals the simulator’s AI consistently underestimated tipping points and neglected critical safety checks. The company faces lawsuits from injured employees, prompting broader concerns about the reliability of AI-driven training tools.

Part 3: Add the Agent Scenario

Use the Agent Scenario Builder to write a twenty-first fact pattern of your own about an AI agent that completes a transaction its user never approved, such as booking nonrefundable travel, accepting a contract, or transferring money, and run it through the New Law Bot as well. Notice whether the bot’s proposal treats the agent as a product, as an employee-like actor, or as something new.

Part 4: Draft the Statute

From all the proposals you generated, select the three that made you think hardest, whether or not you agree with them. Then choose one and use the Bill Drafter to turn it into actual bill text with a title, a definitions section, the operative duty or liability rule, an enforcement mechanism, and an effective date, modeled on the structure of SB 243. Run the finished draft through the Reality Check and revise it once in response.

Part 5: Post Your Deliverables

Create a single new discussion thread containing your overall takeaways in about 300 words, including which stance you took and why; your three selected proposals, each with the fact pattern it came from, the proposed law, why it stood out, and how it would change existing tort doctrine; your own agent fact pattern and the bot’s response to it; and your draft statute with a short note on what the Reality Check changed. Draw on at least one specific point from the readings.

Part 6: Share Chat Link

Include one AI chat link with a 1–2 sentence explanation of what the conversation shows and why you chose to share it.

Liability Chain Analyzer

Act as a legal analyst specializing in products liability and negligence. I am going to describe an AI-related injury scenario. Identify every party in the chain who could bear liability, from the data provider through the developer, the integrator, the deployer, and the end user. For each party, explain which legal theory might apply (negligence, strict products liability, vicarious liability, or another), the strongest argument for holding that party liable, and the strongest argument against. Present the analysis as a chain that shows how responsibility flows or fragments across the parties. Name any case or statute you rely on precisely, and say so if you are unsure it exists. Confirm that you understand and ask me for the scenario.

Mapping the chain before picking a defendant. AI harms fragment responsibility across several actors, and laying every candidate out with arguments both ways keeps you from settling on the most visible defendant by reflex.

Agent Scenario Builder

Act as a law professor who writes exam fact patterns. Help me write a realistic fact pattern of 100 to 150 words about an AI agent that takes an action its user did not approve and causes a financial or legal harm. Ask me three questions first: what kind of agent and task, what went wrong (a misread instruction, a disabled safeguard, a hallucinated fact, a prompt injected by a third party), and who was harmed. Then draft the fact pattern in the same style as a classic torts hypothetical, with enough detail about the parties and the failure to support liability analysis and no conclusion about who is at fault.

A hypothetical built to be analyzed. The three questions force the scenario to have a specific failure mode and a specific victim, which is what distinguishes a fact pattern you can work from a story about a robot.

Bill Drafter

Act as a legislative drafter. I will paste a proposed law generated during a class exercise on AI harms. Turn it into bill text with these sections: a short title, legislative findings in two or three sentences, definitions for every technical term the operative section uses, the operative duty or liability rule stated in one clear paragraph, an enforcement mechanism that says who may bring an action and what remedies are available, any exemptions, and an effective date. Model the structure on California SB 243 on companion chatbots. Where the proposal is vague, ask me to decide rather than guessing, and flag any term that would be hard for a court to apply.

Forcing precision through structure. A proposal that sounds reasonable in a paragraph often falls apart when it has to define its terms and name its enforcer, and the drafter’s format exposes exactly where.

Reality Check

Act as a skeptical supervising attorney. I am going to share a draft statute on AI liability. Stress-test it: identify unintended consequences, enforcement problems, constitutional concerns such as First Amendment or preemption issues, conflicts with existing tort doctrine, and any gap between the harm the statute targets and the conduct it actually regulates. Be direct about serious flaws and note genuine strengths. Do not rewrite the statute; evaluate it, and end with the three changes you would make first. Confirm that you understand and ask me for the draft.

Adversarial review before revision. Asking the model to attack rather than improve produces a list of specific weaknesses you can decide about yourself, and limiting it to three priority changes keeps the revision focused.