Building a product with a conversational AI component used to be a straightforward engineering call. It isn’t anymore. Garcia v. Character Technologies established that conversational AI developers can owe a duty of care to users, and courts are treating AI-generated harm claims as product liability issues — not platform content moderation questions.
Your exposure depends on what your product does, how it is designed, and what engineering controls are in place. Having the right disclaimers in your terms of service is not enough. This article covers five areas: a liability risk matrix across four product categories; an original B2B liability chain analysis; a multi-state compliance engineering map; a 48–72 hour incident response playbook; and board-level governance framing from the Federated Hermes EOS Q1 2026 report. It does not re-litigate Section 230 theory — that’s covered in the product liability framework underlying this risk matrix — or re-specify engineering architecture, which is covered in the engineering controls that reduce legal exposure.
For the broader legal and design landscape, see the AI chatbot safety: the full reckoning pillar.
Where does your product sit on the liability risk matrix?
Not all chatbots carry the same risk profile. The exposure factors that drive legal liability — emotionally responsive persona, session design, user vulnerability, dependency mechanics, age verification, crisis escalation — interact differently depending on what you’ve built.
Garcia v. Character Technologies affirmed that AI chatbots are “products” for product liability purposes. That matters because strict product liability — no proof of negligence required — is the framework the EU’s Product Liability Directive applies from December 2026, and increasingly what US courts are working toward.
Use the matrix below as a triage tool.
| Product Category | RLHF / Emotionally Responsive Persona | Prolonged Session Design (>20 min) | Vulnerable User Population | Emotional Engagement / Dependency Mechanic | Age Verification Absent | Crisis Escalation Absent |
|---|---|---|---|---|---|---|
| General assistant / productivity AI | Low — no persona attachment | Low — task-oriented; bounded sessions | Low — general population assumed | Low — no relationship framing | Elevated — minor access possible | Elevated — duty of care if crisis signal detected |
| Mental health support chatbot | High — persona trust increases harm exposure | High — session depth correlates with reliance | Critical — vulnerable population by definition | High — emotional engagement is core feature | Critical — failure-to-warn doctrine applies | Critical — crisis escalation is the primary legal obligation |
| Companion / social AI | Critical — relationship persona is the product | Critical — prolonged engagement is the design goal | High — isolation-prone users self-select | Critical — dependency mechanics drive retention | High — minor access likely without verification | Critical — no crisis escalation = exposure mechanism |
| Youth-facing product (any category) | High — duty of care elevated for minors | High — minors more susceptible to prolonged engagement effects | Critical — minor status alone elevates all factors | High — dependency mechanics prohibited under WA HB 2225 | Critical — failure-to-warn doctrine; no learned intermediary defence | Critical — California SB 243 and NY companion law impose specific obligations |
The two highest-risk profiles are companion / social AI with emotional engagement mechanics and no crisis escalation, and youth-facing products of any category. With companion AI, the design intent is also the legal exposure mechanism. Youth-facing products get an automatic baseline lift — minor status alone increases exposure across every factor.
For most SaaS companies, the interesting case is the general assistant category. A general assistant with no persona customisation sits in the Low column for most factors. The moment you write a system prompt that adds relationship-building language or persistent persona framing, the product moves toward the companion row regardless of what you call it. The product liability framework underlying this risk matrix explains the doctrinal basis in detail.
Who is liable when a SaaS company integrates OpenAI or Anthropic and a user is harmed?
No court has directly addressed the three-party liability structure for SaaS companies using foundation model APIs — what follows is analytical reasoning from first principles, not settled law.
Three parties: the user, the SaaS deployer (your company), and the model provider (OpenAI, Anthropic). The SaaS deployer is the primary liability bearer. There are three reasons for this.
First, you have the direct user relationship — courts treat chatbot output as company speech, no different from a printed brochure or a call-centre agent (Moffatt v. Air Canada, 2024). Second, you author the system prompt — the design decision that most determines your exposure. Courts in discovery will focus on what you configured, not model defaults. Third, deployment design decisions are yours — whether to add a persona, implement safety classifiers, include crisis escalation. The Ada Lovelace Institute’s “Risky Business” report documents how accepting a provider’s terms adds compliance obligations; it does not transfer liability.
Model provider exposure is limited, but not zero. If a base model has known failure modes that were not addressed, the provider may bear liability for those upstream choices. Walters v. OpenAI (Ga. Super. Ct., May 2025) shows how providers protect themselves — OpenAI’s documentation and disclaimers defeated the defamation claim.
Under the EU AI Act, the deployer/provider distinction is explicit: OpenAI and Anthropic are “providers”; your SaaS company is a “deployer.” LEXR confirms that standard prompt engineering and RAG implementations typically do not cross the Article 25 threshold that would reclassify a deployer as a provider.
The practical upshot: a SaaS company that writes an emotionally responsive system prompt, targets a vulnerable user population, and deploys with no crisis escalation carries the highest liability profile regardless of which foundation model it uses.
What engineering work does each active jurisdiction require right now?
Multi-state compliance is an engineering problem. The regulatory text needs to become tickets in your backlog.
| Jurisdiction / Law | Status | Engineering Obligation | Deadline / Fine |
|---|---|---|---|
| California SB 243 (Companion Chatbot Law) | Active | Crisis signal detection with referral pathway; session break notifications; minor-specific content controls | January 1, 2026 (active now); $1,000/violation + private right of action |
| Washington HB 2225 (Chatbot Disclosure Act) | Active — effective January 1, 2027 | Design audit + removal of prohibited engagement patterns; adversarial testing to confirm removal; periodic disclosures for minor users | January 1, 2027 |
| New York Companion Models Law | Active (November 2025) | Safety protocols for self-harm detection; regular AI disclosure; crisis service referral trigger | Active now |
| Oregon SB 1546 | Active — effective January 1, 2027 | AI disclosure; self-harm detection + crisis referral; minor-specific safeguards; annual filings | January 1, 2027; $1,000/violation |
| Tennessee SB 1580 | Active (July 1, 2026) | Remove / block any AI system presenting itself as a licensed mental health professional | July 1, 2026 |
| Colorado AI Act (SB 24-205) | Active — major provisions June 30, 2026 | Duty of care documentation for high-risk AI deployers; impact assessment records | June 30, 2026 |
| EU AI Act Article 50 | Active — August 2, 2026 | Disclosure banner or first-message disclosure for any chatbot interaction with EU users; AI-generated content must be machine-detectable | August 2, 2026; €15M / 3% turnover for violations |
| EU AI Act Article 5 (Prohibited Practices) | Active — August 2, 2026 | Audit and remove any prohibited manipulation or exploitation practices; subliminal techniques, exploitation of vulnerabilities | August 2, 2026; €35M / 7% global turnover |
| EU Product Liability Directive | Active — December 9, 2026 | Documentation of design choices is a legal necessity from this date — strict liability for defective AI as a product; unlimited damages | December 9, 2026 |
| GUARD Act (US Senate, pending) | Pending | Plan age-gate architecture for companion AI access controls — under-18 access restrictions if passed | TBC on passage |
| TRUMP AMERICA AI Act (pending) | Pending | Federal product liability standard for AI (Title VII); preemption of state patchwork if passed — see final section | TBC on passage |
The EU AI Act has extraterritorial reach — if your product is accessible to EU users, it applies regardless of where you’re incorporated. Selling to German customers means broad liability exclusions in your ToS are unenforceable under German AGB law. US-style disclaimers don’t hold.
For the state-level legislative detail, see the state-by-state compliance obligations.
What should a CTO audit — and in what order — to assess chatbot liability exposure?
Four dimensions, in order of urgency: design, engineering, compliance, documentation. Start with design — highest discovery risk. Finish with documentation — highest legal leverage.
Dimension 1 — Design Audit
Map your design choices against Washington HB 2225’s prohibited categories. Any match is a remediation item: excessive praise mechanics, feigned-distress responses, prompts that encourage user isolation, dependency-creating mechanics that reward extended engagement. Check persona boundary enforcement — can the chatbot be jailbroken into roleplay that removes safety constraints? For the full design pattern analysis, see the companion AI design choices that create legal and safety risk.
Dimension 2 — Engineering Audit
Audit your safety classifier coverage. Does it detect crisis signals — suicidal ideation, self-harm references, abuse disclosures? Does a crisis trigger produce an actual referral pathway (988 in the US) or just a generic “seek professional help” response? Check your adversarial testing records — The Future Society documents sandbagging: models can behave differently in evaluation than in live deployment. Test in realistic conditions. For the full engineering architecture, see the engineering controls that reduce legal exposure.
Dimension 3 — Compliance Audit
Use the multi-state compliance engineering map above as the audit checklist. Confirm which jurisdictions apply based on your user geography. Schedule engineering sprints for pending obligations before they activate — Washington HB 2225 (January 2027) has enough lead time to address now.
Dimension 4 — Documentation Audit
Courts in discovery look for: RLHF training choice records, guardrail configuration history, safety classifier test results, escalation flow logs, prior user harm reports, and known-issues records. The contrast between Garcia v. Character Technologies — where inadequate documentation was central to the plaintiff’s case surviving the motion to dismiss — and Walters v. OpenAI, where documentation defeated the claim, is the practical illustration.
Absence of testing records is worse than imperfect records — it implies no safety diligence was exercised. Quinn Emanuel’s April 2026 Business Litigation Report: “architectural and recordkeeping choices made today may shape liability exposure tomorrow.” Document proactively, under legal privilege where possible. Briana Vecchione of Data & Society frames self-auditing as companies “grading their own homework” — an independent third-party audit programme is a due-diligence signal in potential litigation.
What do you do in the first 48 hours after a user harm event is reported?
No single authoritative source provides a structured AI incident response playbook integrating legal and engineering obligations. What follows is original synthesis from legal best practices and engineering incident response methodology — treat it as a starting framework, not settled legal advice.
Immediate Phase (0–2 Hours)
Contact legal counsel before anything else. Before any public-facing action, before contacting the user, before making internal announcements — get counsel involved. This preserves legal privilege over subsequent communications.
Implement a legal hold. Secure all potentially relevant artefacts immediately: conversation logs (full, unredacted), the system prompt version active at the time of the incident, safety classifier outputs for the session, escalation flow logs, and prior similar user reports. Spoliation of evidence — failure to preserve relevant materials — is an independent liability that can be more damaging than the underlying incident.
Do not: make public statements, contact the user directly, delete or modify any system configuration, or allow automated log cleanup policies to run. All communication routes through counsel.
24-Hour Phase
Conduct a regulatory notification assessment. Different jurisdictions have different notification timeframes; LEXR identifies this as one of the most jurisdiction-variable obligations. Your standard incident communication processes probably don’t cover it. Begin an internal incident log — a written chronological record of all actions and decisions from the moment the incident was reported. This log is a legal asset.
48–72 Hour Phase
Engineering review. Reconstruct the conversation path. Did the safety classifier trigger — or fail to? Include a production-versus-test discrepancy check for sandbagging. Document findings under legal privilege.
Initiate a third-party safety review. A self-assessment after a harm event is the least defensible posture.
Board / investor notification threshold assessment. The Federated Hermes EOS Q1 2026 report frames chatbot safety as a material financial risk — this is a risk management question, not a communications decision.
How do you frame AI chatbot liability as a material financial risk for your board?
The Federated Hermes EOS Q1 2026 Public Engagement Report (Ross Teverson and Navishka Pandit) makes the investor position explicit: chatbot safety is a material financial issue. Federated Hermes is one of the world’s largest institutional asset managers. This is current scrutiny, not future-facing commentary.
The financial materiality has four dimensions: regulatory exposure (EU AI Act fines up to €35 million or 7% of global annual revenue); litigation scale (Garcia v. Character Technologies is a multi-plaintiff suit alleging bodily injury from AI chatbot interaction); reputational damage (harm narratives involving minors hit brand value faster than legal proceedings resolve); and market access risk (EU AI Act compliance is a market entry condition from August 2026).
Character.AI is the cautionary case: an under-18 ban imposed after lawsuits rather than before. Reactive governance is expensive. Apple and Kuaishou are the positive examples: safety-by-design embedded at the platform level before incidents occur.
Your board needs to answer five questions:
- What is the company’s risk tolerance for companion or emotionally engaging AI features — and has that been documented?
- Does the company carry adequate liability insurance for AI user harm claims?
- What is the incident response capability — can we execute the 48–72 hour playbook?
- Is there an independent third-party audit programme in place or planned?
- What is the documentation strategy — are engineering design decisions recorded in a legally defensible format?
These are risk management obligations, not engineering tasks. For the broader liability and design landscape, see AI chatbot safety: the full reckoning.
What would the TRUMP AMERICA AI Act change — and what should you do now?
The TRUMP AMERICA AI Act, sponsored by Senator Marsha Blackburn (R-TN), proposes a federal product liability standard for AI under Title VII. If passed, it would preempt the state-by-state patchwork with a single federal framework. The Act has not passed. Its timeline and final form are unknown.
Do not pause multi-state compliance work. California SB 243 is already enforceable. Washington HB 2225 becomes effective January 2027. Waiting for federal preemption while ignoring active state obligations creates exposure now.
Build documentation and audit frameworks to a demanding standard regardless. Testing artefacts, monitoring signals, and change histories remain central in discovery whether the law is state or federal. If federal preemption passes, the multi-state compliance work converts directly into federal compliance evidence. If it doesn’t, the state compliance work covers active regulatory risk. There is no scenario in which the documentation investment is wasted.
Design to the more demanding standard. Where a state requirement is stricter — Washington’s design pattern prohibitions being the clear example — maintain the state-level control even post-preemption. Over-compliance is cheaper than re-engineering.
For the state-level compliance obligations this section builds on, see the state-by-state compliance obligations.
Frequently asked questions
What is AI chatbot product liability — and how is it different from platform immunity under Section 230?
Product liability holds a manufacturer responsible for defectively designed products without proof of intent. Section 230 historically shielded platforms from liability for third-party content, but courts now treat AI-generated outputs as the company’s own product — the closer a claim comes to challenging system design or safety features, the more likely it is to proceed past the pleading stage. See the product liability framework for the full analysis.
Can I be sued if my SaaS product integrates OpenAI or Anthropic and a user is harmed?
Yes. You have the direct user relationship, you author the system prompt, and you make the deployment design decisions. A provider’s ToS compliance requirements do not transfer liability — they add a compliance obligation. No court has yet directly resolved this three-party structure, but the analytical case for deployer exposure is strong.
What is strict product liability — and why does it matter for AI chatbot documentation?
Strict product liability means liability follows from a defectively designed product without needing to prove negligence. The EU Product Liability Directive (December 2026) applies this to AI as a “product,” with unlimited damages. Engineering records showing a known safety gap that was not addressed become the defect evidence. Documentation showing the gap was identified and addressed is the defence.
What does a court actually look for in AI chatbot discovery?
RLHF training choice records, guardrail configuration history, safety classifier test results, escalation flow logs, prior user harm reports, and known-issues records. Testing artefacts, monitoring signals, and change histories shape causation narratives and settlement leverage. Absence of records implies no diligence.
Do I need to comply with the EU AI Act if my company is not based in Europe?
Yes, if your product is accessible to EU users. The EU AI Act has extraterritorial reach. The transparency disclosure obligation under Article 50 takes effect August 2, 2026 for any AI system interacting with EU users. SaaS companies calling OpenAI or Anthropic APIs are “deployers” under EU law with specific compliance obligations.
What is system prompt authorship — and why does it matter for liability?
The SaaS company writes the prompt that shapes the chatbot’s persona, boundaries, and behaviour. A company that writes an emotionally engaging, relationship-building prompt is analytically more exposed than one using model defaults unchanged. No court has directly litigated this yet, but it is the primary liability differentiator in any three-party analysis.
What are the most important engineering steps to take right now to reduce chatbot liability?
Four priorities: (1) audit design patterns against Washington HB 2225 prohibited categories and remove non-compliant patterns before January 2027; (2) implement California SB 243’s 988-crisis-referral classifier and notification requirements now; (3) begin EU AI Act disclosure implementation before August 2, 2026; (4) establish a documentation programme for safety classifier test results, escalation flow logs, and design decisions — this is the highest-leverage legal-defensive investment regardless of jurisdiction.
What is the Federated Hermes EOS Q1 2026 report?
The Federated Hermes EOS Q1 2026 Public Engagement Report (Ross Teverson and Navishka Pandit) is an institutional investor governance document explicitly framing AI chatbot safety as a material financial risk — EU fine exposure up to 7% of global annual revenue, litigation scale, reputational damage, and market access risk. Institutional investors are already tracking chatbot safety using this framework.
What is the difference between a deployer and a provider under the EU AI Act?
A “provider” develops and places an AI system on the market — OpenAI, Anthropic, Google. A “deployer” integrates it under their own authority — SaaS companies calling foundation model APIs. LEXR confirms that standard prompt engineering and RAG implementations typically do not cross the Article 25 threshold that would reclassify a deployer as a provider.
What is sandbagging in the AI safety context?
Sandbagging is an AI system performing better on safety evaluations than in real-world deployment. The Future Society identifies this as a documented failure mode. Adversarial testing in realistic deployment conditions is more reliable than standard evaluation protocols.
What should I do if my chatbot causes a serious user harm event and I cannot afford external legal counsel immediately?
Preserve all evidence first — do not delete, modify, or allow automated cleanup of conversation logs, system prompt versions, classifier outputs, or escalation logs. Designate one person as the internal record-keeper. Make no public statements. Do not contact the user. The evidence preservation obligation is time-sensitive and irreversible if missed.
Can I just wait for federal AI legislation before investing in state compliance?
No — state laws are active now. California SB 243 is already enforceable. Washington HB 2225 becomes effective January 1, 2027. The TRUMP AMERICA AI Act has not passed. Build compliance to the most demanding active standard now — the documentation and engineering work converts directly to federal compliance evidence if preemption eventually passes.
