When Your AI Chatbot Practices Law or Medicine Without a License
Most AI compliance attention in 2026 goes to privacy, bias, and disclosure — regimes written recently and aimed at AI specifically. The exposure that has already produced enforcement against software companies is older and narrower: state statutes that make it unlawful to render professional services without a license. They apply to conduct, not to what a company calls itself, and a helpful assistant is very good at the conduct they prohibit.
The Statutes Were Written for Humans and Say Nothing About Software
Unauthorized practice of law and unlicensed practice of medicine provisions exist in every state, generally phrased around holding oneself out as qualified, applying professional judgment to another person's circumstances, or performing acts reserved to license holders. None of them contemplate a language model. That is the problem rather than the escape hatch: because the definitions are behavioral, a court or licensing board asks what the service did for the user, and a company arguing "a model produced it, not a person" is arguing that nobody is accountable for conduct the statute prohibits — an argument regulators have not received warmly in analogous contexts.
The practical exposure is broader than legal-tech and health-tech startups. Insurance companies whose claims assistants explain coverage consequences, HR platforms whose bots answer termination questions, tax software whose chat interface goes beyond form mechanics, employee-benefits portals fielding "should I enroll in the HDHP given my condition?" — all of these are ordinary product decisions that route users toward a specific recommendation about their own situation.
Why Generative AI Crosses This Line by Default
A retrieval bot that returns a help-center article stays on the information side because it can only return what exists. A generative assistant is asked to synthesize, and the behavior users reward — reading their uploaded document, weighing their described circumstances, telling them what to do next — is exactly the application of judgment to particular facts that licensure statutes reserve. Worse, the model's confidence is uncorrelated with its correctness, so the outputs most likely to be relied on are not the ones most likely to be right. Companies that built a scoped FAQ bot in 2023 and swapped the backend for a frontier model in 2026 changed their regulatory posture without a review, because the interface looks identical.
Guardrails That Actually Constrain Behavior
A system-prompt instruction not to give legal advice is a suggestion to a model that users are actively trying to talk around, and it fails against the most motivated queries. Structural controls hold up better: refusal classifiers that run on the output rather than the prompt, topic allowlists that scope the assistant to product mechanics, a hard escalation path to a licensed human triggered by categories rather than by sentiment, and disabling document upload in flows where analyzing the document is the regulated act. Each of these can be tested and evidenced later, which is the difference between a control and an intention.
What the Interface Communicates Is Part of the Analysis
Holding out matters independently of what the model says. A bot named after a profession, an avatar in a white coat, a badge reading "AI attorney," or marketing copy promising to replace a $400/hour consultation all support a finding that the service presented itself as qualified to practice. Product and brand decisions therefore carry legal weight that the teams making them rarely price in, and the cheapest fix in this entire area is usually a rename.
Scope Audit Checklist
Work through this with real transcripts, not with the demo script.
The Second Exposure: Ordinary Negligence
Licensure is only one of the two claims that arrive together. If a user follows an assistant's confident, wrong answer and misses a filing deadline, waives a claim, or delays care, the resulting suit is a straightforward negligence and consumer-protection matter — and the company's own marketing about reliability becomes the plaintiff's evidence on reliance. A defense built entirely on a footer disclaimer performs poorly against a product page that promised expert-quality answers.
Regulated-industry sites carry accessibility exposure too
Legal, health, insurance, and benefits platforms serve exactly the populations most likely to file an accessibility complaint — and chat widgets are among the most commonly inaccessible components on any site. RatedWithAI scans your pages for the defects that create that risk.
Scan Your Site for Free →Frequently Asked Questions
Our chatbot only answers questions about our own product. Are we exposed?
Usually much less, and that scope is the strongest position available. The risk creeps in when the product itself is a regulated subject — a claims portal explaining whether a denial is appealable, or a benefits tool weighing plan choices against a described medical condition, is answering a question about the user's situation even though it never left your product.
Does routing every hard question to a human solve it?
It substantially reduces the risk when the routing happens before an answer is shown rather than after. An assistant that gives its best guess and then adds 'you may want to consult a professional' has already rendered the service; the handoff has to replace the answer, not follow it.
Are we safer if a licensed professional wrote the underlying content?
Professionally authored source material improves accuracy and is worth having, but it does not change the analysis, because the regulated act is applying that material to one person's facts. A model that selects, combines, and tailors attorney-written passages to a specific user has done the thing the license governs.
How do state differences affect a national product?
In practice you build to the strictest jurisdiction you serve, since geofencing behavior by state is fragile and users travel. Some states also have specific carve-outs for self-help materials or document preparation that others lack, so a feature that is clearly permissible in one place can be the basis of a board complaint in another.
Do open-source or third-party model providers share this liability?
The deploying company is the party rendering the service to the user, and that is who the statute reaches. Model provider terms typically disclaim responsibility for how output is used, so contractual risk transfer in this area is thin and should not be assumed to exist.