When AI Gives Legal Advice

And Someone Eventually Blames the Robot

👋 Join thousands of others following and learn more about all the complex and fun legal topics. Subscribe below!

---

For the last couple of years, AI companies have been very careful to include a warning whenever their tools answer questions about law, taxes, medicine, or anything else that could end up in a courtroom.

The warning usually says something like: “This tool does not provide legal advice.”

Unfortunately, it often appears immediately after the AI has just written three paragraphs explaining what you should probably do about your legal problem. Which is how we ended up here.

A new lawsuit claims that an AI chatbot provided legal guidance that allegedly caused real harm. According to the complaint, a user relied on responses from ChatGPT suggesting they could challenge a legal settlement and file certain motions. The plaintiff argues that following that guidance violated the settlement agreement and caused financial damage.

Whether the claim ultimately succeeds is almost beside the point. The interesting part is that courts are now being asked to answer a question that didn’t exist five years ago:

If someone relies on advice generated by an AI system, who’s responsible?

The “It’s Just a Tool” Argument

AI companies tend to describe their systems as tools. Search engines, calculators, productivity software; helpful things that produce information but don’t make decisions.

That description makes sense technically, but from a user’s perspective, the experience feels different. Ask a chatbot a legal question and the response often reads like something written by a junior associate who stayed up too late preparing for a meeting.

It explains the issue.

It walks through possible options.

Sometimes it even sounds confident about what the next step should be.

At that point the line between “information” and “advice” gets fuzzy.

Courts Have Had a Preview

Lawyers got an early warning in 2023 when two attorneys filed a brief citing several cases that did not exist. ChatGPT had confidently invented them. The court sanctioned the lawyers, who discovered the hard way that judges expect attorneys to confirm their sources.

That episode was embarrassing, but the responsibility was easy to assign: the lawyers filed the brief.

The new lawsuits are different. Now plaintiffs are asking whether the software itself created the problem. That moves the conversation into a different legal territory entirely.

Product Liability Is the Next Logical Stop

One theory being explored is simple: if a product generates guidance that people reasonably rely on, and that guidance causes harm, the manufacturer might bear some responsibility.

The argument sounds familiar because courts have dealt with similar questions before.

• Medical devices that provide faulty readings.

• Financial tools that produce misleading calculations.

• Navigation systems that send drivers somewhere dangerous.

In each case the legal system eventually asks whether the product was designed in a way that created an unreasonable risk. The lawsuits against AI companies are starting to ask the same question.

Disclaimers Are Not Magic Shields

Developers understandably point to the warnings that appear on nearly every AI interface: “not legal advice,” “consult a professional,” and similar language. Those disclaimers help. Courts do consider them.

But disclaimers are rarely the end of the analysis. Judges tend to look at the entire product experience. If the system produces authoritative-sounding explanations that look like professional advice, a warning label may not completely resolve the issue.

Think about a GPS app that warns you not to rely on its directions, and then confidently instructs you to drive into a lake. At some point the warning and the behavior stop matching.

There’s Also a Regulatory Angle

Another legal thread appearing in early discussions is unauthorized practice of law.

Several states already regulate who can give legal advice. Traditionally that meant licensed attorneys. Software companies never had to worry about it because software didn’t really “advise” anyone.

Generative AI changed that. When a chatbot analyzes a legal problem and suggests next steps, the question becomes whether the tool has crossed a line that regulators care about. That issue is still developing, but it’s already on the radar.

Design Decisions Suddenly Matter

Once liability enters the conversation, product design choices take on new significance. Developers now have to think carefully about how systems respond to questions involving legal rights or obligations.

• Should the AI decline to answer certain questions entirely?

• Should it limit responses to summaries of publicly available law?

• Should it avoid recommending specific actions?

Different companies are experimenting with different guardrails, and it’s likely those guardrails will evolve as courts weigh in.

This Goes Beyond AI Companies

You don’t need to be building AI models for this issue to affect you. Employees are already using AI tools to help with tasks that have legal implications:

• reviewing contracts

• interpreting regulations

• analyzing litigation risk

• drafting internal policies

Sometimes those outputs are excellent. Sometimes they’re confidently wrong.

When organizations begin relying on those answers, the question of responsibility becomes more complicated, and courts are only starting to sort through it.

Where This Probably Leads

Generative AI isn’t going away. It’s too useful. The legal system will eventually settle into a framework that balances innovation with accountability, the same way it has for every major technology before it.

In the meantime, lawsuits like this one will test the boundaries. Lawyers sometimes describe that stage of development as “an emerging area of law.”

A more accurate description might be: the part where everyone starts hiring litigators.