Field Notes / Case Study
Cover illustration for The law firm chatbot that made up case law..
Field Notes / Case Study

The law firm chatbot that made up case law.

A solo estate planning attorney in Boca Raton, Florida. Four years in practice. Strong referral network. Website traffic from local search. She wanted an AI chat assistant to handle five core FAQs.

By Binil Chacko8 minute readBoca Raton, FLScope containedHard FAQ library, full disclaimer

A solo estate planning attorney in Boca Raton, four years in practice, with a strong referral network and website traffic driven by local search. She wanted an AI chat assistant to handle FAQ traffic. What does probate cost in Florida. How long does it take. Do I need a trust. What is the difference between a will and a trust. Who needs a healthcare surrogate designation.

The build

A clean, scoped FAQ chatbot trained on a library of attorney-approved answers to those five core questions, plus general Florida estate planning context. Initial deployment was smooth. Response quality on the approved topics was high.

What almost went wrong

Three weeks in, a prospective client asked a question outside the approved scope, about Florida's elective share laws for a blended-family situation involving children from two marriages and a surviving spouse. The bot answered. Confidently, in plain language, with apparent specificity. It cited a 2022 Florida statute amendment governing spousal elective shares in blended-family estates.

Chat transcript / 03.14Live
ProspectI have kids from a prior marriage and a new spouse. What does Florida's elective share rule say about my situation?
AI assistantUnder the 2022 amendment to Florida Statute 732.2065, the elective share for a surviving spouse in blended-family estates is calculated against an augmented estate that includes pre-marital trusts.
AI assistantIn your specific case, your spouse would be entitled to 30 percent of the augmented estate, including assets you held prior to the marriage.
Cited authority
Fla. Stat. § 732.2065 (2022 amendment)

That citation does not exist. The bot generated a plausible statute number, a plausible date, a plausible policy summary.

The attorney caught it during the follow-up intake call when the prospect referenced the chatbot's answer as the basis for her estate planning assumptions. If that client had acted on the AI's answer without a consultation, the exposure would have been significant, and potentially actionable.

The fix

The chatbot was immediately restricted to a hard-scoped FAQ library. Any out-of-scope question now triggers a single response: "That is a great question for your consultation. Here is how to book 15 minutes with [attorney name]." A mandatory disclaimer was added to every response: "This is general information only. Not legal advice." The chatbot still handles hundreds of FAQ interactions per month and books consultations autonomously. It no longer attempts to answer anything it was not explicitly trained on.

LLMs answer questions they were not asked to answer, confidently. On a law firm website, that is a malpractice vector. The fix is scope containment, not smarter AI.
The lesson, Boca Raton engagement

What this engagement is, and is not

The Boca Raton engagement is the foundational lesson behind the AI for lawyers vertical guide and the Mata-lesson contrarian piece. The pattern generalizes across every consumer-facing AI surface that touches regulated information: medicine, legal, financial advice, immigration. Hard scope first. Smart routing second. Disclaimers always. The model never answers what it was not explicitly trained to answer.

Want a similar engagement

Talk to us about a legal practice build.

Free AI audit first. We read your site, your reviews, your visible stack, then return a prioritized roadmap with a savings estimate. No deck, no upsell. Sometimes we tell you not to build anything yet.

Run free AI auditBook 15 minutes

Related reading