Skip to content
Lexyno
Back to Academy
Legal TechNov 17, 202610 min readJun LeeJun Lee

ChatGPT for Lawyers: What PI Firms Can Safely Use It For

ChatGPT for Lawyers: What PI Firms Can Safely Use It For

ChatGPT for lawyers at a PI firm: what to paste and what not to, what the bar says about client data, and how to check a citation before you file it.

Jun Lee
Jun Lee
Co-Founder & CTO

It's 9 p.m. The demand is due Friday, the ER note is open in one window, and ChatGPT is open in the other. You could paste the note in and ask for a summary. It would take a second. Should you?

ChatGPT for lawyers works for some PI tasks and is the wrong tool for others. The line isn't about how smart it is. It's about where your client's data goes and who checks what comes back. I build AI software for PI firms, so I have a stake in this. Here's where I'd draw the line, and why.

Woman in a red blazer typing on a laptop at an office desk next to a stack of file trays
The question isn't whether to open the chat box. It's what goes into it.

Can Lawyers Use ChatGPT?

Yes. No rule we know of bans it, and a federal court in Mata v. Avianca said there's "nothing inherently improper" about using a reliable AI tool. The duties don't change, though. You protect client confidences, you check every output, and you answer for anything you file. ChatGPT is allowed. Trusting it blindly isn't.

So the real question is narrower. Which tasks can go into ChatGPT, under which account, and what do you check before using the result? The rest of this post answers those three questions for a PI practice.

The Myth: Turn Off Training and It's Confidential

Here's the comfortable version. "I turned off training in settings, so nothing I paste leaves the conversation."

It sounds right. It mixes up two different promises.

OpenAI's own pages describe what each kind of account does. Read them closely and you'll see what's promised, and what isn't.

What OpenAI saysPersonal ChatGPTBusiness plans and the API
Training on your content"We may use your content to train our models," unless you opt out"By default, we do not train on any inputs or outputs"
After you opt out"New conversations will not be used to train our models"Not needed; off by default
Temporary ChatNot in history, no memories, not used for trainingn/a
Control over how long data is kept (enterprise privacy page)Not covered by that pageListed for Enterprise, Healthcare and Edu, not Business

Sources: OpenAI's training policy (updated Mar 13, 2026) and enterprise privacy page (updated Jan 8, 2026). They change, so read the current versions.

Notice what the table is about. Training. Not confidentiality, not who can access the data, not how long it's kept on a personal account.

Opting out answers one question. Your ethics duty asks a bigger one.

What Does the Bar Say About Client Data?

California and Washington both say the same thing in different words: don't put confidential client information into an AI tool unless you've checked that its terms and security protect it. A training opt-out doesn't settle that. California adds that, absent informed client consent, you must not use a tool that presents material risks to confidentiality.

SourceWhat it says
California State Bar, Practical Guidance on Generative AI (May 14, 2026)An attorney "must not input any confidential information of the client into a generative AI solution that may present material risks to confidentiality or security, absent informed client consent"
Same guidance"Even if the product does not utilize or share inputted information, it may lack reasonable or adequate security"
Same guidance"Reasonable efforts require more than reliance on generalized marketing assurances." A user "may agree to the terms of use and privacy policy simply by using the product"
WSBA Advisory Opinion 2025-05"Some AI-enabled consumer products don't include contractual assurances of confidentiality thereby posing an unreasonable risk to confidential client information"
Same opinionUsing an "incognito" setting "may not completely preserve confidentiality." A vendor's confidentiality terms "must meet a lawyer's duty of confidentiality under RPC 1.6"

The incognito line is the one I'd tape to the monitor. Temporary Chat is the closest thing ChatGPT has to incognito.

Temporary Chat keeps the chat out of your history and out of training. It's still not a confidentiality agreement.

California also notes that a breach can happen "via both prompt content and uploaded materials." In PI work, the upload is the risk.

A medical record carries the client's name, date of birth and diagnoses, usually on the first page. Our HIPAA and AI demand letters guide covers the health-privacy side.

Young woman at a law office desk pulling papers from a manila envelope, with a laptop and scales of justice nearby
The records envelope is where the confidentiality question starts.

Why Does ChatGPT Invent Cases?

ChatGPT writes by predicting the next likely word. It doesn't look a case up unless the system around it is built to retrieve one. So it can produce a citation with the right shape, a believable court and a plausible year, for a case that doesn't exist. California's guidance calls these "purported citations to legal authority that does not exist."

Researchers have a partial explanation.

A 2025 paper argues that the way models are trained and scored rewards guessing over admitting uncertainty. A model that says "I don't know" scores worse on many tests than one that guesses. So it guesses, fluently.

That fluency is the danger. California's guidance warns that these tools project confidence and emulate human responses, which encourages "greater reliance and trust on its outputs."

A wrong answer that sounds unsure gets checked. A wrong answer that sounds sure gets filed.

You Can't Ask It to Check Itself

The Mata record shows this better than any study. The lawyer had used ChatGPT for research, testifying to having "falsely assumed" it was "like a super search engine." When the court questioned the cases, the lawyer went back to ChatGPT and asked, as the court's findings record it, "Is Varghese a real case."

According to the sanctions opinion, ChatGPT "responded that it had supplied 'real' authorities that could be found through Westlaw, LexisNexis and the Federal Reporter."

It couldn't. The court sanctioned the lawyers and their firm.

Asking the model whether its answer is right just runs the same prediction again. The check has to come from outside the chat: the reporter, the docket, the record.

Our post Will AI Replace Lawyers? covers what the court said about responsibility.

Two attorneys at a wooden desk with stacks of legal documents, one reading a page closely
Verification comes from the reporter and the record, not from a second prompt.

What Can ChatGPT Do in a PI Firm?

ChatGPT is useful for drafting and explaining work that contains no client data and no legal authority you'll rely on. Think plain-language rewrites, intake scripts, deposition topic lists from made-up facts, and explaining a medical term. Anything with client records, case law or the numbers in a demand belongs somewhere you can check.

TaskUse ChatGPT?What to check
Rewrite a client status update in plain EnglishYes, with names and case details removedTone, and that nothing reads as advice you didn't give
Draft an intake script or a firm FAQYesEvery legal statement, against your state's rules
Brainstorm deposition topicsYes, from a made-up fact patternFit to your real case; it doesn't know it
Explain a medical term or procedureYes, as a starting pointThe record itself and a medical source
Summarize a statuteOnly with the official text in front of youEach sentence against the statute; the version in force
Find case lawNoUse a research database; pull and read every case
Summarize a client's medical recordsNot in a consumer accountSee the bar guidance above
Add up specialsNoUse a spreadsheet from the bills

The pattern behind ChatGPT for lawyers is simple. It's good with language and weak with facts it has to remember.

So feed it language problems. Keep fact problems in tools that point back to a source. Our guide to AI for personal injury lawyers splits the rest of a PI file the same way.

Can ChatGPT Write a Demand Letter?

ChatGPT can write a letter shaped like a demand. That's different from a demand built from your file. To use your file, it needs the records, and a consumer account is the wrong place for them. It also predicts totals and dates instead of adding them from the bills. So every figure in a ChatGPT demand letter needs checking.

The numbers are where this bites. A model can produce a specials total that's close but wrong, and close looks fine on a quick read.

The adjuster will add it up. If your total is off, the rest of the letter loses credibility. Our post on why AI demand letters get numbers wrong explains the mechanism.

Special damages table in a demand letter listing seven providers with amounts and a total of $16,681.50
Seven bills, one total. Each line should come from a bill, and the sum from arithmetic, not prediction. Demo case (synthetic data).

If you want to see how a full demand is built from the records, start with our sample car accident demand letter and the guide to demand letter sections. How specials and other losses are counted is in our economic damages guide.

Bad Prompt, Better Prompt

Most confidentiality slips happen in the prompt, not the settings. Here are two common versions, and how to fix them.

Weak: "Summarize this ER note for my client Jane Doe, DOB 3/2/1985, rear-ended on I-5 on June 4. [pasted note]"

Why it fails: A name, a birth date, an accident date and a location, plus the whole medical note. That's identifiable client information in a consumer chat.

Better: "Explain in plain English what 'cervical strain with paraspinal tenderness, no acute fracture on CT' usually means for a patient. Two short paragraphs."

Weak: "Give me three California cases where a rear-end plaintiff with a prior back injury still recovered."

Why it fails: It asks the model to remember case law. That's the exact task that went wrong in Mata.

Better: "List the questions I should research about recovery for aggravation of a prior injury in California. Don't cite cases."

The better prompts share a shape. No client facts, no request for authority, and an answer you can check against something you already have. That's not a workaround. It's using the tool for what it's good at.

One test settles most cases. If you can't write the prompt without client facts, the task doesn't belong in a consumer chat. Move it to a tool your firm has reviewed, or do it by hand.

Try Lexyno FreeGenerate a mock demand letter from your own records.

How Do You Check a Citation From ChatGPT?

Check it outside ChatGPT. Pull the case from a research database or the court's own site, read it, and confirm the quote and the holding are really there. Then confirm it's still good law. California's 2026 guidance says an attorney must review AI outputs, "including, but not limited to analysis and citations to authority before submission to the court."

  1. Pull it. Find the case by citation in a research database or on the court's site. If you can't find it, it doesn't go in.
  2. Read the page. Open the pin cite. Confirm the quote appears there, word for word.
  3. Check the holding. Confirm the case says what the draft claims, not something near it.
  4. Check its status. Run a citator. A real case that's been reversed hurts you too.
  5. Note who checked. Put the checker's initials and the date in the file.

Every step is yes or no. A paralegal can run it, and you can audit it.

Washington's opinion describes a Colorado lawyer disciplined for filing a motion with AI-generated citations that weren't checked. That's the failure these five checks are for.

Two men in suits at a table, one pointing at a laptop screen while the other reads it
Decide the firm's rules once, before someone decides them at 9 p.m.

When This Advice Is Wrong

I've drawn the line conservatively. Here's where a reasonable firm might draw it differently.

  • A business account you've actually reviewed. If your firm uses a business plan, has read the terms against your confidentiality duty, and has client consent where needed, the "no client data" rule may be stricter than you need. California's guidance describes that review, and says it may include consulting IT or security experts.
  • Public information only. Summarizing a published statute or a public court order raises no confidentiality issue. You still check the summary against the text, and you still pull the version in force on your date.
  • Your state's rules. I've quoted California and Washington. If you practice elsewhere, read your own bar's current guidance. It keeps changing.

What We Promise Instead

We don't claim our AI makes no mistakes. Nobody's AI is there yet, and a vendor who tells you otherwise is selling the same confidence that got the Mata lawyers in trouble.

Our position is narrower: AI for assembly, humans for verification and strategy.

In practice, that means you shouldn't have to trust a draft. You should be able to check it.

Each fact points back to the page it came from, so a paralegal checks a date against its source instead of rereading the file. Our post on Fact-Lock explains how that works and why we won't claim zero.

Medical chronology visit table listing providers, treatment periods, visit counts and the source file and page for each row
Every row names the page it came from, so checking it doesn't mean rereading the file. Demo case (synthetic data).

Where Lexyno Fits

Lexyno doesn't value a case, choose a causation theory or give advice. Those are the attorney's calls.

Lexyno's AI demand letter software drafts the demand from the case's own records, with source references you can check, and exports to Word.

Medical Summary builds the treatment chronology and flags gaps over 30 days. Our medical chronology guide shows what a finished one looks like.

For your staff's side of the change, read our post on paralegals and AI. Plans are on the pricing page.

Use It for Words. Keep the Facts at Home.

ChatGPT for lawyers is a writing tool, not a records system or a research database. Give it language problems with no client in them. Keep the records, the authorities and the numbers where you can trace each one to its source.

The tool will keep getting better. Your duty to check won't get smaller.

Start from our PI demand letter template, or try Lexyno on one case from your own file, free.

Frequently asked questions

Yes. No rule we know of bans it, and a federal court in Mata v. Avianca said there is nothing inherently improper about using a reliable AI tool. The duties don’t change, though: you protect client confidences, you check the output, and you answer for anything you file.

Not into a consumer account. California’s 2026 bar guidance says an attorney must not put confidential client information into an AI tool that presents material risks to confidentiality or security, absent informed client consent. Washington’s bar says consumer products without contractual confidentiality terms pose an unreasonable risk.

No. Opting out stops OpenAI from training on new conversations. It isn’t a confidentiality agreement. California’s guidance notes that a tool which doesn’t use your inputs may still lack adequate security, and that reasonable efforts require more than generalized marketing assurances.

It can write text shaped like a demand letter. It can’t read your file unless you give it the records, and a consumer account is the wrong place for them. It also predicts totals and dates rather than adding them from the bills, so every figure needs checking against the source.

Pull the case yourself from a research database or the court’s own site. Read it, confirm the quote appears in it, and check that it is still good law. Don’t ask ChatGPT whether the case is real. In Mata, the lawyer did, and it said yes.

It depends on your state and the use. California’s 2026 guidance says an attorney must consider disclosing the intent to use AI in the representation, including how it will be used and its benefits and risks. Check your own bar’s guidance and any client instructions.

Related posts

What's next?

Ready to see the difference in your next case?

Put these strategies to work. Try a mock demand generated by Lexyno today.

Request Evaluation Access