Hallucination and omission in AI-summarized notes

Ambient and visit-summary tools fail in opposite directions. Hallucination adds confident falsehoods. Omission removes truths while leaving a note that still “looks finished.” Both are more dangerous than a blank field, because blank fields invite questions and fluent paragraphs do not.

This post is educational. clinicgpt.ai hosts a browser-only SOAP demo and articles — not a production scribe product. Nothing here is clinical or legal advice.

Why fluency hides the problem

Large language models are trained to produce coherent continuation. Clinical notes are a genre the models have seen often: chief complaint, denials, med lists, follow-up intervals. When audio is noisy or the transcript is partial, the model still has a strong prior for what a “good” note looks like. That prior is exactly how “Patient denies chest pain” appears when chest pain was never discussed.

Human tired typing usually leaves scars — fragments, TODOs, uneven detail. Model drafts leave polish. Reviewers who equate polish with accuracy will miss errors systematically.

Hallucination patterns worth training for

Invented negatives. Reviewers should treat unsolicited “denies X / Y / Z” blocks as high-risk when the visit was problem-focused.

Medication confabulation. ASR turns a mumbled drug into a neighbor on the formulary; the summarizer “helpfully” adds dose and frequency from common regimens.

Timeline smoothing. “Symptoms for two weeks” becomes “three months” because an earlier visit’s context leaked in — or the reverse.

Exam theater. Elements of a complete exam appear though only a focused exam occurred. This is both a safety issue and a billing integrity issue if someone later up-codes from the fiction.

Attribution swap. Caregiver history becomes patient history; clinician counseling becomes patient preference.

Omission patterns worth training for

Single-mention loss. The allergy stated once at the door gets dropped when the model prioritizes the longer HPI.

Soft social risk. Food insecurity, intimate partner safety, and transportation barriers are often whispered or half-said; models trained on “clinical-sounding” text under-weight them.

Plan contingencies. “If the culture is positive, switch to X” disappears, leaving only the immediate prescription.

Clinician uncertainty. “Unclear if cardiac vs reflux; follow-up if…” becomes a decisive diagnosis statement.

Omission is harder to catch in dual review unless the second reviewer listens to audio or reads the full transcript, not only the draft note.

A practical sampling method

For a pilot or ongoing QA, pick a weekly sample:

Random 10 notes per specialty pod
Second clinician (or trained QA) compares draft vs audio/transcript for: meds, allergies, problems addressed, plan items, explicit denials
Tag each defect as hallucination, omission, or attribution
Feed aggregate rates to the vendor and to your training curriculum

You do not need a biostatistics department. You need a denominator and honesty about what you found. Vendors that refuse to discuss defect rates in your specialty are telling you something useful.

What does not fix the problem

A longer system prompt alone
A prettier SOAP template alone
Turning on “more creative” model settings
Assuming specialty fine-tunes eliminate omission

What helps: forced review UX, field-level confidence flags when the vendor has them, shorter visit types first (lower complexity), and cultural permission for clinicians to delete large wrong sections instead of micro-editing them into truth.

Specialty notes (not rankings)

Primary care. High volume, mixed agenda visits. Omission of secondary complaints is common when the model latches onto the first problem mentioned.

Behavioral health. Long narrative, few vitals, high attribution risk when family is present. Invented mental-status exam phrases are a recurring review finding in informal pilot reports practices share with each other — treat that as a training focus, not a vendor smear.

Procedural / surgical follow-up. Template expectations are strong; models fill expected post-op denials. Verify wound checks and red-flag counseling actually occurred.

None of this is a claim that any named commercial product is better or worse. It is a claim that your sample should be stratified by visit type.

Teaching reviewers what “good skepticism” looks like

New users often either trust everything or trust nothing. A middle path:

Read the assessment and plan first (highest clinical leverage).
Check meds/allergies against what you remember from the room.
Scan for denials you did not elicit.
Only then tidy the HPI prose.

If step 2 or 3 fails often, the tool is not ready for unsupervised volume — regardless of how pretty the HPI is.

How this relates to the clinicgpt.ai demo

Our homepage demo uses regex/templates and often leaves fields as “not detected.” That emptiness is educational: rule-based systems fail loudly; neural systems fail politely. Neither failure mode belongs in the chart without a human. More: how this demo works and keeping the clinician responsible. For pilots, pair defect sampling with the metrics in ambient scribe pilots: what to measure.

Sources and further reading (as of 2026-07-21)

Documentation burden context: Sinsky C, et al. Ann Intern Med. 2016 (time-motion; why practices adopt ambient tools).
Federal health IT / documentation burden framing: HealthIT.gov.
When vendors handle PHI: HHS HIPAA for Professionals.

This post was drafted by AI and reviewed by our editorial team. Last updated 2026-07-21.