CLINICAL QUESTION: Can large language models (LLMs) produce hospital discharge summary narratives comparable in quality and safety to those written by physicians?
BACKGROUND: High-quality hospital discharge summaries are necessary to facilitate safe transition of care from the hospital to the post-acute setting; however, they contribute to clinical documentation burden. LLMs may provide the opportunity to draft narratives to be reviewed and edited by physicians.
STUDY DESIGN: Cross-sectional study
SETTING: University of California, San Francisco (UCSF)
SYNOPSIS: Between 2019 and 2022, 100 inpatient hospital medicine encounters at UCSF with a length of stay between three and six days were randomly selected for LLM-generated discharge summary narratives. LLMs and physician-generated hospital discharge narratives were blindly reviewed by 22 attending physicians for errors and overall quality. LLM-generated narratives were more likely to contain errors, with a mean error per summary [standard deviation] of 2.91 [2.54], compared to physician-generated summaries at 1.82 [1.92]; however, there was no significant difference in the potential for harm between the two groups. LLM- and physician-generated narratives were equivalent in overall quality on a Likert scale. LLM-generated narratives were more concise and coherent, but less comprehensive. This study was limited by the number of patient encounters included and the shorter length of stay. The ability of LLMs to generate narratives for more complex encounters is unclear.
BOTTOM LINE: LLMs have the potential to draft hospital discharge narratives for clinician review and editing.
CITATION: Williams CYK, et al. Physician and large language model-generated hospital discharge summaries. JAMA Intern Med. 2025;185(7):818-825. doi: 10.1001/jamainternmed.2025.0821.
Dr. Nguyen
Dr. Nguyen is a hospitalist at the Nebraska Medical Center and assistant professor and associate medical director of hospital medicine informatics in the department of internal medicine at the University of Nebraska Medical Center, both in Omaha, Neb.