Clinical Question: In physicians with training in AI literacy, is clinical reasoning impaired by reliance on large language model (LLM)-generated content?
Background: While some studies have shown improvement in clinical reasoning with the use of accurate LLM content, the impact on physicians’ clinical reasoning when provided erroneous LLM-generated content is unknown.
Study Design: Single-blinded, randomized, controlled trial
Setting: University-based research setting in Lahore, Pakistan
Synopsis: 44 physicians who completed a 20-hour course on AI literacy were randomized to two groups that reviewed six case vignettes. Both groups received the option to review ChatGPT-4o-generated recommendations, with the control group receiving error-free diagnostic recommendations and the study group receiving standardized, flawed recommendations in three of six cases. Participants were graded by three blinded raters on their top three differential diagnoses, top diagnosis, and recommended next steps in diagnosis, using a standardized rubric. The primary outcome measured mean diagnostic reasoning, for which the control group achieved 84.9%, and the study group achieved 73.3% (95% CI, −18.9 to −9.1; P <0.0001). Subgroup analysis was notable for greater drops in reasoning accuracy in physicians with more clinical experience. These findings were attributed to physicians’ susceptibility to automation bias when interacting with LLM outputs. Limitations included evaluation of a single LLM, unknown applicability of intentional errors compared with real-world LLM errors, lack of assessment of the AI literacy course, and unclear impact on clinical care, particularly with the limited complexity provided by case vignettes.
Bottom Line: Physicians are susceptible to automation bias in diagnostic reasoning when using AI-generated outputs, even when trained to be critical of AI outputs.
Citation: Qazi IA, et al. Automation bias in large language model–assisted diagnostic reasoning among physicians trained in AI literacy — a randomized clinical trial. NEJM AI. 2026;3(5). doi:10.1056/AIoa2501001.
Dr. Heery
Dr. Heery is a hospitalist at NYU Langone’s Tisch Hospital and a clinical assistant professor at NYU Grossman School of Medicine, both in New York.