# Challenges of GPT-3-based Conversational Agents for Healthcare

Source: [Challenges of GPT-3-based Conversational Agents for Healthcare](<https://www.sota2.com/research/paper/68fa70781e209131cea4b4b3>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T01:42:00.795Z

Authors: Fabian Lechner, Allison Lahnala, Charles Welch, Lucie Flek

Published: 2023-08-28T15:12:34.000Z

## Abstract

The potential to provide patients with faster information access while allowing medical specialists to concentrate on critical tasks makes medical domain dialog agents appealing. However, the integration of large-language models (LLMs) into these agents presents certain limitations that may result in serious consequences. This paper investigates the challenges and risks of using GPT-3-based models for medical question-answering (MedQA). We perform several evaluations contextualized in terms of standard medical principles. We provide a procedure for manually designing patient queries to stress-test high-risk limitations of LLMs in MedQA systems. Our analysis reveals that LLMs fail to respond adequately to these queries, generating erroneous medical information, unsafe recommendations, and content that may be considered offensive.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2308.14641>)

[PDF](<https://arxiv.org/pdf/2308.14641.pdf>)
