Chatbot Effectiveness in Health Education: A  Comparative Study of Gemini, ChatGPT, and Llama

Patrick Kramml

doi:10.60522/o:7212

Open in new window

Title (eng)

Chatbot Effectiveness in Health Education: A Comparative Study of Gemini, ChatGPT, and Llama

Author

Patrick Kramml

St. Pölten University of Applied Sciences

Description (eng)

This paper presents a focused analysis of the application of AI-driven chatbots in health education, drawing upon a comparative evaluation of OpenAI’s ChatGPT, Google’s Gemini, and Meta’s Llama. The study examines existing literature on the efficacy of large language models (LLMs) in healthcare, emphasizing their potential and challenges. Through a detailed expert evaluation, the response quality of these chatbots to health-related queries is assessed based on medical correctness, safety, and user satisfaction. Findings from the expert evaluations indicate that while ChatGPT and Gemini consistently deliver responses closely aligned with established medical guidelines, Llama2 shows limitations in clarity and precision. The literature review underscores the rapid advancements in AI technology, yet it also highlights the ongoing concerns about data privacy, response accuracy, and the risk of misinformation in medical contexts. This paper contributes to the growing discourse on integrating AI in healthcare, proposing that while AI chatbots hold promise for enhancing health education and accessibility, careful consideration of their limitations and ethical implications is crucial for their effective deployment.

Keywords (eng)

Large Language ModelsAIHealthcareAssessment

Subject (eng)

ÖFOS 2012 -- 5080 -- Media and Communication Sciences

Subject (eng)

ÖFOS 2012 -- 202022 -- Information technology

Subject (eng)

ÖFOS 2012 -- 202002 -- Audiovisual media

Subject (eng)

ÖFOS 2012 -- 6040 -- Arts

Type (eng)

conference object