P0977 - Personalizing Hepatology: The Role of Chatbots in Tailored Treatment Plans in the Field of Hepatology

Rajmohan Rammohan, MD, Melvin Joy, MD, Dilman Natt, MD, Tulika Saggar, MD, Sai Greeshma Magam, MD, Achal Patel, MD, Abhishek Tadikonda, MD, Paul Mustacchia, MD, MBA
Nassau University Medical Center, East Meadow, NY

Introduction: Chatbot’s like ChatGPT and Google Bard have significantly impacted healthcare, particularly hepatology, enhancing liver disease diagnosis and treatment. These advanced solutions interpret ultrasound images, analyze various samples, streamline administrative tasks, and aid in medical imaging and device automation. By enabling personalized treatments and side-effect predictions, they have considerably improved hepatology disorder management. Integrating these AI tools provides patients with tailored treatment evaluations, promoting informed decision-making. However, the effectiveness of these chatbots still warrants further scrutiny.

Methods: The study set out to evaluate the precision of two prominent Chat Bots - Chat GPT and Google BARD - in addressing medical management queries. Both bots were tasked with a series of questions, and their responses were evaluated on a 1-10 Likert scale, with 1 indicating utmost accuracy. Two unbiased evaluators examined each bot's responses to ensure objective evaluation. The research aimed to provide insight into these Chat Bots' capabilities via a structured performance appraisal, focusing on accuracy and dependability. The dual-evaluator system and the Likert scale approach helped minimize possible bias, thereby lending credibility to the findings.

Results: Our research contrasted the effectiveness of Chat GPT and Google BARD in the realm of liver disease management. Chat GPT excelled, scoring 74% in the accuracy of information compared to Google BARD's 49% (p = 0.015) and 48% vs.29% in medical information reliability (p = 0.032). This study emphasizes the critical role of accuracy, and dependability, in developing liver disease-oriented chatbots. While Chat GPT displayed impressive performance, indicating its potential as a reliable tool, it also underscored the necessity for continued research and development in this field.

Discussion: This research revealed the relative strengths of Chat GPT and Google Bard in liver disease management, with Chat GPT outperforming in both accuracy and reliability. As AI technologies continue to permeate healthcare, ensuring their accuracy is crucial. While Chat GPT demonstrated promising potential as a reliable hepatology tool, it also highlighted the continuous need for research and development in this domain. The significant disparity in performance emphasizes the importance of rigorous testing and validation before integrating AI tools into medical practice.




Disclosures:


Rajmohan Rammohan, MD, Melvin Joy, MD, Dilman Natt, MD, Tulika Saggar, MD, Sai Greeshma Magam, MD, Achal Patel, MD, Abhishek Tadikonda, MD, Paul Mustacchia, MD, MBA. P0977 - Personalizing Hepatology: The Role of Chatbots in Tailored Treatment Plans in the Field of Hepatology, ACG 2023 Annual Scientific Meeting Abstracts. Vancouver, BC, Canada: American College of Gastroenterology.