Comparison of Ophthalmologist and Large Language Model Chatbot Responses to Online Patient Eye Care Questions

Key Points Question How does ophthalmology advice generated by a large language model chatbot compare with advice written by ophthalmologists? Findings In this cross-sectional study of responses to 200 eye care questions from an online advice forum, a masked panel of 8 ophthalmologist reviewers were able to discern human- from chatbot-generated responses with 61% accuracy. Ratings of the quality of chatbot and human answers were not significantly different regarding inclusion of incorrect information, likelihood of harm caused, extent of harm, or deviation from ophthalmologist community standards. Meaning These results suggest ophthalmologists and a large language model may provide comparable quality of ophthalmic advice for a range of patient questions, regardless of their complexity.