Do AI Chatbots Tell the Truth About Dentin Hypersensitivity? A Comparative Evaluation of Quality, Accuracy, and Readability

Seval Ceylan Şen; Özlem Saraç Atagün; Gülbahar Ustaoğlu; Zeynep Hazan Yildiz; Rumeysa Nur Kayacı

doi:10.7126/cumudj.1848545

TR EN

Yapay zekâ sohbet robotları dentin hipersensitivitesi hakkında doğru bilgi veriyor mu? Kalite, doğruluk ve okunabilirliğin karşılaştırmalı değerlendirmesi

Öz

Amaç: Dentin hassasiyeti (DH), sık karşılaşılan bir dental yakınma olup, günümüzde birçok hasta bu konuda bilgi edinmek amacıyla yapay zekâ (YZ) tabanlı sohbet botlarına başvurmaktadır. Ancak, sohbet botları tarafından üretilen DH içeriklerinin doğruluğu, güvenilirliği ve okunabilirliği henüz netlik kazanmamıştır. Yöntemler: Uzman görüşüne dayalı olarak oluşturulan DH soru seti, üç farklı YZ sohbet botuna (ChatGPT-4o, DeepSeek ve Copilot) bağımsız ve standartlaştırılmış oturumlarda yöneltilmiştir. Elde edilen yanıtlar, üç körlenmiş periodontolog tarafından CLEAR kriterleri, modifiye Global Kalite Skoru (mGQS), doğruluk puanlaması, DISCERN aracı ve okunabilirlik ölçütleri (Flesch Reading Ease [FRE], Flesch–Kincaid Grade Level [FKGL]) kullanılarak değerlendirilmiştir. Modeller arası karşılaştırmalar için parametrik olmayan istatistiksel testler uygulanmıştır. Bulgular: FKGL (p=0,025), DISCERN puanları (p=0,004) ve yanıt uzunluğu (p<0,001) açısından YZ modelleri arasında istatistiksel olarak anlamlı farklar saptanmıştır. Copilot, en yüksek güvenilirlik ve kalite düzeyini gösterirken; DeepSeek en okunabilir içeriği üretmiştir. ChatGPT ise en yüksek değişkenliği sergilemiştir. Ayrıca Copilot, tamamen doğru ve yüksek kaliteli yanıtların en yüksek oranına sahipken, düşük doğruluk düzeyindeki yanıtlar yalnızca ChatGPT’de gözlenmiştir. Doğruluk, içerik bütünlüğü ve genel kalite arasında güçlü korelasyonlar tespit edilmiştir. Sonuç: YZ tabanlı sohbet botları, dentin hassasiyeti hakkında klinik açıdan anlamlı bilgiler üretebilmekle birlikte, performansları modeller arasında farklılık göstermektedir. Copilot, doğruluk ve güvenilirlik açısından en dengeli performansı sergilerken; DeepSeek daha erişilebilir bir dil kullanımı sunmuş, ChatGPT ise tutarsız sonuçlar göstermiştir. Hasta eğitimi amacıyla YZ tarafından üretilen içeriklerin kullanımında klinisyen denetimi hâlen büyük önem taşımaktadır.

Anahtar Kelimeler

Do AI Chatbots Tell the Truth About Dentin Hypersensitivity? A Comparative Evaluation of Quality, Accuracy, and Readability

Abstract

Aim: Dentin hypersensitivity (DH) is a common dental complaint, and many patients now seek information from AI chatbots. Yet, the accuracy, reliability, and readability of chatbot-generated DH content remain uncertain. Materials and Methods: A consensus-based DH question set was presented to three AI chatbots (ChatGPT-4o, DeepSeek, Copilot) in independent, standardized sessions. Three blinded periodontologists evaluated the responses using CLEAR, mGQS, accuracy scores, DISCERN, and readability metrics (FRE, FKGL). Non-parametric tests compared inter-model differences. Results: Significant differences were found in FKGL (p=0.025), DISCERN (p=0.004), and response length (p<0.001). Copilot yielded the highest reliability and quality, DeepSeek produced the most readable content, and ChatGPT showed the greatest variability. Copilot also had the highest proportion of fully accurate and high-quality answers, whereas low-accuracy outputs occurred only in ChatGPT. Strong correlations were noted among accuracy, completeness, and overall quality. Conclusions: AI chatbots can generate clinically relevant DH information, but performance varies. Copilot showed the best balance of accuracy and reliability, DeepSeek provided the most accessible language, and ChatGPT demonstrated inconsistent results. Clinician oversight remains essential when using AI-generated content for patient education.

Keywords

Supporting Institution

The authors did not receive support from any organization for the submitted work.

Ethical Statement

The present study is a public application, and there are no human/animal participants; therefore, ethics committee approval was not required.

Thanks

None

References

1. Holland GR, Narhi MN, Addy M, Gangarosa L, Orchardson R. Guidelines for the design and conduct of clinical trials on dentine hypersensitivity. J Clin Periodontol 1997;24(11):808-813.
2. Eyuboglu GB, Kalay T. The effects of different desensitizers and their combinations with Er, Cr: Ysgg laser on dentin tubules, and shear bond strength to dentin. Cumhuriyet Dent J 2022;25:47-56.
3. Addy M. Tooth brushing, tooth wear and dentine hypersensitivity--are they associated? Int Dent J 2005;55(4):261-267.
4. Gillam D, Orchardson R. Advances in the treatment of root dentin sensitivity: mechanisms and treatment principles. Endod Topics 2006;13:13-33.
5. Gibson B, Boiko O, Robinson P, Robinson PG, Barlow A, Player T, et al. The everyday impact of dentine sensitivity. Social Science and Dentistry. 06/01 2010;1:11-20.
6. Davari A, Ataei E, Assarzadeh H. Dentin hypersensitivity: etiology, diagnosis and treatment; a literature review. J Dent (Shiraz) 2013;14(3):136-145.
7. Liu XX, Tenenbaum HC, Wilder RS, Quock R, Hewlett ER, Ren YF. Pathogenesis, diagnosis and management of dentin hypersensitivity: an evidence-based overview for dental practitioners. BMC Oral Health2020;20(1):220.
8. Miglani S, Aggarwal V, Ahuja B. Dentin hypersensitivity: Recent trends in management. J Conserv Dent 2010;13(4):218-24.

9. Moor J. The Dartmouth College Artificial Intelligence Conference: The Next Fifty Years. AI Magazine 2006;27:87-91.
10. Mintz Y, Brodie R. Introduction to artificial intelligence in medicine. Minim Invasive Ther Allied Technol 2019;28(2):73-81.
11. Ding H, Wu J, Zhao W, et al. Artificial intelligence in dentistry-A review. Front Dent Med 2023;4:1085251.
12. Yilmaz B, Gökkurt Yilmaz B, Ozbey F. Artificial intelligence performance in answering multiple-choice oral pathology questions: a comparative analysis. BMC Oral Health 2025;25(1);573.
13. Bindra S, Jain R. Artificial intelligence in medical science: a review. Irish journal of medical science 2024;93(3):1419-1429
14. Tayman M. Validity and reliability of responses to periodontology questions by 4 different artificial intelligence chatbots as public information sources. Cumhuriyet Dent J 2025;28:390-396.
15. Nardi GM, Sabatini S, Acito G, Colavito A, Chiavistelli L, Campus G. The decision tree for clinical management of dentin hypersensitivity. a consensus report. Oral Health Prev Dent 2022;20:27-32.
16. Faul F, Erdfelder E, Buchner A, Lang AG. Statistical power analyses using G*Power 3.1: tests for correlation and regression analyses. Behav Res Methods 2009;41(4):1149-1160.
17. Esmailpour H, Rasaie V, Babaee Hemmati Y, Falahchai M l. Performance of artificial intelligence chatbots in responding to the frequently asked questions of patients regarding dental prostheses. BMC Oral Health 2025;25(1):574.
18. Sallam M, Al-Salahat K, Eid H, Egger J, Puladi B. Human versus artificial intelligence: ChatGPT-4 outperforming bing, bard, ChatGPT-3.5 and humans in clinical chemistry multiple-choice questions. Adv Med Educ Pract 2024;15:857-871.
19. Bernard A, Langille M, Hughes S, Rose C, Leddin D, Veldhuyzen van Zanten S. A systematic review of patient inflammatory bowel disease information resources on the World Wide Web. Am J Gastroenterol 2007;102(9):2070-2077.
20. Hatia A, Doldo T, Parrini S, Chisci E, Cipriani L, Montagna L, et al. Accuracy and completeness of ChatGPT-Generated Information on interceptive orthodontics: a multicenter collaborative study. J Clin Med 2024;13(3):735.
21. Helvacioglu-Yigit D, Demirturk H, Ali K, Tamimi D, Koenig L, Almashraqi A. Evaluating artificial intelligence chatbots for patient education in oral and maxillofacial radiology. Oral Surg Oral Med Oral Pathol Oral Radiol 2025;139(6):750-759.
22. Medicine NLo. How to write easy-to-read health materials.Updated November 2020. Accessed December 12, 2022. Available at: https://medlineplus.gov/all_easytoread.html.
23. Kılınç DD, Mansız D. Examination of the reliability and readability of Chatbot Generative Pretrained Transformer's (ChatGPT) responses to questions about orthodontics and the evolution of these responses in an updated version. Am J Orthod Dentofacial Orthop 2024;165(5):546-555.
24. Meade MJ, Dreyer CW. Web-based information on orthodontic clear aligners: a qualitative and readability assessment. Aust Dent J 2020;65(3):225-232.
25. Stvilia B, Mon L, Yi Y. A Model for Online Consumer Health Information Quality. JASIST 2009;60:1781-1791.
26. Onder CE, Koc G, Gokbulut P, Taskaldiran I, Kuskonmaz SM. Evaluation of the reliability and readability of ChatGPT-4 responses regarding hypothyroidism during pregnancy. Sci Rep 2024;14(1):243.
27. Saraç Atagün Ö, Ceylan Şen S, Paksoy T. Analysis of YouTube videos as a source of information about dentin hypersensitivity. Int J Dent Hyg 2024;22(2):432-443.
28. Alqutaibi AY, Algabri RS, Alamri AS, Alhazmi LS, Almadani SM, Alturkistani AM, et al. Advancements of artificial intelligence algorithms in predicting dental implant prognosis from radiographic images: a systematic review. J Prosthet Dent 2025;134(6):2177-2188
29. Mertens S, Krois J, Cantu AG, Arsiwala LT, Schwendicke F.l. Artificial intelligence for caries detection: randomized trial. J Dent 2021;115:103849.
30. Thorat V, Rao P, Joshi N, Talreja P, Shetty AR. Role of Artificial Intelligence (AI) in patient education and communication in dentistry. Cureus 2024;16(5):e59799.
31. Mugri MH. Accuracy of artificial intelligence models in detecting peri-implant bone loss: a systematic review. Diagnostics (Basel) 2025;15(6):655
32. Terzi M, Yavuz MC, Bicer T, Buyuk SK. Evaluation of artificial intelligence robot's knowledge and reliability on dental implants and peri-implant phenotype. Sci Rep 2025;15(1):9519.
33. Çoban E, Altay B. ChatGPT may help inform patients in dental implantology. Int J Oral Maxillofac Implants 2024;39(5):203-208.
34. Ibraheem WI. Accuracy of artificial intelligence models in dental implant fixture identification and classification from radiographs: a systematic review. Diagnostics (Basel) 2024;14(8):806.
35. Abu Arqub S, Al-Moghrabi D, Allareddy V, Upadhyay M, Vaid N, Yadav S. Content analysis of AI-generated (ChatGPT) responses concerning orthodontic clear aligners. Angle Orthod 2024;94(3):263-272.
36. Özcivelek T, Özcan B. Comparative evaluation of responses from DeepSeek-R1, ChatGPT-o1, ChatGPT-4, and dental GPT chatbots to patient inquiries about dental and maxillofacial prostheses. BMC Oral Health 2025;25(1):871.
37. Büker M, Mercan G. Readability, accuracy and appropriateness and quality of AI chatbot responses as a patient information source on root canal retreatment: A comparative assessment. Int J Med Inform. 2025;201:105948.

Details

Primary Language

English

Subjects

Periodontics

Journal Section

Research Article

Authors

Seval Ceylan Şen ^*
0000-0002-3286-7819
Türkiye

Özlem Saraç Atagün
0000-0002-2964-8244
Türkiye

Gülbahar Ustaoğlu
0000-0002-4205-861X
Türkiye

Zeynep Hazan Yildiz
0009-0004-6222-420X
Türkiye

Rumeysa Nur Kayacı
0009-0000-8753-5806
Türkiye

Publication Date

March 27, 2026

Submission Date

December 24, 2025

Acceptance Date

January 31, 2026

Published in Issue

Year 2026 Volume: 29 Number: 1

DOI

https://doi.org/10.7126/cumudj.1848545

IZ

https://izlik.org/JA54DN86AM

Cite

RIS / Bibtex

EndNote

Ceylan Şen S, Saraç Atagün Ö, Ustaoğlu G, Yildiz ZH, Kayacı RN (March 1, 2026) Do AI Chatbots Tell the Truth About Dentin Hypersensitivity? A Comparative Evaluation of Quality, Accuracy, and Readability. Cumhuriyet Dental Journal 29 1 138–147.