Assessment of AI-Driven Large Language Models for Orthodontic Aesthetic Scoring Using the IOTN-AC
Ahmet Yıldırım, Orhan Çiçek
Abstract
Background/Objectives: The aim of this study was to evaluate the accuracy of aesthetic assessments performed by artificial intelligence (AI)-based large language models (LLMs) using the Aesthetic Component of the Index of Orthodontic Treatment Need (IOTN-AC), which is widely applied to determine the need for orthodontic treatment. Methods: A total of 150 frontal intraoral photographs from patients in the permanent dentition, scored from 1 to 10 on the IOTN-AC, were assessed by two AI-based LLMs (ChatGPT-5 and ChatGPT-5 Pro). Two experienced clinicians independently scored all photographs, with one evaluator’s scores used as the reference (κ = 0.91, ICC = 0.88). Model performance was analyzed by comparing IOTN-AC scores and treatment need classifications. In addition, performance parameters such as accuracy, precision, specificity, and sensitivity were evaluated. Statistical analyses included Spearman correlation, Cohen’s Kappa, ICC, Mean Absolute Error (MAE), Wilcoxon signed-rank test, and Bland–Altman analysis. Results: Both models demonstrated positive and significant correlations with the reference values for scoring and classification (p < 0.001). Compared to GPT-5 Pro, the GPT-5 model exhibited superior performance, with a lower error rate (MAE = 1.47) and higher classification accuracy (66.7%). Bland–Altman analysis showed that most predictions fell within the 99% confidence interval, and regression analysis revealed no systematic bias (p > 0.05). Conversely, the models failed to achieve consistently high performance in each of the performance parameters. Conclusions: The findings revealed that although AI-based LLMs are promising, statistical accuracy alone is insufficient for safe clinical use, and they should demonstrate consistently high performance across all parameters.