Development and internal validation of reduced-feature machine-learning models for identifying severe periodontitis from routine clinical records

Abstract

Background: Machine-learning classifiers that include clinical attachment loss (CAL), probing pocket depth (PPD), or other components of the periodontal staging reference standard are vulnerable to incorporation bias and may overstate clinical utility. We therefore reframed the analysis around a reduced-feature model that excludes all direct staging measurements. Aim: To develop and internally validate reduced-feature logistic regression, Random Forest, and XGBoost models for identifying Stage III/IV periodontitis from routine non-staging clinical-record variables, and to quantify the performance inflation produced by adding stage-related periodontal measurements. Methods: This retrospective observational study included 482 unique adults with adjudicated 2017 World Workshop staging. The primary models used age, sex, current smoking, diabetes, and plaque index only; PPD, CAL, bleeding on probing (BOP), radiographic bone loss, periodontitis-related tooth loss, and complexity factors were excluded. A patient-level stratified 80:20 split yielded 385 training patients and 97 held-out test patients. All preprocessing and hyperparameter tuning occurred within training data. Test-set AUC, accuracy, sensitivity, specificity, precision, F1-score, and Brier score were accompanied by 95% confidence intervals from 5,000 paired nonparametric patient-level bootstrap resamples. A secondary full-record benchmark added BOP, mean PPD, and maximum CAL solely to quantify incorporation-related performance inflation. Results: Stage III/IV periodontitis was present in 158/482 patients (32.8%). In the primary reduced-feature analysis, XGBoost showed the strongest held-out discrimination: AUC 0.841 (95% CI 0.755-0.913), accuracy 77.3% (69.1-85.6), sensitivity 75.0% (59.4-89.2), specificity 78.5% (68.1-88.1), precision 63.2% (47.2-78.6), F1-score 0.686 (0.548-0.800), and Brier score 0.163 (0.132-0.196). Adding BOP, PPD, and CAL increased XGBoost AUC to 0.945 (0.893-0.983), an absolute paired-bootstrap increase of 0.103 (0.011-0.205), confirming substantial performance inflation when stage-related measurements are included. Conclusion: A reduced-feature XGBoost model provided moderate internal discrimination for identifying patients with severe periodontitis without using the measurements that define staging. Its potential role is limited to pre-staging triage or case-finding support. Once complete periodontal staging measurements are available, deterministic application of the 2017 World Workshop criteria remains the appropriate method for stage assignment.

Keywords: Periodontitis, Incorporation bias, XGBoost, Clinical decision support, Electronic dental records, Severity classification