From data to prediction: development and clinical validation of a preterm birth risk assessment tool based on machine learning technologies

Introduction

. Preterm birth (PTB) remains one of the most serious complications of pregnancy, being the leading cause of neonatal mortality and contributing to long-term disability along with chronic morbidity in newborns, as well as imposing substantial socioeconomic costs. Despite preventive efforts, the global PTB rate has remained largely unchanged comprising 5–18 %, underscoring a need for developing more effective prediction tools to enable timely prevention.

Aim

: using an independent sample to develop and validate a PTB risk-assessment tool based on machine learning (ML) and routinely collected clinical data retrieved from electronic health records (EHRs) of pregnant patients.

Materials and Methods

. We analyzed a dataset of 10,000 de-identified EHRs entries containing 54 variables, including historical, clinical, laboratory, and instrumental (diagnostic/imaging) data. The predictive system comprised two interconnected ML components: (1) an NLP model based on RuBERT (а pre-trained ML model for processing Russian texts) for extracting PTB-relevant features from unstructured Russian-language clinical text, and (2) a downstream predictive ML model, for which 14 algorithms were benchmarked.

Results

. The NLP model demonstrated high performance with a median sensitivity = 0.998, F1-score = 0.976, and AUC-ROC = 0.974. Among the ML algorithms, the algorithm based on gradient boosting – CatBoost classifier (Categorical Boosting Classifier) achieved the best risk-prediction results: accuracy = 0.81, sensitivity (recall) = 0.87, precision = 0.76, F1-score = 0.81, and AUC-ROC = 0.82.

Conclusion

. The developed model showed performance comparable to that of international counterparts, and validation confirmed its robustness to previously unseen data, indicating strong potential for use in routine clinical practice. This study represents the first step toward an integrated PTB risk-assessment solution combining NLP and ML. Future work will include incorporation of additional predictors (e.g., biochemical markers) and multicenter validation studies.