Comparing the Accuracy of Examinee Ability Estimation Using Machine Learning and Multidimensional Item Response Theory

This study compares the performance of multidimensional item response theory (MIRT) and machine learning (ML) methods in predicting students’ latent abilities. Using a series of simulation experiments, it evaluates how these models perform under varying ability structures, item characteristics, and data conditions. The simulation conditions include correlations among examinees’ latent abilities, the form of the ability distribution, the degree of item cross-loading, model misspecification in ability estimation, and the presence of missing data mechanisms. Findings indicate that MIRT generally exhibits strong performance and offers several advantages; however, when its underlying assumptions are violated, ML approaches, particularly tree-based methods such as Random Forest and XGBoost, outperform MIRT due to their nonparametric flexibility. Overall, the study affirms the robustness of MIRT in well-specified measurement contexts while highlighting the value of ML methods in scenarios where traditional IRT assumptions may not hold.