Towards Trustworthy Automated Essay Scoring: Explainable Transformers and Efficient Fine-Tuning Strategies

Artificial intelligence allowed instructors to automate the grading of students’ work. Many studies show the high performance of neural networks in automated grading. However, their “Blackbox” nature reduces the educators’ trust in adopting it in the educational context. In this study, we propose a framework for automated essay scoring using Transformer-based models and Explainable AI (XAI). We evaluated the five models on two different benchmarking datasets, ASAP 2.0 and Learning Agency Lab, on models: Linear Regression, Random Forest, BiLSTM with attention, BERT, and FLAN-T5. Our studies show the BERT model achieved a Quadratic Weighted Kappa (QWK) score of 0.8431, which is 19.5% better than traditional machine learning models. Moreover, the study also shows the cross-prompt generalisation with 70.3% QWK degradation in leave-one-prompt-out evaluation, but it improves when training data contains diverse prompts. For resource optimisation, we apply Parameter-Efficient Fine-Tuning (PEFT) using LoRA on FLAN-T5. This approach achieves 86.2% of BERT’s Pearson correlation while training only 1.8% of parameters. Additionally, we integrated LIME and SHAP to provide pedagogically transparent explanations across all model families. Overall, the results show that training strategies play a more important role than model architecture, and parameter-efficient methods provide practical solutions for real educational systems.