Teachers' Framework in Assessing Quality of MOOCs

The quality of MOOCs has long been debated, especially regarding learner retention. While many models exist, practical tools for assessing course design fundamentals from a teacher’s perspective are rare. This study introduces a user-friendly, binary evaluation tool focusing on pedagogical and instructional design, tested on 30 Coursera and Udemy MOOCs across three subjects. Results reveal poor learner engagement and unclear outcomes, highlighting the need to improve course creators’ pedagogical skills. The study also shows non-experts can reliably assess Math MOOCs, but English MOOCs require subject expertise. Agreement between raters in the study has been analysed using various agreement coefficients: percent agreement, Light’s kappa and Gwet’s AC1. Overall, the findings emphasize that evaluating core course design is key to gauging MOOC quality.