Design and validation of a diagnostic MOOC evaluation method combining AHP and text mining algorithms

ABSTRACT The proliferation of massive open online courses (MOOCs) highlights the necessity of developing accurate and diagnostic evaluation methods to assess the courses’ quality and effectiveness. Hence, this study proposes a diagnostic MOOC evaluation (DME) method that combines the Analytic Hierarchy Process algorithm and learner review mining to integrate expert opinions, standardized rubrics, and learner feedback into the evaluation process. For this purpose, 30 MOOCs from the Coursera website were purposively selected and evaluated using the DME method and the results compared with expert evaluation and learner rating scores. The preliminary findings, in general, support the feasibility, accuracy, and diagnostic utility of the DME method and its suitability as a low-cost, sophisticated, and accurate method for MOOC evaluation. Finally, the study discusses several limitations and technical issues of the DME method that should be addressed in future research and practice.