Answering the Call for a Standard Reliability Measure for Coding Data

This work proposes Krippendorff's alpha as the standard reliability measure, general in that it can be used regardless of the number of observers, levels of measurement, sample sizes, and presence or absence of missing data.