UPDF AI

On collocations and topic models

Jey Han Lau,Timothy Baldwin,David Newman

2013 · DOI: 10.1145/2483969.2483972
69 Citations

TLDR

It is shown how the Akaike information criterion is a more appropriate measure, which suggests that using a modest number of top-ranked bigrams is the optimal topic modelling configuration, and that using these 1000 bigrams results in improved topic quality over unigram tokenization.