Performance Evaluation of Transformer-based NLP Models on Fake News Detection Datasets

Fake news has become a major concern due to its spread on social media. To combat this, various machine learning (ML) techniques have been proposed. However, there is a lack of research on the performance of transformer models using datasets from a wide range of domains. This paper investigates the performance of ML algorithms on three fake news datasets: LIAR, FNC-1 and Balanced Dataset for Fake News Analysis. Pretrained transformer language models such as BERT, RoBERTa, ALBERT and DistilBERT were chosen for this paper. The performance of the models was consistent across all datasets. RoBERTa obtained an accuracy of 69% when trained on the LIAR dataset, an 11% improvement over the existing traditional and deep learning ML model implementations, and an accuracy of 97% when trained on the FNC-1 dataset, proving to be the best-performing model across all the fake news detection datasets utilized in the experiments. DistilBERT trains at a significantly faster rate than the other three variants. The experimental results from the paper can help the research community to continue investigating and gain insights into fake news detection.