Improving Fake News Detection with Advanced Pre-Trained Models and Statistical Analysis

Identifying fabricated information is one of the critical tasks that can be completed to fight misinformation. This paper represents a detailed review of transformer-based, DL and ML models to differentiate between fake and real news. The authors employed the dataset obtained from the Fake News Detection System GitHub repository; in addition to employing traditional preprocessing (i.e., stopwords elimination, BERT Tokenizer) the authors created the stratified subsets for training, validation and testing datasets. Using PyTorch framework to complete experiments, the authors determined that the optimal hyperparameter configurations for the models were as follows: SVM, CNN, BiLSTM with attention, BERT, and DistilBERT for the detection of fake news. When comparing traditional models to deep learning-based models, the performance of transformer-based models clearly outperformed both groups of models regarding standard metrics (P, R, A and F1 Score). Specifically, DistilBERT had the highest accuracy at 81.00% relative to all of the models, and models tested by this study. Additionally, the results of the authors were substantiated and supported through additional analyses of the confusion matrices and Chi-Square Statistical Test Results. Furthermore, this study provides insight into future studies aimed at increasing model interpretability, scalability, and the practicality of real-time model deployment; as well as demonstrating the efficacy of transformers for detecting fake news.