A Comparative Study of TF-IDF, BERT and Gemma Representations for Fake News Detection

The proliferation of fake news has spurred research into employing machine and deep learning techniques for its detection. Researchers have focused on detecting fake news within the field of NLP, leveraging various deep learning text representations to combat this growing threat and protect online discourse. This study investigates the efficacy of various deep learning embeddings, including TF-IDF, BERT, and Gemma, in conjunction with six classification algorithms (SVM, Naive Bayes, Random Forest, Decision Tree, LSTM, K-NN, and XGBoost) for accurate fake news identification. Experiments are conducted on publicly available Fake News Detection Datasets. Performance is assessed based on Accuracy, Precision, Recall, F1 score, and Inference Timings. The findings reveal that SVM classification faces difficulties when paired with advanced tokenizers due to the high-dimensional input embeddings. These tokenizers capture subtle linguistic cues that reveal deception, adding a complexity that surpasses the capabilities of SVM classifiers. By exploring feature engineering techniques, this study aims to identify the most effective and efficient solution for fake news detection.