Stock Market Prediction using LSTM Model on the News and Social Media Data

Accurately predicting future trends in stock market is essential for investors because it increases the chances of a successful investment in the market. However, making precise predictions is challenging due to stock market volatility and influence of external factors from news and social media. Although various machine and deep learning techniques have been used to predict stock markets, none of them have been evaluated for short and medium-term forecasting. Therefore, we propose time series prediction method called Long Short-Term Memory (LSTM) to forecast stock markets over the short and medium terms, utilizing data sets with external variables. The proposed model is compared with baseline models including Multilayer Perceptron (MLP), Support Vector Machine (SVM), and K-Nearest Neighbor (KNN). The external features are collected from the news and social media after preprocessing and feature engineering of the textual data. The models are then applied on historical, social media, and financial news data from HPQ, IBM, ORCL, and MSFT stock markets. The experimental results demonstrate that LSTM performs best for medium-term predictions, with maximum accuracies of 81.5% and 87.5% on social media and news data, respectively, while MLP shows best performance for short-term predictions.