LSTM beats traditional models for Twitter sentiment with 92% ROC-AUC

A recent study by Atiq Ur Rehman, published in the 2025 IEEE Conference on Computing, Communication, and Data Engineering (C-CODE), evaluates multiple machine learning and deep learning approaches for sentiment analysis on Twitter. The research compares logistic regression, random forest, naïve Bayes, gradient boosting, and LSTM networks to classify tweets into positive, negative, or neutral sentiments. The dataset, sourced from Kaggle, was preprocessed using tokenization, lemmatization, and stopword removal to clean the text. The goal was to identify the optimal model for interpreting public opinion and forecasting trends in real-time social media data.

The results show that the LSTM model significantly outperforms traditional classifiers across all metrics. It achieved a training accuracy of 90.98%, a testing accuracy of 80.00%, and a micro-average ROC-AUC score of 0.92. These numbers demonstrate LSTM's ability to capture contextual and sequential dependencies in text, which simpler models like logistic regression and random forest miss. The study concludes that deep learning architectures are more effective for sentiment analysis on platforms like Twitter, where language is informal and context-dependent. This work provides a benchmark for future applications in social media monitoring, brand reputation management, and political trend analysis.