Sentiment Analysis of Machine Learning Algorithms: A Transformer-Based Approach

Saad Iqbalbhai Mandli

Vol. 11, Issue 1, Jan-Dec 2025

Page Number: 76 - 81

Abstract:

Sentiment analysis is one of the most widely used applications of Natural Language Processing because it helps convert unstructured opinions into measurable insights. Traditional machine learning algorithms such as Naive Bayes, Logistic Regression, Support Vector Machine, Random Forest and Gradient Boosting have been used extensively for classifying reviews, tweets, feedback and news articles into positive, negative or neutral categories. However, these methods depend heavily on manual feature engineering, bag-of-words, TF-IDF vectors and domain-specific pre-processing. Deep learning methods improved performance by learning distributed representations, but recurrent and convolutional architectures still struggle with long-range context, sarcasm, negation and aspect-level sentiment. Transformer-based models, especially BERT and its variants, address many of these limitations through self-attention, contextual embeddings and transfer learning. This paper studies sentiment analysis from the perspective of machine learning algorithms and proposes a transformer-based approach for improving classification accuracy, contextual understanding and adaptability across domains. A comparative analysis is presented between traditional machine learning, deep learning and transformer-based models. The paper concludes that transformer-based models provide superior contextual representation and generalization, although they require higher computational resources and careful fine-tuning.

References

• Ali, H., Hashmi, E., Yayilgan Yildirim, S., & Shaikh, S. (2024). Analyzing Amazon products sentiment: A comparative study of machine learning, deep learning, and transformer-based techniques. Electronics, 13(7), 1305. https://doi.org/10.3390/electronics13071305
• Barbieri, F., Camacho-Collados, J., Espinosa Anke, L., & Neves, L. (2020). TweetEval: Unified benchmark and comparative evaluation for tweet classification. In Findings of the Association for Computational Linguistics: EMNLP 2020 (pp. 1644–1650). https://doi.org/10.18653/v1/2020.findings-emnlp.148
• Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2019) (pp. 4171–4186). https://doi.org/10.18653/v1/N19-1423
• Han, K.-X., Chien, W., Chiu, C.-C., & Cheng, Y.-T. (2020). Application of support vector machine in the sentiment analysis of Twitter dataset. Applied Sciences, 10(3), 1125. https://doi.org/10.3390/app10031125
• Perikos, I., & Diamantopoulos, A. (2024). Explainable aspect-based sentiment analysis using transformer models. Big Data and Cognitive Computing, 8(11), 141. https://doi.org/10.3390/bdcc8110141
• Pilicita Garrido, A., & Barra, E. (2025). Sentiment analysis with transformers applied to education: Systematic review. International Journal of Interactive Multimedia and Artificial Intelligence, 9(2), 72–83. https://doi.org/10.9781/ijimai.2025.02.008
• Sun, C., Huang, L., & Qiu, X. (2019). Utilizing BERT for aspect-based sentiment analysis via constructing auxiliary sentence. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2019) (pp. 380–385). https://doi.org/10.18653/v1/N19-1035
• Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems (Vol. 30). Curran Associates, Inc.
• Wang, J., Wei, J., Tian, F., et al. (2024). A comparative study of machine learning models for sentiment analysis of transboundary rivers news media articles. Soft Computing, 28, 13331–13347. https://doi.org/10.1007/s00500-024-10357-2
• Wankhade, M., Rao, A. C. S., & Kulkarni, C. (2022). A survey on sentiment analysis methods, applications, and challenges. Artificial Intelligence Review, 55, 5731–5780. https://doi.org/10.1007/s10462-022-10144-1
• Xu, H., Shu, L., Yu, P., & Liu, B. (2020). Understanding pre-trained BERT for aspect-based sentiment analysis. In Proceedings of the 28th International Conference on Computational Linguistics (COLING 2020) (pp. 244–250). https://doi.org/10.18653/v1/2020.coling-main.21
• Zhang, L., Wang, S., & Liu, B. (2018). Deep learning for sentiment analysis: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 8(4), Article e1253. https://doi.org/10.1002/widm.1253

Back Download