Sentiment Analysis Using Transformer Method Sentiment Analysis Using Transformer Method
Article Sidebar
Published
Feb 29, 2024
Main Article Content
Abstract
This research delves into sentiment analysis, employing the transformative capabilities of transformer method. Specifically, leveraging BERT Models, the study aims to advance sentiment classification accuracy by intricately capturing contextual nuances and positive or negative comment on IMDB movies reviews. The transformer architecture's distinctive attention mechanism proves pivotal in comprehending intricate relationships between words, facilitating a profound understanding of sentiment in textual data. Through extensive experimentation, the study establishes the transformative prowess of these methods, showcasing their effectiveness in achieving state-of-the-art results in sentiment analysis tasks. This investigation not only contributes to the evolving landscape of sentiment analysis but also underscores the significance of transformer-based approaches in deciphering the subtleties of human expression within textual data specially for Bert models. This research will predict sentiment analysis on comment of IMDB movies and shows some results which are 3% of loss, 60% off loss validation, 98% of accuracy and 90% of validation accuracy
Article Details
How to Cite
Aljabar, A., Ali, I., & Karomah, B. (2024). Sentiment Analysis Using Transformer Method. Journal of Informatics Information System Software Engineering and Applications (INISTA), 6(2), 90-97. https://doi.org/10.20895/inista.v6i2.1383
Section
Centive 2023
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Copyright Notice
Authors who publish with Journal of Informatics, Information System, Software Engineering and Applications (INISTA) agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (CC BY-SA 4.0) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
References
[1] J. Cheng, I. Fostiropoulos, B. Boehm, and M. Soleymani, “Multimodal Phased Transformer for Sentiment Analysis,” Association for Computational Linguistics. [Online]. Available: https://github.com/chengjunyan1/
[2] L. Zhu, Z. Zhu, C. Zhang, Y. Xu, and X. Kong, “Multimodal sentiment analysis based on fusion methods: A survey,” Information Fusion, vol. 95. Elsevier B.V., pp. 306–325, Jul. 01, 2023. doi: 10.1016/j.inffus.2023.02.028.
[3] A. Aljabar and A. A. Abd Karim, “ANALISIS SENTIMEN MENGGUNAKAN ALGORITMA LSTM PADA MEDIA SOSIAL,” JUPIKOM, vol. 1, no. 3, 2022, [Online]. Available: http://ejurnal.stie-trianandra.ac.id/index.php/jupkomHalamanUTAMAJurnal:http://ejurnal.stie-trianandra.ac.id/index.php
[4] D. Tri Hermanto, A. Setyanto, and E. T. Luthfi, “Algoritma LSTM-CNN untuk Sentimen Klasifikasi dengan Word2vec pada Media Online LSTM-CNN Algorithm for Sentiment Clasification with Word2vec On Online Media”.
[5] W. E. Nurjanah, R. Setya Perdana, and M. A. Fauzi, “Analisis Sentimen Terhadap Tayangan Televisi Berdasarkan Opini Masyarakat pada Media Sosial Twitter menggunakan Metode K-Nearest Neighbor dan Pembobotan Jumlah Retweet,” 2017. [Online]. Available: http://j-ptiik.ub.ac.id
[6] S. Palani, P. Rajagopal, and S. Pancholi, “T-BERT -- Model for Sentiment Analysis of Micro-blogs Integrating Topic Model and BERT,” Jun. 2021, [Online]. Available: http://arxiv.org/abs/2106.01097
[7] A. E. Yüksel, Y. A. Türkmen, A. Özgür, and A. B. Altınel, “Turkish tweet classification with transformer encoder,” in International Conference Recent Advances in Natural Language Processing, RANLP, Incoma Ltd, 2019, pp. 1380–1387. doi: 10.26615/978-954-452-056-4_158.
[8] J.-B. Delbrouck, N. Tits, M. Brousmiche, and S. Dupont, “A Transformer-based joint-encoding for Emotion Recognition and Sentiment Analysis,” Jun. 2020, doi: 10.18653/v1/2020.challengehml-1.1.
[9] Z. Yuan, W. Li, H. Xu, and W. Yu, “Transformer-based Feature Reconstruction Network for Robust Multimodal Sentiment Analysis,” in MM 2021 - Proceedings of the 29th ACM International Conference on Multimedia, Association for Computing Machinery, Inc, Oct. 2021, pp. 4400–4407. doi: 10.1145/3474085.3475585.
[10] Z. Wang, Z. Wan, and X. Wan, “TransModality: An End2End Fusion Method with Transformer for Multimodal Sentiment Analysis,” in The Web Conference 2020 - Proceedings of the World Wide Web Conference, WWW 2020, Association for Computing Machinery, Inc, Apr. 2020, pp. 2514–2520. doi: 10.1145/3366423.3380000.
[11] S. Tabinda Kokab, S. Asghar, and S. Naz, “Transformer-based deep learning models for the sentiment analysis of social media data,” Array, vol. 14, Jul. 2022, doi: 10.1016/j.array.2022.100157.
[12] S. Alaparthi and M. Mishra, “Bidirectional Encoder Representations from Transformers (BERT): A sentiment analysis odyssey.”
[13] Z. Jianqiang, G. Xiaolin, and Z. Xuejun, “Deep Convolution Neural Networks for Twitter Sentiment Analysis,” IEEE Access, vol. 6, pp. 23253–23260, Jan. 2018, doi: 10.1109/ACCESS.2017.2776930.
[14] L. Geni, E. Yulianti, and D. I. Sensuse, “Sentiment Analysis of Tweets Before the 2024 Elections in Indonesia Using IndoBERT Language Models,” Jurnal Ilmiah Teknik Elektro Komputer dan Informatika (JITEKI), vol. 9, no. 3, pp. 746–757, 2023, doi: 10.26555/jiteki.v9i3.26490.
[15] M. E. Basiri, S. Nemati, M. Abdar, E. Cambria, and U. R. Acharya, “ABCDM: An Attention-based Bidirectional CNN-RNN Deep Model for sentiment analysis,” Future Generation Computer Systems, vol. 115, pp. 279–294, Feb. 2021, doi: 10.1016/j.future.2020.08.005.
[16] A. Alsaeedi and M. Z. Khan, “A study on sentiment analysis techniques of Twitter data,” International Journal of Advanced Computer Science and Applications, vol. 10, no. 2, pp. 361–374, 2019, doi: 10.14569/ijacsa.2019.0100248.
[17] M. Boukabous and M. Azizi, “Crime prediction using a hybrid sentiment analysis approach based on the bidirectional encoder representations from transformers,” Indonesian Journal of Electrical Engineering and Computer Science, vol. 25, no. 2, pp. 1131–1139, Feb. 2022, doi: 10.11591/ijeecs.v25.i2.pp1131-1139.
[18] L. B. Hutama and D. Suhartono, “Indonesian Hoax News Classification with Multilingual Transformer Model and BERTopic,” Informatica (Slovenia), vol. 46, no. 8, pp. 81–90, 2022, doi: 10.31449/inf.v46i8.4336.
[2] L. Zhu, Z. Zhu, C. Zhang, Y. Xu, and X. Kong, “Multimodal sentiment analysis based on fusion methods: A survey,” Information Fusion, vol. 95. Elsevier B.V., pp. 306–325, Jul. 01, 2023. doi: 10.1016/j.inffus.2023.02.028.
[3] A. Aljabar and A. A. Abd Karim, “ANALISIS SENTIMEN MENGGUNAKAN ALGORITMA LSTM PADA MEDIA SOSIAL,” JUPIKOM, vol. 1, no. 3, 2022, [Online]. Available: http://ejurnal.stie-trianandra.ac.id/index.php/jupkomHalamanUTAMAJurnal:http://ejurnal.stie-trianandra.ac.id/index.php
[4] D. Tri Hermanto, A. Setyanto, and E. T. Luthfi, “Algoritma LSTM-CNN untuk Sentimen Klasifikasi dengan Word2vec pada Media Online LSTM-CNN Algorithm for Sentiment Clasification with Word2vec On Online Media”.
[5] W. E. Nurjanah, R. Setya Perdana, and M. A. Fauzi, “Analisis Sentimen Terhadap Tayangan Televisi Berdasarkan Opini Masyarakat pada Media Sosial Twitter menggunakan Metode K-Nearest Neighbor dan Pembobotan Jumlah Retweet,” 2017. [Online]. Available: http://j-ptiik.ub.ac.id
[6] S. Palani, P. Rajagopal, and S. Pancholi, “T-BERT -- Model for Sentiment Analysis of Micro-blogs Integrating Topic Model and BERT,” Jun. 2021, [Online]. Available: http://arxiv.org/abs/2106.01097
[7] A. E. Yüksel, Y. A. Türkmen, A. Özgür, and A. B. Altınel, “Turkish tweet classification with transformer encoder,” in International Conference Recent Advances in Natural Language Processing, RANLP, Incoma Ltd, 2019, pp. 1380–1387. doi: 10.26615/978-954-452-056-4_158.
[8] J.-B. Delbrouck, N. Tits, M. Brousmiche, and S. Dupont, “A Transformer-based joint-encoding for Emotion Recognition and Sentiment Analysis,” Jun. 2020, doi: 10.18653/v1/2020.challengehml-1.1.
[9] Z. Yuan, W. Li, H. Xu, and W. Yu, “Transformer-based Feature Reconstruction Network for Robust Multimodal Sentiment Analysis,” in MM 2021 - Proceedings of the 29th ACM International Conference on Multimedia, Association for Computing Machinery, Inc, Oct. 2021, pp. 4400–4407. doi: 10.1145/3474085.3475585.
[10] Z. Wang, Z. Wan, and X. Wan, “TransModality: An End2End Fusion Method with Transformer for Multimodal Sentiment Analysis,” in The Web Conference 2020 - Proceedings of the World Wide Web Conference, WWW 2020, Association for Computing Machinery, Inc, Apr. 2020, pp. 2514–2520. doi: 10.1145/3366423.3380000.
[11] S. Tabinda Kokab, S. Asghar, and S. Naz, “Transformer-based deep learning models for the sentiment analysis of social media data,” Array, vol. 14, Jul. 2022, doi: 10.1016/j.array.2022.100157.
[12] S. Alaparthi and M. Mishra, “Bidirectional Encoder Representations from Transformers (BERT): A sentiment analysis odyssey.”
[13] Z. Jianqiang, G. Xiaolin, and Z. Xuejun, “Deep Convolution Neural Networks for Twitter Sentiment Analysis,” IEEE Access, vol. 6, pp. 23253–23260, Jan. 2018, doi: 10.1109/ACCESS.2017.2776930.
[14] L. Geni, E. Yulianti, and D. I. Sensuse, “Sentiment Analysis of Tweets Before the 2024 Elections in Indonesia Using IndoBERT Language Models,” Jurnal Ilmiah Teknik Elektro Komputer dan Informatika (JITEKI), vol. 9, no. 3, pp. 746–757, 2023, doi: 10.26555/jiteki.v9i3.26490.
[15] M. E. Basiri, S. Nemati, M. Abdar, E. Cambria, and U. R. Acharya, “ABCDM: An Attention-based Bidirectional CNN-RNN Deep Model for sentiment analysis,” Future Generation Computer Systems, vol. 115, pp. 279–294, Feb. 2021, doi: 10.1016/j.future.2020.08.005.
[16] A. Alsaeedi and M. Z. Khan, “A study on sentiment analysis techniques of Twitter data,” International Journal of Advanced Computer Science and Applications, vol. 10, no. 2, pp. 361–374, 2019, doi: 10.14569/ijacsa.2019.0100248.
[17] M. Boukabous and M. Azizi, “Crime prediction using a hybrid sentiment analysis approach based on the bidirectional encoder representations from transformers,” Indonesian Journal of Electrical Engineering and Computer Science, vol. 25, no. 2, pp. 1131–1139, Feb. 2022, doi: 10.11591/ijeecs.v25.i2.pp1131-1139.
[18] L. B. Hutama and D. Suhartono, “Indonesian Hoax News Classification with Multilingual Transformer Model and BERTopic,” Informatica (Slovenia), vol. 46, no. 8, pp. 81–90, 2022, doi: 10.31449/inf.v46i8.4336.