Sentiment Classification of User Reviews for KAI Access Application Using Naive Bayes Method

Abstract

KAI Access is a train ticket booking application that offers convenience and various features for its users. However, the app has received a low rating of 2.4 out of 5 stars on the Play Store, indicating user dissatisfaction. This study conducts a quantitative sentiment analysis of the KAI Access application based on user sentiments expressed on Twitter. Using the CRISP-DM method, data were collected from Twitter with the Tweepy tool, amassing around 4,000 tweets from June to August 2023. The data underwent a preprocessing stage to ensure the quality and accuracy of the analysis. This stage involved removing duplicate tweets, eliminating retweets, and filtering out emoticons and other non-text elements. In the modeling stage, the Multinomial Naive Bayes Classifier algorithm was employed, achieving an accuracy rate of 84.6%. The model performed better at identifying negative reviews, with a precision of 0.96, recall of 0.86, and an F1-score of 0.91. In contrast, the identification of positive reviews was less effective, with a precision of 0.41, recall of 0.75, and an F1-score of 0.53. These findings shed light on the low ratings for KAI Access, particularly in the context of user reviews. The results of this study provide further understanding regarding the low rating given to KAI Access, particularly in the context of user reviews. By using this classification system, it is hoped that developers can design more specific improvements to enhance the user experience, especially in handling positive reviews which have the potential for performance improvement.