Logo PTI
Polish Information Processing Society
Logo FedCSIS

Annals of Computer Science and Information Systems, Volume 8

Proceedings of the 2016 Federated Conference on Computer Science and Information Systems

Predicting Star Ratings based on Annotated Reviews of Mobile Apps

,

DOI: http://dx.doi.org/10.15439/2016F141

Citation: Proceedings of the 2016 Federated Conference on Computer Science and Information Systems, M. Ganzha, L. Maciaszek, M. Paprzycki (eds). ACSIS, Vol. 8, pages 421–428 ()

Full text

Abstract. This paper presents and evaluates different computational models for review rating prediction. The models rely solely on star ratings from an annotated corpus of customer reviews of mobile apps that were collected from the Google Play Store in a related work. Fine-granular opinions and the classification of their sentiment orientation were already available. The models build upon them to make predictions based on their polarity. Predicting star ratings is of importance to the sentiment analysis community because it can better be understood how customers subjectively rate products. Rating them consistently with corresponding written reviews, however, remains a difficult task for automated predictors. This paper sheds new light in that direction.

References

  1. A. Walz and R. Ganguly, Apptentive 2015 Consumer Survey: The Mobile Marketer’s Guide to App Store Ratings & Reviews. Apptentive, 2015.
  2. Applause, “Listen to the Voice of Your Customers,” n.d., available online at http://www.applause.com/resources#whitepapers, retrieved March 13, 2016.
  3. M. Galligan, “The right way to ask users to review your app,” 2014, available online at https://medium.com/circa/the-right-way-to-ask-users-to-review-your-app-9a32fd604fca#.kud43shhq, retrieved March 13, 2016.
  4. A. Walz, “Dissecting the App Store Top Charts: The Anatomy of a Top App,” 2015, available online at http://www.apptentive.com/blog/app-store-top-charts/, retrieved March 29, 2016.
  5. M. Smith, “Feedback and Loyalty on the Mobile Frontier: New Research From Apptentive and SurveyMonkey,” 2016, available online at http://www.apptentive.com/blog/feedback-and-loyalty-on-the-mobile-frontier/ retrieved March 30, 2016.
  6. B. Pang and L. Lee, “Opinion Mining and Sentiment Analysis,” Foundations and Trends in Information Retrieval, vol. 2, no. 1–2, pp. 1–135, 2008.
  7. B. Liu, Sentiment Analysis and Opinion Mining. Morgan & Claypool Publishers, 2012.
  8. B. Pang, L. Lee, and S. Vaithyanathan, “Thumbs up? Sentiment Classification using Machine Learning Techniques,” in Proceedings of the 43rd Conference on Empirical Methods in Natural Language Processing, EMNLP’02. Stroudsburg, PA, USA: Association for Computational Linguistics, 2002, pp. 79–86.
  9. B. Liu, “Sentiment Analysis and Subjectivity,” in Handbook of Natural Language Processing, 2nd ed., N. Indurkhya and F. J. Damerau, Eds. Boca Raton, FL, USA: CRC Press, Taylor and Francis Group, 2010.
  10. B. Pang and L. Lee, “Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with respect to Rating Scales,” in Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, ACL’05. Stroudsburg, PA, USA: Association for Computational Linguistics, 2005, pp. 115–124.
  11. A. B. Goldberg and X. Zhu, “Seeing stars when there aren’t many stars: Graph-based semi-supervised learning for sentiment categorization,” in Proceedings of the 1st Workshop on Graph Based Methods for NaturalLanguage Processing, TextGraphs-1’06.
  12. D. Tang, B. Qin, T. Liu, and Y. Yang, “User Modeling with Neural Network for Review Rating Prediction,” in Proceedings of the 24th International Joint Conference on Artificial Intelligence, IJCAI’15, Q. Yang and M. Wooldridge, Eds. Palo Alto, CA, USA: AAAI Press, 2015, pp. 1340–1346.
  13. F. Li, N. Liu, H. Jin, K. Zhao, Q. Yang, and X. Zhu, “Incorporating Reviewer and Product Information for Review Rating Prediction,” in Proceedings of the 22nd International Joint Conference on Artificial Intelligence, IJCAI’11, T. Walsh, Ed., vol. 3. Menlo Park, CA, USA: AAAI Press, 2011, pp. 1820–1825.
  14. L. Qu, G. Ifrim, and G. Weikum, “The Bag-of-Opinions Method for Review Rating Prediction from Sparse Text Patterns,” in Proceedings of the 23rd International Conference on Computational Linguistics, Coling’10, C.-R. Huang and D. Jurafsky, Eds., vol. 2. Beijing, China: Tsinghua University Press, 2010, pp. 913–921.
  15. Y. Zhang, M. Zhang, Y. Liu, and S. Ma, “Boost Phrase-level Polarity Labelling with Review-level Sentiment Classification,” Computational Linguistics, vol. 1, no. 1, pp. 1–25, 2006.
  16. G. Ganu, N. Elhadad, and A. Marian, “Beyond the Stars: Improving Rating Predictions Using Review Text Content,” in Proceedings of the 12th International Workshop on the Web and Databases, WebDB’09, 2009, pp. 1–6.
  17. G. Ganu, Y. Kakodkar, and A. Marian, “Improving the Quality of Predictions using Textual Information in Online User Reviews,” Information Systems, vol. 38, no. 1, pp. 1–15, March 2013.
  18. N. Gupta, G. Di Fabbrizio, and P. Haffner, “Capturing the Stars: Predicting Ratings for Service and Product Reviews,” in Proceedings of the NAACL HLT 2010 Workshop on Semantic Search, SS’10. Stroudsburg, PA, USA: Association for Computational Linguistics, 2010, pp. 36–43.
  19. S. O. Orimaye, S. M. Alhashmi, E. Siew, and S. J. Kang, “Review-Level Sentiment Classification with Sentence-Level Polarity Correction,” Computer Science, OALib Journal, pp. 1–15, 2015.
  20. M. Sänger, “Aspektbasierte Meinungsanalyse von Bewertungen mobiler Applikationen,” Master Thesis, Computer Science Dept., Humboldt-Universität zu Berlin, Berlin, Germany, December 2015.
  21. R. Klinger and P. Cimiano, “Bi-directional Inter-dependencies of Subjective Expressions and Targets and their Value for a Joint Model,” in Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, ACL’13, R. Navigli, J.-S. Chang, and S. Faralli, Eds., vol. 2. Sofia, Bulgaria: Association for Computational Linguistics, August 2013, pp. 848–854.
  22. R. Klinger and P. Cimiano, “Joint and Pipeline Probabilistic Models for Fine-grained Sentiment Analysis: Extracting Aspects, Subjective Phrases and their Relations,” in Proceedings of the IEEE 13th International Conference on Data Mining Workshops, ICDMW’13, W. Ding, T. Washio, H. Xiong, G. Karypis, B. Thuraisingham, D. Cook, and X. Wu, Eds. Dallas, TX, USA: IEEE Computer Society, December 2013, pp. 937–944.
  23. M. Sänger, U. Leser, S. Kemmerer, P. Adolphs, and R. Klinger, “SCARE – The Sentiment Corpus of App Reviews with Fine-grained Annotations in German,” in Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC’16, N. Calzolari, K. Choukri, T. Declerck, M. Grobelnik, B. Maegaard, J. Mariani, A. Moreno, J. Odijk, and S. Piperidis, Eds. Paris, France: European Language Resources Association (ELRA), May 2016.
  24. R. Klinger and P. Cimiano, “The USAGE review corpus for fine grained multi lingual opinion analysis,” in Proceedings of the 9th International Conference on Language Resources and Evaluation, LREC’14, N. Calzolari, K. Choukri, T. Declerck, H. Loftsson, B. Maegaard, J. Mariani, A. Moreno, J. Odijk, and S. Piperidis, Eds. Reykjavik, Iceland: European Language Resources Association, May 2014, pp. 2211–2218.
  25. M. Sänger, private communication, 2016.
  26. T. Wilson, J. Wiebe, and R. Hwa, “Just how mad are you? Finding strong and weak opinion clauses,” in Proceedings of the 19th National Conference on Artificial Intelligence, AAAI’04, G. Ferguson and D. McGuinness, Eds. Menlo Park, California: AAAI Press, 2004, pp. 761–767.