Logo PTI
Polish Information Processing Society
Logo FedCSIS

Annals of Computer Science and Information Systems, Volume 18

Proceedings of the 2019 Federated Conference on Computer Science and Information Systems

Do online reviews reveal mobile application usability and user experience? The case of WhatsApp

,

DOI: http://dx.doi.org/10.15439/2019F289

Citation: Proceedings of the 2019 Federated Conference on Computer Science and Information Systems, M. Ganzha, L. Maciaszek, M. Paprzycki (eds). ACSIS, Vol. 18, pages 747–754 ()

Full text

Abstract. The variety of hardware devices and the diversity of their users imposes new requirements and expectations on designers and developers of mobile applications (apps). While the Internet has enabled new forms of communication platform, online stores provide the ability to review apps. These informal online app reviews have become a viral form of electronic word-of-mouth (eWOM), covering a plethora of issues. In our study, we set ourselves the goal of investigating whether online reviews reveal usability and user experience (UUX) issues, being important quality-in-use characteristics. To address this problem, we used sentiment analysis techniques, with the aim of extracting relevant keywords from eWOM WhatsApp data. Based on the extracted keywords, we next identified the original users' reviews, and individually assigned each attribute and dimension to them. Eventually, the reported issues were thematically synthesized into 7 attributes and 8 dimensions. If one asks whether online reviews reveal genuine UUX issues, in this case, the answer is definitely affirmative.

References

  1. Ono, A., Kikumori, M. (2018). Consumer Adoption of PC-Based/Mobile-Based Electronic Word-of-Mouth. In Encyclopedia of Information Science and Technology, Fourth Edition, 6019–6030.
  2. Mobile App Daily (2019). Why Your App Needs Reviews - Importance And Benefits. https://www.mobileappdaily.com/2018/11/20/why-your-app-need-reviews
  3. Walz, A., Ganguly, R. (2015). The Mobile Marketer's Guide to App Store Ratings and Reviews. Apptentive. http://cdn2.hubspot.net/hubfs/232559/The_Mobile_Marketers_Guide_To_App_Store_Ratings_and_Reviews.pdf
  4. Kukulska-Hulme, A. (2007). Mobile usability and user experience. In Mobile Learning, 61–72. Routledge.
  5. Park, J., Han, S. H., Kim, H. K., Cho, Y., Park, W. (2013). Developing elements of user experience for mobile phones and services: survey, interview, and observation approaches. Human Factors and Ergonomics in Manufacturing & Service Industries, 23(4), 279–293.
  6. Parsazadeh, N., Ali, R., Rezaei, M., Tehrani, S. Z. (2018). The construction and validation of a usability evaluation survey for mobile learning environments. Studies in Educational Evaluation, 58, 97–111.
  7. Weichbroth, P. (2019). Usability of mobile applications: a systematic literature study. (In Print).
  8. ISO 9241-11 (2018). Ergonomics of human-system interaction – Part 11: Usability: Definitions and concepts.
  9. ISO 9241-210 (2010), https://www.iso.org/obp/ui/#iso:std:iso:9241:-210:ed-1:v1:en, [02-04-2019].
  10. Hedegaard, S., Simonsen, J. G. (2013). Extracting usability and user experience information from online user reviews". The SIGCHI Conference on Human Factors in Computing Systems, 2089–2098.
  11. Korhonen, H., Arrasvuori, J., & Väänänen-Vainio-Mattila, K. (2010). Let users tell the story: evaluating user experience with experience reports. In CHI'10 Extended Abstracts on Human Factors in Computing Systems, 4051–4056. ACM.
  12. Lewis, J. R. (1995). IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. International Journal of Human‐Computer Interaction, 7(1), 57–78.
  13. Basińska, B., Dąbrowski, D., & Sikorski, M. (2013). Usability and relational factors in user-perceived quality of online services. Studia Ekonomiczne, 158, 18–28.
  14. Moumane, K., Idri, A., & Abran, A. (2016). Usability evaluation of mobile applications using ISO 9241 and ISO 25062 standards. SpringerPlus, 5(1), 548.
  15. Kaikkonen, A., Kekäläinen, A., Cankar, M., Kallio, T., & Kankainen, A. (2005). Usability testing of mobile applications: A comparison between laboratory and field testing. Journal of Usability studies, 1(1), 4–16.
  16. McMillan, D., McGregor, M., & Brown, B. (2015). From in the wild to in vivo: Video Analysis of Mobile Device Use. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services, 494–503. ACM.
  17. Hussain, A., Mkpojiogu, E. O., Jamaludin, N. H., & Moh, S. T. (2017). A usability evaluation of Lazada mobile application. In AIP Conference Proceedings, Vol. 1891(1), AIP Publishing.
  18. Iacob, C., Harrison, R. (2013). Retrieving and analyzing mobile apps feature requests from online reviews. In Proceedings of the 10th Working Conference on Mining Software Repositories, 41–44. IEEE.
  19. He, T., Hao, R., Qi, H., Liu, J., & Wu, Q. (2016). Mining Feature-Opinion from Reviews Based on Dependency Parsing. International Journal of Software Engineering and Knowledge Engineering, 26(09n10), 1581–1591.
  20. Jin, J., Ji, P., & Gu, R. (2016). Identifying comparative customer requirements from product online reviews for competitor analysis. Engineering Applications of Artificial Intelligence, 49, 61–73.
  21. Genc-Nayebi, N., Abran, A. (2017). A systematic literature review: Opinion mining studies from mobile app store user reviews. Journal of Systems and Software, 125, 207–219.
  22. LangPipe. (2019). Home. http://alias-i.com/lingpipe/index.html
  23. Vu, P. M., Nguyen, T. T., Pham, H. V., & Nguyen, T. T. (2015). Mining user opinions in mobile app reviews: A keyword-based approach (t). In 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE), 749–759. IEEE.
  24. Provalis Research Group (2019), Sentiment dictionaries, https://provalisresearch.com/products/content-analysis-software/wordstat-dictionary/sentiment-dictionaries/, [04-04-2019].
  25. Wiegand M., Balahur A., Montoyo A., Roth B., Klakow, D. (2010). A Survey on the Role of Negation in Sentiment Analysis, Proceedings of the Workshop on Negation and Speculation in Natural Language Processing, Uppsala, 60–68.
  26. Asmi, A., Ishaya, T. (2012). Negation Identification and Calculation in Sentiment Analysis. The Second International Conference on Advances in Information Mining and Management.
  27. Baj-Rogowska A. (2017). Sentiment Analysis of Facebook Posts: the Uber case. IEEE Eighth International Conference on Intelligent Computing and Information Systems, pp. 391–395, Cairo (Egypt).
  28. Weichbroth, P. (2011). The visualisation of association rules in market basket analysis as a supporting method in customer relationship management systems. Prace Naukowe Uniwersytetu Ekonomicznego we Wrocławiu, (232), 136–145.
  29. Pondel, J., & Pondel, M. (2015). BI and Big Data solutions in project management. Informatyka Ekonomiczna, 4 (38), 55–63.
  30. Owoc, M., & Pondel, M. (2016). Selection of Free Software Useful in Business Intelligence. Teaching Methodology Perspective. In IFIP International Workshop on Artificial Intelligence for Knowledge Management, 93–105. Springer, Cham.
  31. Loughran T., McDonald B. (2011). When is a liability not a liability? Textual Analysis, Dictionaries and 10-Ks. The Journal of Finance, 66(1), pp. 35–66.
  32. Provalis Research Group (2019), ProSuite, https://provalisresearch.com/products/qualitative-data-analysis-software/, [02-03-2019].
  33. Lock, I., Seele, P. (2016). The credibility of CSR (corporate social responsibility) reports in Europe. Evidence from a quantitative content analysis in 11 countries". Journal of Cleaner Production, 122, 186–200.
  34. Du Plessis, C. (2017). The role of content marketing in social media content communities. South African Journal of Information Management, 19(1), 1–7.
  35. Jones-Diette, J. S., Dean, R. S., Cobb, M., Brennan, M. L. (2019). Validation of text-mining and content analysis techniques using data collected from veterinary practice management software systems in the UK. Preventive Veterinary Medicine.
  36. Provost G., Robert JM. (2013) The Dimensions of Positive and Negative User Experiences with Interactive Products. In: Marcus A. (eds) Design, User Experience, and Usability. Design Philosophy, Methods, and Tools. DUXU 2013. Lecture Notes in Computer Science, vol 8012. Springer, Berlin, Heidelberg.
  37. Cimpanu, C. (2018). Google announces crackdown on Play Store ratings and reviews. https://www.zdnet.com/article/google-announces-crackdown-on-play-store-ratings-and-reviews/