Logo PTI
Polish Information Processing Society
Logo FedCSIS

Annals of Computer Science and Information Systems, Volume 11

Proceedings of the 2017 Federated Conference on Computer Science and Information Systems

Integrated Approach to e-Commerce Websites Evaluation with the Use of Surveys and Eye Tracking Based Experiments

, , , ,

DOI: http://dx.doi.org/10.15439/2017F320

Citation: Proceedings of the 2017 Federated Conference on Computer Science and Information Systems, M. Ganzha, L. Maciaszek, M. Paprzycki (eds). ACSIS, Vol. 11, pages 1019–1030 ()

Full text

Abstract. Due to high availability of e-commerce websites providing similar services and products, the website usability becomes one of the most critical factors affecting online businesses' success. Therefore, website quality and user experience evaluation is an important research task. There are multiple methodologies for performing the evaluation. The proposed in our earlier studies PEQUAL methodology extends the classical eQual method by taking into account different aspects of preference modeling and aggregation derived from Multi-Criteria Decision Analysis (MCDA). This paper extends the PEQUAL methodology further by incorporating eye tracking based measurement and analysis into the criteria set. The results of the conducted empirical verification of proposed approach are presented.

References

  1. S. Kemp. (2017, jan) Digital in 2017 global overview. [Online]. Available: https://www.slideshare.netwearesocialsgdigital-in-2017-global-overview
  2. S. Kemp. (2016, jan) Digital in 2016. [Online]. Available: https://www.slideshare.net/wearesocialsg/digital-in-2016/537
  3. E. N. Europe. (2016, sep) Ecommerce in europe. [Online]. Available: https://ecommercenews.eu/ecommerce-per-country/ecommerce-in-europe/
  4. R. Paul. (2015, jun) Just how big is the ecommerce market? you'll never guess! http://blog.lemonstand.com/just-how-big-is-the-ecommerce-market-youll-never-guess/
  5. A. Strzelecki, M. Furmankiewicz, and P. Ziuziański, “The use of management dashboard in monitoring the efficiency of the internet advertising campaigns illustrated on the example of google analytics,” Studia Ekonomiczne, vol. 296, pp. 136–150, 2016.
  6. J. Grigera, A. Garrido, J. I. Panach, D. Distante, and G. Rossi, “Assessing refactorings for usability in e-commerce applications,” Empirical Software Engineering, vol. 21, no. 3, pp. 1224–1271, 2016. http://dx.doi.org/10.1007/s10664-015-9384-6. [Online]. Available: http://dx.doi.org/10.1007/s10664-015-9384-6
  7. O. Sohaib and K. Kang, “Assessing web content accessibility of e-commerce websites for people with disabilities,” 2016.
  8. M. Olsson, Build a Profitable Online Business: The No-Nonsense Guide, 1st ed. Berkely, CA, USA: Apress, 2013. ISBN 1430263792, 9781430263791
  9. R. M. Kruger, H. Gelderblom, and W. Beukes, “The value of comparative usability and ux evaluation for e-commerce organisations,” 2016.
  10. S. J. Barnes and R. Vidgen, “The equal approach to the assessment of e-commerce quality: A longitudinal study of,” 2005.
  11. H. W. Webb and L. A. Webb, “Sitequal: an integrated measure of web site quality,” Journal of Enterprise Information Management, vol. 17, no. 6, pp. 430–440, 2004.
  12. A. Parasuraman, V. A. Zeithaml, and A. Malhotra, “Es-qual a multiple-item scale for assessing electronic service quality,” Journal of service research, vol. 7, no. 3, pp. 213–233, 2005.
  13. G. Kabir and M. Hasin, “Comparative analysis of topsis and fuzzy topsis for the evaluation of travel website service quality.” International Journal for Quality Research, vol. 6, no. 3, 2012.
  14. J.-P. Brans and B. Mareschal, “Promethee methods,” in Multiple criteria decision analysis: state of the art surveys. Springer, 2005, pp. 163–186.
  15. C.-C. Sun and G. T. Lin, “Using fuzzy topsis method for evaluating the competitive advantages of shopping websites,” Expert Systems with Applications, vol. 36, no. 9, pp. 11 764–11 771, 2009.
  16. R. U. Bilsel, G. Büyüközkan, and D. Ruan, “A fuzzy preference-ranking model for a quality evaluation of hospital web sites,” International Journal of Intelligent Systems, vol. 21, no. 11, pp. 1181–1197, 2006.
  17. J. Wątróbski, P. Ziemba, J. Jankowski, and W. Wolski, “Pequal-e-commerce websites quality evaluation methodology,” in Computer Science and Information Systems (FedCSIS), 2016 Federated Conference on. IEEE, 2016, pp. 1317–1327.
  18. L. Chen and P. Pu, “Eye-tracking study of user behavior in recommender interfaces,” in International Conference on User Modeling, Adaptation, and Personalization. Springer, 2010, pp. 375–380.
  19. K. Chang Lee, S. Wook Chae, and K. Chang Lee, “Exploring the effect of the human brand on consumers’ decision quality in online shopping: An eye-tracking approach,” Online Information Review, vol. 37, no. 1, pp. 83–100, 2013.
  20. L. Cowen, L. J. Ball, and J. Delin, “An eye movement analysis of web page usability,” in People and Computers XVI-Memorable Yet Invisible. Springer, 2002, pp. 317–335.
  21. A. Bojko, Eye tracking the user experience. Rosenfeld Media, 2013.
  22. S. P. Roth, A. N. Tuch, E. D. Mekler, J. A. Bargas-Avila, and K. Opwis, “Location matters, especially for non-salient features–an eye-tracking study on the effects of web object placement on different types of websites,” International journal of human-computer studies, vol. 71, no. 3, pp. 228–235, 2013.
  23. R. V. Menon, V. Sigurdsson, N. M. Larsen, A. Fagerstrøm, and G. R. Foxall, “Consumer attention to price in social commerce: Eye tracking patterns in retail clothing,” Journal of Business Research, vol. 69, no. 11, pp. 5008–5013, 2016.
  24. J. Hernández-Méndez and F. Muñoz-Leiva, “What type of online advertising is most effective for etourism 2.0? an eye tracking study based on the characteristics of tourists,” Computers in Human Behavior, vol. 50, pp. 618–625, 2015.
  25. Q. Wang, S. Yang, M. Liu, Z. Cao, and Q. Ma, “An eye-tracking study of website complexity from cognitive load perspective,” Decision support systems, vol. 62, pp. 1–10, 2014.
  26. S. Khodambashi, H. Gilstad, and Ø. Nytrø, “Usability evaluation of clinical guidelines on the web using eye-tracker.” Studies in health technology and informatics, vol. 228, p. 95, 2016.
  27. P. Štrach and N. Slivkin, “Adaptation needed: Eye-tracking study of cross-cultural differences in perception of b2b websites,” 2017.
  28. X. Yuan, M. Guo, F. Ren, and F. Peng, “Usability analysis of online bank login interface based on eye tracking experiment,” Sensors & Transducers, vol. 165, no. 2, p. 203, 2014.
  29. D. Albayrak and K. Cagiltay, “Analyzing turkish e-government websites by eye tracking,” in Software Measurement and the 2013 Eighth International Conference on Software Process and Product Measurement (IWSM-MENSURA), 2013 Joint Conference of the 23rd International Workshop on. IEEE, 2013, pp. 225–230.
  30. B. Pan and L. Zhang, “An eyetracking study on online hotel decision making: The effects of images and umber of options,” 2016.
  31. L. Hu, W. Zhang, and Q. Xu, “The determinants of online payment method choice: Insight from an eye-tracking study.” in WHICEB, 2013, p. 80.
  32. K. McSpadden, “You now have a shorter attention span than a goldfish,” Time Online Magazine. Retrieved May, vol. 7, p. 2016, 2015.
  33. R. Weatherhead, “Say it quick, say it well–the attention span of a modern internet consumer,” The Guardian Online, vol. 19, 2012.
  34. J. Nielsen, Usability engineering. Elsevier, 1994.
  35. I. Standardization, ISO 9241-11: Ergonomic Requirements for Office Work with Visual Display Terminals (VDTs): Part 11: Guidance on Usability, 1998. [Online]. Available: https://books.google.pl/books?id=TzXYZwEACAAJ
  36. I. Iso, “Iec 25010: 2011,” Systems and software engineering-Systems and software Quality Requirements and Evaluation (SQuaRE)-System and software quality models, 2011.
  37. A. Fernandez, E. Insfran, and S. Abrahão, “Usability evaluation methods for the web: A systematic mapping study,” Information and Software Technology, vol. 53, no. 8, pp. 789–817, 2011.
  38. P. ZIEMBA and M. PIWOWARSKI, “Metody oceny jakości portali internetowych,” 2010. [Online]. Available: http://www.pszw.edu.pl/images/publikacje/t027_pszw_2010_ziemba_piwowarski_-_metody_oceny_jakosci_portali_internetowych.pdf
  39. F. D. Davis, “Perceived usefulness, perceived ease of use, and user acceptance of information technology,” MIS quarterly, pp. 319–340, 1989.
  40. T. Ahn, S. Ryu, and I. Han, “The impact of the online and offline features on the user acceptance of internet shopping malls,” Electronic Commerce Research and Applications, vol. 3, no. 4, pp. 405–420, 2005.
  41. H. Webb and L. Webb, “Business to consumer electronic commerce website quality: integrating information and service dimensions,” AMCIS 2001 Proceedings, p. 111, 2001.
  42. A. Parasuraman, V. A. Zeithaml, and L. L. Berry, “Servqual: A multiple-item scale for measuring consumer perc,” Journal of retailing, vol. 64, no. 1, p. 12, 1988.
  43. J. Wątróbski, P. Ziemba, J. Jankowski, and W. Wolski, “Using pequal methodology in auction platforms evaluation process,” in Conference on Advanced Information Technologies for Management. Springer, 2016, pp. 222–241.
  44. T. M. La Porte, C. C. Demchak, and C. Friis, “Webbing governance: global trends across national-level public agencies,” Communications of the ACM, vol. 44, no. 1, pp. 63–67, 2001.
  45. S. Elling, L. Lentz, and M. De Jong, “Website evaluation questionnaire: development of a research-based tool for evaluating informational websites,” in International Conference on Electronic Government. Springer, 2007, pp. 293–304.
  46. Z. Yang, S. Cai, Z. Zhou, and N. Zhou, “Development and validation of an instrument to measure user perceived service quality of information presenting web portals,” Information & Management, vol. 42, no. 4, pp. 575–589, 2005.
  47. G. M. Ping Zhang, “User expectations and rankings of quality factors in different web site domains,” International Journal of Electronic Commerce, vol. 6, no. 2, pp. 9–33, 2001.
  48. M. Hassenzahl, “User experience (ux): towards an experiential perspective on product quality,” in Proceedings of the 20th Conference on l’Interaction Homme-Machine. ACM, 2008, pp. 11–15.
  49. S. Kim and L. Stoel, “Dimensional hierarchy of retail website quality,” Information & Management, vol. 41, no. 5, pp. 619–633, 2004.
  50. W. Chmielarz and M. Zborowski, “Comparative analysis of electronic banking websites in selected banks in poland in 2014,” in Computer Science and Information Systems (FedCSIS), 2015 Federated Conference on. IEEE, 2015, pp. 1499–1504.
  51. W. Chmielarz, “Evaluation of selected mobile applications stores from the user's perspective,” Online Journal of Applied Knowledge Management, vol. 3, no. 1, pp. 21–36, 2015.
  52. W. Chmielarz, “Methods of comparative analysis of electronic bankings' websites. case of poland,” in 1-st CEE Symposium on Business Informatics, red. G. Chroust, G. Kotsis, V. Risak, N. Rozsenich, P. Zinterhof, Osterreichische Computer Gesellschaft, Vienna. Citeseer, 2009, pp. 73–84.
  53. A. Zenebe, L. Zhou, and A. F. Norcio, “User preferences discovery using fuzzy models,” Fuzzy Sets and Systems, vol. 161, no. 23, pp. 3044–3063, 2010.
  54. L. Del Vasto-Terrientes, A. Valls, R. Slowinski, and P. Zielniewicz, “Electre-iii-h: An outranking-based decision aiding method for hierarchically structured criteria,” Expert Systems with Applications, vol. 42, no. 11, pp. 4910–4926, 2015.
  55. T. Kaya, “Multi-attribute evaluation of website quality in e-business using an integrated fuzzy ahptopsis methodology,” International Journal of Computational Intelligence Systems, vol. 3, no. 3, pp. 301–314, 2010.
  56. J. Huang, X. Jiang, and Q. Tang, “An e-commerce performance assessment model: Its development and an initial test on e-commerce applications in the retail sector of china,” Information & Management, vol. 46, no. 2, pp. 100–108, 2009.
  57. A. Guitouni, J.-M. Martel, P. Vincke, and P. North, “A framework to choose a discrete multicriterion aggregation procedure,” Defence Research Establishment Valcatier (DREV), 1998.
  58. P. Ziemba, J. Wątróbski, J. Jankowski, and M. Piwowarski, “Research on the properties of the ahp in the environment of inaccurate expert evaluations,” in Selected Issues in Experimental Economics. Springer, 2016, pp. 227–243.
  59. P. Ziemba and J. Wątróbski, “Selected issues of rank reversal problem in anp method,” in Selected Issues in Experimental Economics. Springer, 2016, pp. 203–225.
  60. D. Bouyssou, E. Jacquet-Lagrèze, P. Perny, R. Slowiński, D. Vanderpooten, and P. Vincke, Aiding decisions with multiple criteria: essays in honor of Bernard Roy. Springer Science & Business Media, 2012, vol. 44.
  61. B. Roy, Multicriteria methodology for decision aiding. Springer Science & Business Media, 2013, vol. 12.