Logo PTI
Polish Information Processing Society
Logo FedCSIS

Annals of Computer Science and Information Systems, Volume 8

Proceedings of the 2016 Federated Conference on Computer Science and Information Systems

PEQUAL - E-commerce websites quality evaluation methodology

, , ,

DOI: http://dx.doi.org/10.15439/2016F469

Citation: Proceedings of the 2016 Federated Conference on Computer Science and Information Systems, M. Ganzha, L. Maciaszek, M. Paprzycki (eds). ACSIS, Vol. 8, pages 1317–1327 ()

Full text

Abstract. Website quality evaluation is an important research task. Evolution and a growing set of available methods are observed. The article presents the authors' evaluation methodology of the quality of websites named PEQUAL. The formal foundation of the proposed methodology is the broadening of the classical EQUAL method with aspects of preference modelling and evaluation aggregation used in Multi-Criteria Decision Analysis (MCDA). Its empirical verification has been carried out for top e-commerce websites. The conducted research has revealed significant practical possibilities of analysis and interpretation of obtained final rankings.

References

  1. eMarketer, “Retail ecommerce sales worldwide from 2013 to 2018,” http://www.emarketer.com/Article.aspx?R=1011765, 2014. date accessed 25.03.2016
  2. Statista, “Digital buyer penetration worldwide from 2011 to 2018,” http://www.statista.com/statistics/261676/digital-buyer-penetration-worldwide, 2014. date accessed 25.03.2016
  3. A. Chaudhury and J.P. Kuilboer, E-business and E-commerce Infrastructure: Technologies Supporting the E-business Initiative. McGraw-Hill Higher Education, 2001.
  4. P. De, Y. Hu, and M.S. Rahman, “Technology usage and online sales: An empirical study,” Management Science, vol. 56, no. 11, pp. 1930-1945, 2010.
  5. Y. Purwati, “Standard features of e-commerce user interface for the web,” Journal of Arts, Science & Commerce, vol. 2, pp. 77-87, 2011.
  6. F. Abbattista, M. Degemmis, O. Licchelli, P. Lops, G. Semeraro, and F. Zambetta, “Improving the usability of an e-commerce website through personalization,” in Proc. 2nd International Conference on Adaptive Hypermedia and Adaptive Web Based Systems, RPeC’02, Malaga, Spain, 2002, pp. 20-29.
  7. B. Xiao and I. Benbasat, “E-commerce product recommendation agents: Use, characteristics, and impact,” Mis Quarterly, vol. 31, no. 1, pp. 137-209, 2007.
  8. S. Özkan, G. Bindusara, and R. Hackney, “Facilitating the adoption of e-payment systems: theoretical constructs and empirical analysis,” Journal of enterprise information management, vol. 23, no. 3, pp. 305-325, 2010.
  9. R. R. Yager, “Targeted e-commerce marketing using fuzzy intelligent agents,” IEEE Intelligent Systems and their Applications, vol. 15, no. 6, 42-45, 2000.
  10. C. Forman, A. Ghose, and A. Goldfarb, “Competition between local and electronic markets: How the benefit of buying online depends on where you live,” Management Science, vol. 55, no. 1, pp. 47-57, 2009.
  11. L. Hasan, A. Morris, and S. Probets, “Using Google Analytics to evaluate the usability of e-commerce sites,” in Human centered design, Berlin Heidelberg: Springer, pp. 697-706, 2009.
  12. Y. Jyoti and B. Mallick, “Web Mining: Characteristics and Application in Ecommerce,” Intl. J. of IJECSE, vol.1, no. 4, 2011.
  13. X. Wu and A. Bolivar, “Predicting the conversion probability for items on C2C ecommerce sites,” in Proc. of the 18th ACM conference on Information and knowledge management, ACM, 2009, pp. 1377-1386.
  14. S. S. Srinivasan, R. Anderson, and K. Ponnavolu, “Customer loyalty in e-commerce: an exploration of its antecedents and consequences,” Journal of Retailing, vol. 78, no. 1, pp. 41-50, 2002.
  15. D. W. Manchala, “E-commerce trust metrics and models,” IEEE Internet Computing, vol. 4, no. 2, pp. 36-44, 2000.
  16. M. Cao, Q. Zhang, and J. Seydel, “B2C e-commerce web site quality: an empirical examination,” Industrial Management & Data Systems, vol. 105, no. 5, pp. 645-661, 2005.
  17. A. K. Ghosh, E-commerce Security and Privacy, New York: Springer, 2012.
  18. O. Sohaib and K. Kang, “The Importance of Web Accessibility in Business to-Consumer (B2C) Websites,” in Proc. 22nd Australasian Software Engineering Conference (ASWEC 2013), 2002.
  19. T. R. Lituchy and R.A. Barra, “International issues of the design and usage of websites for e-commerce: Hotel and airline examples,” Journal of Engineering and Technology Management, vol. 25, no. 1, pp. 93-111, 2008.
  20. O. Sohaib, “Usability and Cultural Issues in Global E-Commerce,” Journal of engineering and technology management, vol. 25, no. 1, pp. 156-166, 2012.
  21. D. Belanche, L.V. Casaló, and M. Guinalíu, “Website usability, consumer satisfaction and the intention to use a website: The moderating effect of perceived risk,” Journal of retailing and consumer services, vol. 19, no. 1, pp. 124-132, 2012.
  22. P. Ziemba, J. Jankowski, J. Wątróbski, W. Wolski, and J. Becker, “Integration of Domain Ontologies in the Repository of Website Evaluation Methods,” Annals of Computer Science and Information Systems, vol. 5, pp. 1585-1595, 2015. http://dx.doi.org/10.15439/2015F297
  23. P. Ziemba, M. Piwowarski, J. Jankowski, and J. Wątróbski, “Method of Criteria Selection and Weights Calculation in the Process of Web Projects Evaluation,” Lecture Notes in Artificial Intelligence, vol. 8733, pp. 684-693, 2014. http://dx.doi.org/10.1007/978-3-319-11289-3_69
  24. P. Ziemba, J. Wątróbski, J. Jankowski, and W. Wolski, “Construction and Restructuring of the Knowledge Repository of Website Evaluation Methods,” Lecture Notes in Business Information Processing, vol. 243, pp. 29-52, 2016. http://dx.doi.org/ 10.1007/978-3-319-30528-8_3
  25. S. J. Barnes and R. Vidgen, “Measuring Web site quality improvements: a case study of the forum on strategic management knowledge exchange,” Industrial Management & Data Systems, vol. 103, no. 5, pp. 297-309, 2003.
  26. S. J. Barnes and R. Vidgen, “The eQual Approach to the Assessment of E-Commerce Quality: A Longitudinal Study of Internet Bookstories,” in Web Engineering: Principles and Techniques, W. Suh, Ed. London: Idea Group Publishing, 2005, pp. 161-181.
  27. S. J. Barnes and R. Vidgen, “Data Triangulation in action: using comment analysis to refine web quality metrics,” in Proc. 13th European Conference on Information Systems, 2005.
  28. S. J. Barnes and R. Vidgen, “Data triangulation and web quality metrics: A case study in e-government,” Information & Management, vol. 43, no. 6, pp. 767-777, 2006.
  29. S. J. Barnes and R. Vidgen, “WebQual: An Exploration of Web-site Quality,” in Proc. 8th European Conference on Information Systems, vol. 1, 2000, pp. 298–305.
  30. S. J. Barnes, K. Liu, and R. Vidgen, “Evaluating WAP News Sites: The WebQual/M Approach,” in Proc. 9th European Conference on Information Systems, 2001.
  31. H. P. Shih, “Extended technology acceptance model of Internet utilization behavior,” Information & Management, vol. 41, no. 6, pp. 719-729, 2004.
  32. W. H. DeLone and E.R. McLean, “Information Systems Success: The Quest for the Dependent Variable,” Information Systems Research, vol. 3, no. 1, pp. 60-95, 1992.
  33. P. B. Seddon, “A Respecification and Extension of the DeLone nad McLean Model of IS Success,” Information Systems Research, vol. 8, no. 3, pp. 240-253, 1997.
  34. Z. Yang, S. Cai, Z. Zhou, and N. Zhou, “Development and validation of an instrument to measure user perceived service quality of information presenting Web Portals,” Information & Management, vol. 42, no. 4, pp. 575-589, 2005.
  35. T. Ahn, S. Ryu, and I. Han, “The impact of Web quality and playfulness on user acceptance of online retailing,” Information & Management, vol. 44, no. 3, pp. 263-275, 2007.
  36. B. Suh and I. Han, “Effect of trust on customer acceptance of Internet banking,” Electronic Commerce Research and Applications, vol. 1, no. 3-4, pp. 247-263, 2002.
  37. W. H. DeLone and E.R. McLean, “The DeLone and McLean Model of Information Systems Success: A Ten-Year Update,” Journal of Management Information Systems, vol. 19, no. 4, pp. 9-30, 2003.
  38. S. M. Jafari, N. A. Ali, M. Sambasivan, and M.F. Said, “A Respecification and Extension of DeLone and McLean Model of IS Success in the Citizen-centric e-Governance,” in Proc. IEEE International Conference on Information Reuse and Integration, 2011, pp. 342-346.
  39. T. Ahn, S. Ryu, and I. Han, “The impact of the online and offline features on the user acceptance of Internet shopping malls,” Electronic Commerce Research and Applications, vol. 3, no. 4, pp. 405-420, 2004.
  40. H. W. Webb and L. A. Webb, “Business to consumer electronic commerce Website quality: integrating information and service dimensions,” in Proc. 7th Americas Conference on Information Systems, 2001.
  41. G. J. Udo, K. K. Bagchi, and P. J. Kirs, “Using SERVQUAL to assess the quality of e-learning experience,” Computers in Human Behavior, vol. 27, no. 3, pp. 1272-1283, 2011.
  42. R. Y. Wang and D. M. Strong, “Beyond Accuracy: What Data Quality Means to Data Consumers,” Journal of Management Information Systems, vol. 12, no. 4, pp. 5-33, 1996.
  43. H. W. Webb and L. A. Webb, “SiteQual: an integrated measure of Web site quality,” Journal of Enterprise Information Management, vol. 17, no. 6, pp. 430-440, 2004.
  44. S. Muylle, R. Moenaert, and M. Despontin, “The conceptualization and empirical validation of web site user satisfaction,” Information & Management, vol. 41, no. 5, pp. 543-560, 2004.
  45. S. Elling, L. Lentz, and M. de Jong, “Website Evaluation Questionnaire: Development of a Research-Based Tool for Evaluating Informational Websites,” Lecture Notes in Computer Science, vol. 4656, pp. 293-304, 2007.
  46. S. Elling, L. Lentz, M. de Jong, and H. van den Bergh, “Measuring the quality of governmental websites in a controlled versus an online setting with the ‘Website Evaluation Questionnaire’,” Government Information Quarterly, vol. 29, no. 3, pp. 383-393, 2012.
  47. A. Parasuraman, V.A. Zeithaml, and L.L. Berry, “SERVQUAL: A Multiple-item Scale for Measuring Consumer Perceptions of Service Quality,” Journal of Retailing, vol. 64, no. 1, pp.12-40, 1988.
  48. S. Akinci, E. Atilgan-Inan, and S. Aksoy, “Re-assessment of E-S-Qual and E-RecS-Qual in a pure service setting,” Journal of Business Research, vol. 63, no. 3, pp. 232-240, 2010.
  49. A. Parasuraman, V.A. Zeithaml, and A. Malhotra, “E-S-QUAL A Multiple-Item Scale for Assessing Electronic Service Quality,” Journal of Service Research, vol. 7, no. 10, pp. 1-21, 2005.
  50. G.M. von Dran, P. Zhang, and R. Small, “Quality Websites: An Application of the Kano Model to Website Design,” in Proc. 5th Americas Conference on Information Systems, 1999, pp. 898-900.
  51. P. Zhang and G. von Dran, “User Expectations and Rankings of Quality Factors in Different Web Site Domains,” International Journal of Electronic Commerce, vol. 6, no. 2, pp. 9-33, 2002.
  52. P. Zhang and G. von Dran, “Expectations and Rankings of Website Quality Features: Results of Two Studies on User Perceptions,” in Proc. 34th Hawaii International Conference on System Sciences, 2001.
  53. C. C. Demchak, C. Friis, and T.M. La Porte, “Webbing Governance: National Differences in Constructing the Face of Public Organizations,” in Handbook of Public Information Systems, G. D. Garson, Ed. New York: Marcel Dekker, 2000, pp.179-196.
  54. A. Holzinger, “Usability Engineering Methods for Software Developers,” Communications of ACM, vol. 48, no. 1, pp. 71-74, 2005.
  55. A. Zenebe, L. Zhou, F. Norcio, “User preferences discovery using fuzzy models,” Fuzzy Sets and Systems, vol. 161, no. 23, pp. 3044-3063, 2010.
  56. P. Ziemba, J. Jankowski, J. Wątróbski, and M. Piwowarski, “Web Projects Evaluation Using the Method of Significant Website Assessment Criteria Detection,” Transactions on Computational Collective Intelligence, no. 22, pp. 167-188, 2016. http://dx.doi.org/10.1007/978-3-662-49619-0_9
  57. P. Ziemba and M. Piwowarski, “Procedure of Reducing Website Assessment Criteria and User Preference Analyses,” Foundations of Computing and Decision Sciences, vol. 36, no.3-4, pp. 315-325, 2011.
  58. S. Kim and L. Stoel, “Dimensional hierarchy of retail website quality,” Information & Management, vol. 11, no. 2, pp. 109-117, 2004. http://dx.doi.org/10.1016/S0969-6989(03)00010-9
  59. Y. Lee and K.A. Kozar, “Investigating the effect of website quality on e-business success: An analytic hierarchy process (AHP) approach,” Decision Support Systems, vol. 42, no. 3, pp. 1383-1401, 2006.
  60. W. Chmielarz and M. Zborowski, “The Application Of A Conversion Method In A Confrontational Pattern-Based Design Method Used For The Evaluation Of It Systems,” Annals of Computer Science and Information Systems, vol. 2, pp. 1227-1234, 2014. http://dx.doi.org/10.15439/2014F198
  61. W. Chmielarz and M. Zborowski, “Comparative Analysis of Electronic Banking Websites in Selected Banks in Poland in 2014,” Annals of Computer Science and Information Systems, vol. 5, pp. 1499-11504, 2015. http://dx.doi.org/10.15439/2015F43
  62. W. Chmielarz, “Evaluation of selected mobile applications stores from the user’s perspective,” Online Journal of Applied Knowledge Management, vol. 3, no. 1, pp. 21-36, 2015.
  63. C.C. Sun and G.T.R. Lin, “Using fuzzy TOPSIS method for evaluating the competitive advantages of shopping websites,” Expert Systems with Applications, vol. 36, no. 9, pp. 11764-11771, 2009. http://dx.doi.org/10.1016/j.eswa.2009.04.017
  64. L. Del Vasto-Terrientes, A. Valls, R. Słowiński, and P. Zielniewicz, “ELECTRE-III-H: An outranking-based decision aiding method for hierarchically structured criteria,” Expert Systems with Applications, vol. 42, no. 11, pp. 4910-4926, 2015. http://dx.doi.org/10.1016/j.eswa.2015.02.016
  65. H.F. Lin, “An application of fuzzy AHP for evaluating course website quality,” Computers & Education, vol. 54, no. 4, pp. 977-888, 2010.
  66. F. Kong and H. Liu, “Applying fuzzy analytic hierarchy process to evaluate success factors of e-commerce,” International Journal of Information and System Sciences, vol. 1, no. 3-4, pp. 406-412, 2005.
  67. R. U. Bilsel, G. Buyukozkan, and D. Ruan, “A Fuzzy Preference-Ranking Model for a Quality Evaluation of Hospital Web Sites,” International Journal of Intelligent Systems, vol. 21, pp. 1181-1197, 2006.
  68. T. Kaya, “Multi-attribute Evaluation of Website Quality in E-business Using an Integrated Fuzzy AHPTOPSIS Methodology,” International Journal of Computational Intelligence Systems, vol. 3, no. 3, pp. 301-314, 2010. http://dx.doi.org/10.1080/18756891.2010.9727701
  69. J. Huang, X. Jiang, and Q. Tang, “An e-commerce performance assessment model: Its development and an initial test on e-commerce applications in the retail sector of China,” Information & Management, vol. 46, no. 2, pp. 100-108, 2009.
  70. A. Guitouni, J.M. Martel, and P. Vincke, “A Framework to Choose a Discrete Multicriterion Aggregation Procedure,” Defence Research Establishment Valcatier (DREV), 1998.
  71. J. Wątróbski and J. Jankowski, “Knowledge Management in MCDA Domain,” Annals of Computer Science and Information Systems, vol. 5, pp. 1445-1450, 2015. http://dx.doi.org/10.15439/2015F295
  72. J. Wątróbski and J. Jankowski, “An Ontology-Based Knowledge Representation of MCDA Methods,” Lecture Notes in Artificial Intelligence, vol. 9621, pp. 54-64, 2016. http://dx.doi.org/10.1007/978-3-662-49381-6_6
  73. J. Wątróbski and J. Jankowski, “Guideline for MCDA Method Selection in Production Management Area,” in New Frontiers in Information and Production Systems Modelling and Analysis, Intellingent Systems Reference Library 98, Berlin Heidelberg: Springer, 2016, pp. 119-138. http://dx.doi.org/10.1007/978-3-319-23338-3_6
  74. J. Jankowski, J. Wątróbski, P. Ziemba, “Modelling the impact of visual components on verbal comunication in online advertising,” in Computational Collective Intelligence. ICCCI 2015, Part II. LNAI, vol. 9330, Heidelberg: Springer, 2015, pp. 44-53.
  75. J. Jankowski, J. Watrobski, and M. Piwowarski, “Fuzzy Modeling of Digital Products Pricing in the Virtual Marketplace,” in Proceedings of 6th International Conference on Hybrid Artificial Intelligent Systems, LNCS, vol. 6678. Heidelberg: Springer, 2011, pp. 338-346.
  76. J. Jankowski, K. Kolomvatsos, P. Kazienko, J. Wątróbski, "Fuzzy Modeling of User Behaviors and Virtual Goods Purchases in Social Networking Platforms," Journal of Universal Computer Science, vol. 22, no. 3, pp. 416-437, 2016.
  77. P. Ziemba, J. Wątróbski, J. Jankowski, and M. Piwowarski, “Research on the Properties of the AHP in the Environment of Inaccurate Expert Evaluations,” in Selected Issues in Experimental Economics, K. Nermend and M. Łatuszyńska, Ed. Switzerland: Springer, 2016, pp. 227-243.
  78. P. Ziemba and J. Wątróbski, “Selected Issues of Rank Reversal Problem in ANP Method,” in Selected Issues in Experimental Economics, K. Nermend and M. Łatuszyńska, Ed. Switzerland: Springer, 2016, pp. 203-225.
  79. A. Ishizaka and P. Nemery, Multi-Criteria Decision Analysis. Methods and Software. Chichester: Wiley, 2013.
  80. B. Roy, Multicriteria Methodology for Decision Aiding. Dordrecht: Springer, 1996.
  81. http://www.alexa.com/topsites/category;3/Business/E-Commerce. date accessed 25.04.2016
  82. http://www.mbaskool.com/fun-corner/top-brand-lists/13991-top-10-e-commerce-companies-in-the-world-2015.html?start=1. date accessed 25.04.2016
  83. http://www.dollarfry.com/worlds-top-10-ecommerce-sites-alexa-rank-basis/. date accessed 25.04.2016
  84. https://www.mitx.org/files/zmags-top100-web.pdf. date accessed 25.04.2016
  85. http://unctad.org/en/PublicationsLibrary/ier2015_en.pdf. date accessed 27.04.2016
  86. J. Jankowski, P. Ziemba, J. Wątróbski, P. Kazienko, “Towards the Tradeoff Between Online Marketing Resources Exploitation and the User Experience with the Use of Eye Tracking,” Lecture Notes in Artificial Intelligence, ACIIDS 2016, vol. 9621, pp. 330-343, 2016. http://dx.doi.org/10.1007/978-3-662-49381-6_32
  87. J. Jankowski, J. Wątróbski, K. Witkowska, P. Ziemba, “Eye Tracking Based Experimental Evaluation of the Parameters of Online Content Affecting the Web User Behaviour,” in Selected Issues in Experimental Economics, K. Nermend and M. Łatuszyńska, Ed. Switzerland: Springer, 2016, pp. 311-332.
  88. J. Wątróbski, P. Ziemba, and W. Wolski, “Methodological Aspects of Decision Support System for the Location of Renewable Energy Sources,” Annals of Computer Science and Information Systems, vol. 5, pp. 1451-1459, 2015. http://dx.doi.org/10.15439/2015F294