Logo PTI Logo FedCSIS

Communication Papers of the 17th Conference on Computer Science and Intelligence Systems

Annals of Computer Science and Information Systems, Volume 32

Replication of first-click eye tracking A/B test of webpage interactive elements

,

DOI: http://dx.doi.org/10.15439/2022F51

Citation: Communication Papers of the 17th Conference on Computer Science and Intelligence Systems, M. Ganzha, L. Maciaszek, M. Paprzycki, D. Ślęzak (eds). ACSIS, Vol. 32, pages 145–151 ()

Full text

Abstract. In this paper, we present a replication of our experiments of first click eye tracking A/B test of interactive elements on a website. The main difference between these two experiments is the equipment used; the first study was done using Tobi X-60 eye-tracker, while the replication described in this paper was conducted using Tobii TX 300, a higher frequency eye tracker. Eye tracking metrics used to evaluate the user experience of websites were almost identical. This paper presents the results of an experiment in which eight commercial websites were tested with the A / B first click test. This work examines the validity of a specific set of eye tracking metrics for their broad application in user experience research on websites.

References

  1. M. Thüring,S. Mahlke, “Usability, aesthetics and emotions in human–technology interaction”, in International Journal of Psychology, vol. 42(4), 2007, pp. 253–264.
  2. V. Roto, E. C Law, A. P. Vermeeren, J. Hoonhout, “User experience white paper: Bringing clarity to the concept of user experience”. 2011.
  3. B. Nigel, “What is the difference between the purpose of usability and user experience evaluation methods”, in Proceedings of the Workshop UXEM, 2009, pp. 1–4.
  4. S. Hellweger, X. Wang, “What is user experience really: towards a UX conceptual framework”, arXiv preprint https://arxiv.org/abs/1503.01850, 2015.
  5. M. Bratkowska, J. Falkowska, J. Sobecki, “Webcam eye tracking platforms for usability testing”, in Proc. of 35th IBIMA Conference; 1–2 April 2020, Seville, Spain, pp. 3091–3099.
  6. P. Chynał, J. Sobecki, M. Rymarz, B. Kilijańska, “Shopping behaviour analysis using eyetracking and EEG”, in 9th International Conference on Human System Interactions (HSI) July 2016, pp. 458–464, IEEE.
  7. K. Cichoń, J. Sobecki, J.M. Szymański, “Gesture tracking and recognition in touchscreens usability testing” in Proceedings of the International Conference on Multimedia, Interaction, Design and Innovation, June 2013, pp. 1–8.
  8. P. Chynał, J. Sobecki, “Application of thermal imaging camera in eye tracking evaluation”, in 9th International Conference on 341 Human System Interactions (HSI) 2016, July. pp. 451–457. IEEE.
  9. P. Weichbroth, K. Redlarski, I. Garnik, “ Eye-tracking web usability research”, in 2016 Federated Conference on Computer Science and Information Systems (FedCSIS), pp. 1681–1684. IEEE.
  10. A. Schall, J. Bergstrom, “Introduction to eye tracking”, Elsevier, 2014.
  11. M. Horsley, M. Eliot, B.A. Knight, “Current Trends in Eye tracking Research”, Springer, 2014.
  12. A. Bojko, “Using Eye Tracking to Compare Web Page Designs: A Case Study”, Journal of Usability Studies 2006. Issue 3, Vol. 1, pp. 112–120.
  13. J. Falkowska, J Sobecki, “First Click eye Tracking A/B Tests of Web-page Interactive Elements”. in Proc. of 36th IBIMA Con-349 ference; Grenada, Spain 4–5 November 2020, pp. 12524–12530.
  14. B. Rogos-Turek, I. Moscichowska, “Research as a basis for User Experience design”, (in Polish), PWN:Warszawa, Poland, 2015.
  15. P. Chynał, J. Falkowska, J.Sobecki, “Web page graphic design usability testing enhanced with eye tracking” in proceedings of the 1st International Conference on Intelligent Human Systems Integration (IHSI 2018), pp. 515–520.
  16. J. Falkowska, J. Sobecki, M. Pietrzak,”Eye tracking usability testing enhanced with EEG analysis”, in proc. of 5th International Conference, held as Part of HCI International 2016, Toronto, Canada, July 17–22, 2016, pp. 399–411.
  17. J. Falkowska, B. Kilijańska, J. Sobecki, K. Zerka, “Microinteractions of forms in web based 360 systems usability and eye tracking metrics analysis”, in proceedings of the AHFE 2018 International Conference on Human Factors and Systems Interaction, Springer, cop. 2019, pp. 164–174.
  18. “Flat UI Elements Attract Less Attention and Cause Uncertainty”, available online: https://www.nngroup.com/articles/flat-ui-less-attention-cause-uncertainty (accessed on 1.09.2021)
  19. X. Kotval, J. Goldberg, “Eye Movements and Interface Component Grouping: An Evaluation Method”, in Proceedings of the Human Factors and Ergonomics Society Annual Meeting 1998, 42, pp. 486–490.
  20. J. Goldberg, X. Kotval, “Computer interface evaluation using eye movements: Methods and constructs”, International Journal of Industrial Ergonomics 1999, vol. 24, pp. 631–645.
  21. A. Bojko, “Eye tracking the user experience: A practical guide to research”, Rosenfeld Media, 2013, pp. 121–143.
  22. J. Nielsen, K. Pernice, “Eye tracking Web Usability”, New Riders: Berkley, CA, USA 2009.
  23. K Holmqvist, M. Nyström, R. Andersson, R. Dewhurst, H. Jarodzka, J. Van de Weijer, “Eye tracking: A comprehensive guide to methods and measures”, OUP: Oxford, UK, 2011.
  24. P. Krehahn, V. Wohlgemuth, H.A. Meyer, “UxLab: Usability Optimization Case Study of a Environmental Management Information System (EMIS) Using Eyetracking Studies”, in EnviroInfo 2010: Integration of Environmental Information in Europe, Shaker Verlag, Aachen, 2010, pp. 248–249.
  25. M.C. Pretorius, J. van Biljon, E. de Kock,”Added Value of Eye Tracking in Usability Studies: Expert and Non-expert Participants”, in proc. of Human-Computer Interaction 2010, Vol. 332, pp. 110–121.
  26. C. Ehmke, S. Wilson, “Identifying Web Usability Problems from Eye-Tracking Data”, in Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI...but not as we know it, Volume 1; BCS 384 Learning Development Ltd.: Swindon, GBR, September 3 2007; pp. 119–128.
  27. A. Çöltekin, B. Heil, S. Garlandini, S.I. Fabrikant, “Evaluating the Effectiveness of Interactive Map Interface Designs: A Case Study Integrating Usability Metrics with Eye-Movement Analysis”, 2013.
  28. A. Çöltekin, B. Heil, S. Garlandini, S.I. Fabrikant, “Evaluating the Effectiveness of Interactive Map Interface Designs: A Case Study Integrating Usability Metrics with Eye-Movement Analysis”, in Cartography and Geographic Information Science 2009, vol. 36, pp. 5–17.
  29. H.-F. Ho, “The Effects of Controlling Visual Attention to Handbags for Women in Online Shops: Evidence from Eye Movements”, in Computers in Human Behavior 2014, vol. 30, pp. 146–152.
  30. N. Steinfeld,”I Agree to the Terms and Conditions: (How) Do Users Read Privacy Policies Online? An Eye-Tracking Experiment”, in Computers in Human Behavior 2016, 55, pp. 992–1000.
  31. H.-C. Liu, M.-L. Lai, H.-H. Chuang, “Using Eye-Tracking Technology to Investigate the Redundant Effect of Multimedia Web Pages on Viewers”, in Cognitive Processes. Computers in Human Behavior 2011, vol. 27, pp. 2410–2417.
  32. C.-Y. Wang, M.-J. Tsai, C.-C. Tsai, “Multimedia Recipe Reading: Predicting Learning Outcomes and Diagnosing Cooking Interest Using Eye-Tracking Measures”, in Computers in Human Behavior 2016, vol. 62, pp. 9–18.
  33. L. Grigg, A. Griffin, “A Role for Eye-Tracking Research in Accounting and Financial Reporting”, in Current trends in eye tracking research Springer, Cham 2014; pp. 225–230.
  34. S. Djamasbi, M. Siegel, T. Tullis, “Can Fixation on Main Images Predict Visual Appeal of Homepages”, in Proceedings of the 2014 47th Hawaii International Conference on System Sciences; IEEE: Waikoloa, HI, January 2014; pp. 371–375.
  35. Y. Habuchi, M. Kitajima, H. Takeuchi, “Comparison of Eye Movements in Searching for Easy-to-Find and Hard-to-Find Information in a Hierarchically Organized Information Structure”, in the Proceedings of the the 2008 symposium on Eye tracking research applications, Association for Computing Machinery: New York, NY, USA, March 26 2008; pp. 131–134.
  36. A. Bojko, “Using Eye Tracking to Compare Web Page Designs: A “, inase Study”, J. Usability Stu 2006, 1, pp. 112–120.
  37. J. Luan, Z. Yao, Z. F. Zhao, H. Liu, “Search Product and Experience Product Online Reviews: An Eye-Tracking Study on Consumers’ Review Search Behavior”, Computers in Human Behavior 2016, vol. 65, pp. 420–430.
  38. S.-F. Yang, “An Eye-Tracking Study of the Elaboration Likelihood Model in Online Shopping”, in Electronic Commerce Research and Applications 2015, vol. 14, pp. 233–240.
  39. S. Djamasbi, M. Siegel, T. Tullis, “Designing Noticeable Bricklets by Tracking Users’ Eye Movements”, in Proceedings of the 2012 45th Hawaii International Conference on System Sciences; IEEE: Maui, HI, USA, January 2012; pp. 525–532.
  40. J. Hernández-Méndez, F. Muñoz-Leiva, “What Type of Online Advertising Is Most Effective for ETourism 2.0, An Eye Tracking Study Based on the Characteristics of Tourists”, in Computers in Human Behavior 2015, vol. 50, pp. 618–625.
  41. H. Lin, Y.-C. Hsieh, F.-G. Wu, “A Study on the Relationships between Different Presentation Modes of Graphical Icons and Users’ Attention”, in Comput. Hum. Behav. 2016, vol. 63, pp. 218–228.
  42. Q. Wang, S. Yang, M. Liu, Z. Cao, Q. Ma, “An Eye-Tracking Study of Website Complexity from Cognitive Load Perspective”, in Decision Support Systems 2014, vol. 62, pp. 1–10.
  43. S.P. Roth, A.N. Tuch, E.D. Mekler, J.A. Bargas-Avila, K. Opwis, “Location Matters, Especially for Non-Salient Features-An Eye-Tracking Study on the Effects of Web Object Placement on Different Types of Websites”, in Int. J. Hum.-Comput. Stud. 2013, vol. 71, pp. 228–235.
  44. C. Cuadrat Seix, M.S. Veloso, J.J.R. Soler, “Towards the Validation of a Method for Quantitative Mobile Usability Testing 422 Based on Desktop Eyetracking”, in Proceedings of the Proceedings of the 13th International Conference on Interacción Persona-Ordenador; Association for Computing Machinery: New York, NY, USA, October 3 2012; pp. 1–8.
  45. R. Bednarik, M. Tukiainen, “Validating the Restricted Focus Viewer: A Study Using Eye-Movement Tracking”, in Behavior Research Methods 2007, 39, vol. pp. 274–282.
  46. Q. Wang, S. Yang, M. Liu, Z. Cao, Q. Ma, “An Eye-Tracking Study of Website Complexity from Cognitive Load Perspective”, in Decision Support Systems 2014, vol. 62, pp. 1–10.
  47. A. Çöltekin, B. Heil, S. Garlandini, S.I. Fabrikant, “Evaluating the Effectiveness of Interactive Map Interface Designs: A Case Study Integrating Usability Metrics with Eye-Movement Analysis”, in Cartography and Geographic Information Science 2009, vol. 36, pp. 5–17.
  48. C. Ehmke, S.G. Wilson, “Identifying Web Usability Problems from Eye-Tracking Data”, September 1 2007; Vol. 1, pp. 119–128.
  49. J. Luan, Z. Yao, F. Zhao, H. Liu, “Search Product and Experience Product Online Reviews”, in Comput. Hum. Behav. 2016, vol. 65, pp. 420–430.
  50. J. Falkowska, B. Kilijańska, J. Sobecki, K. Zerka, “Microinteractions of Forms in Web Based Systems Usability and Eye 436 Tracking Metrics Analysis” in International Conference on Applied Human Factors and Ergonomics. Springer, Cham, 2018, pp. 164–174.
  51. J. Sauro, J.R. Lewis, “Quantifying the User Experience: Practical Statistics for User Research”, 2nd edition, Morgan Kaufmann: Amsterdam Boston Heidelberg, 2016.