User Experience (UX) Evaluation in Gamified Processes: a bibliographic Review
Article Sidebar
Main Article Content
Introduction: in the context of technology design, the measurement and understanding of user experience (UX) have gained crucial relevance. This aspect addresses the pressing need to understand in detail how users interact with and experience technological artifacts. This review focuses on user experience in the design of gamified processes and products, considering that despite advances in inclusivity and accessibility, some individuals still cannot access the benefits of technological tools.
Objective: the objective of this documentary review is to identify and analyze the techniques and methods employed to evaluate user experience in the context of gamified processes and products.
Methods: a structured literature review was utilized to gather information on the techniques and metrics used in UX evaluation. A diversity of indicators was examined, ranging from usability to more subjective aspects such as aesthetics, as well as the evaluation approaches applied in previous studies.
Results: the findings revealed a wide variety of indicators and metrics used in the evaluation of UX in technological environments. Both the differences and similarities in evaluation practices were highlighted. The review underscored the complexity of user experience and the importance of considering not only usability but also broader aspects, especially in the realm of gamification.
Conclusions: is concluded that adopting multidisciplinary approaches and using validated tools is essential to achieve a more comprehensive and applicable understanding of gamified products or processes. This will enhance understanding at the intersection of technology and gamification.
Acosta Espinoza JL, Lenin León Yacelga AR, Sanafria Michilena WG, Acosta Espinoza JL, Lenin León Yacelga AR, Sanafria Michilena WG. Las aplicaciones móviles y su impacto en la sociedad. Revista Universidad y Sociedad [Internet]. 2022 [cited 2024 Sep 25];14(2):237–43. Available from: http://scielo.sld.cu/scielo.php?script=sci_arttext&pid=S2218-36202022000200237&lng=es&nrm=iso&tlng=en
Brooke J. SUS-A quick and dirty usability scale. 1996 [cited 2024 Sep 25]; Available from: https://seg.nju.edu.cn/tools/smarttv/download/A%20quick%20and%20dirty%20usability%20scale.pdf
Córdoba Cely C. La experiencia de usuario: de la utilidad al afecto. Iconofacto [Internet]. 2013 Oct [cited 2024 Sep 25];9(4):56–70. Available from: https://repository.upb.edu.co/handle/20.500.11912/7334
Cziksentmihalyi M. Flow: The Psychology of Optimal Experience. Harper & Row [Internet]. 1990 Jan [cited 2024 Sep 26]; Available from: https://www.researchgate.net/publication/224927532_Flow_The_Psychology_of_Optimal_Experience
Davis FD, Bagozzi RP, Warshaw PR. Fred D. Davis, Richard P. BaUser Acceptance of Computer Technology: A Comparison of Two Theoretical Models. Management_Science [Internet]. 1989 [cited 2024 Sep 26];35(8):982–1003. Available from: https://pubsonline.informs.org/doi/10.1287/mnsc.35.8.982 DOI: https://doi.org/10.1287/mnsc.35.8.982
Palomino Diaz FP, Freddy Alberto Paz Espinoza (Ass), Mag Miguel Arturo Moquillaza Vizarreta (Ass). Investigación de los métodos de evaluación de la experiencia de usuario que aplican analítica web: una revisión sistemática. 2021 Mar 8 [cited 2024 Sep 26]; Available from: https://tesis.pucp.edu.pe/repositorio//handle/20.500.12404/18523
Nosheen M, Sayed Z, Saad Malik M, Fahiem MA. An Evaluation Model for Measuring the Usability of Mobile. Office Applications through User Interface Design Metrics. 641 Mehran University Research Journal of Engineering & Technology [Internet]. 2019 [cited 2024 Sep 26];38(3):2413–7219. Available from: https://pdfs.semanticscholar.org/b552/f4bf41ecbc1e85aed7afffb39f86d86ec560.pdf
Rodriguez J, Cardoso P, Monteiro J, Figueiredo M. Handbook of Research on Human-Computer Interfaces, Developments, and Applications. Vol. 1. Hershey_PA: IGI_Global; 2016. Avalaible from: DOI: https://doi.org/10.4018/978-1-5225-0435-1
Google. Google Trends. 2023 [cited 2024 Sep 26]. Avalaible from: https://trends.google.es/trends/
Hay L. Researching UX: Analytics: Understanding Is the Heart of Great UX [Internet]. 1st ed. Vol. 1. SitePoint; 2017 [cited 2024 Sep 26]. Available from: https://books.google.com.co/books?hl=es&lr=&id=ZczDEAAAQBAJ&oi=fnd&pg=PT2&dq=Researching+UX:+Analytics:+Understanding+Is+the+Heart+of+Great+UX&ots=SicJ-kARPU&sig=5ZZHyqCSAJIiLIPRn05smK1x_X8&redir_esc=y#v=onepage&q=Researching%20UX%3A%20Analytics%3A%20Understanding%20Is%20the%20Heart%20of%20Great%20UX&f=false
Hassenzahl M, Monk A. The inference of perceived usability from beauty. Human-Computer Interaction [Internet]. 2010 Jul [cited 2024 Sep 27];25(3):235–60. Available from: https://doi.org/10.1080/07370024.2010.500139
Hassenzahl M, Monk A. The inference of perceived usability from beauty. Human-Computer Interaction [Internet]. 2010 Jul [cited 2024 Sep 27];25(3):235–60. Available from: https://doi.org/10.1080/07370024.2010.500139 DOI: https://doi.org/10.1080/07370024.2010.500139
International Organization for Standardization OIS. ISO 9241-210:2019(en), Ergonomics of human-system interaction — Part 210: Human-centred design for interactive systems [Internet]. 2017 [cited 2024 Sep 27]. Available from: https://www.iso.org/obp/ui/en/#iso:std:iso:9241:-210:ed-2:v1:en
Joyce A. Inclusive Design [Internet]. 2022 [cited 2024 Sep 27]. Available from: https://www.nngroup.com/articles/inclusive-design/
Law ELC, Roto V, Hassenzahl M, Vermeeren APOS, Kort J. Understanding, scoping and defining user experience: A survey approach. Conference on Human Factors in Computing Systems - Proceedings [Internet]. 2009 Apr [cited 2024 Sep 27];719–28. Available from: https://dl.acm.org/doi/10.1145/1518701.1518813 DOI: https://doi.org/10.1145/1518701.1518813
U.S. General Services Administration (GSA), Department of Health and Human Services (HHS). Research-Based Web Design & Usability Guidelines. 2006. Available from: https://www.hhs.gov/sites/default/files/research-based-web-design-and-usability-guidelines_book.pdf
Lewis C, Rieman J. Task-Centered User Interface Design A Practical Introduction. 2008 Dec 16 [cited 2024 Sep 27]; Available from: http://www.hcibib.org/tcuid/
Mordecki D. Miro y entiendo. Guía práctica de usabilidad web. Ed. Acad. Española; 2012. Available from: https://mordecki.com/wp-content/uploads/2021/06/MiroyEntiendo.pdf
Max-Neef M, Elizalde A, Hopenhayn M, Herrera F, Zemelman H, Jatoba J, et al. Desarrollo a Escala Humana una opcion para el future version de Cepaur Fundacion Dag Hammarskjold [Internet]. Development Dialogue Numero especial. Londres: CEPAUR; 1986 [cited 2024 Sep 27]. Available from: https://www.daghammarskjold.se/wp-content/uploads/1986/08/86_especial.pdf
Nielsen J. Usability Engineering [Internet]. MountainView: Morgan_Kaufmann; 1994 [cited 2024 Sep 27]. Available from: https://books.google.com.co/s?hl=es&lr=&id=95As2OF67f0C&oi=fnd&pg=PR9&dq=Usability+Engineering.+Morgan+Kaufmann.+&ots=3dAECmbp0r&sig=_8lbmfL1L9O1wdc1MDxg69yp498&redir_esc=y#v=onepage&q=Usability%20Engineering.%20Morgan%20Kaufmann.&f=false
O’Brien HL, Toms EG. What is user engagement? A conceptual framework for defining user engagement with technology. Journal of the American Society for Information Science and Technology [Internet]. 2008 Apr 1 [cited 2024 Sep 27];59(6):938–55. Available from: https://onlinelibrary.wiley.com/doi/full/10.1002/asi.20801 DOI: https://doi.org/10.1002/asi.20801
Zaharias P, Mehlenbacher B. Exploring User Experience (UX) in virtual learning environments. International Journal of Human Computer Studies [Internet]. 2012 [cited 2024 Sep 27];70(7):475–7. Available from: https://www.researchgate.net/publication/257481786_Exploring_User_Experience_UX_in_virtual_learning_environments DOI: https://doi.org/10.1016/j.ijhcs.2012.05.001
Bowman DA. The Encyclopedia of Human-Computer Interaction, 2nd Ed. | IxDF [Internet]. 2014 [cited 2024 Sep 27]. Available from: https://www.interaction-design.org/literature/book/the-encyclopedia-of-human-computer-interaction-2nd-ed?srsltid=AfmBOor64-mg_jtQ3c42qYD8B4_4gfYn5YQLa5TdDcAWKamQjE5y562W
Sharp Helen, Rogers Yvonne, Preece Jenny. Interaction design : beyond human-computer interaction [Internet]. 4th ed. John Wiley & Sons, Inc.; 2023 [cited 2024 Sep 27]. 720. Available from: https://katalog.ub.uni-heidelberg.de/cgi-bin/titel.cgi?katkey=69228294
Baranauskas C. Palanque, P. Abascal JDS, Barbosa J. Human-Computer Interaction – INTERACT 2007. Baranauskas C, Palanque P, Abascal J, Barbosa SDJ, editors. 2007 Sep 10 [cited 2024 Sep 27];4663. Available from: http://link.springer.com/10.1007/978-3-540-74800-7 DOI: https://doi.org/10.1007/978-3-540-74796-3
Yáñez-Gómez R, Cascado-Caballero D, Sevillano JL. Academic methods for usability evaluation of serious games: a systematic review. Multimedia Tools and Applications [Internet]. 2017 Feb 1 [cited 2024 Sep 27];76(4):5755–84. Available from: https://link.springer.com/article/10.1007/s11042-016-3845-9 DOI: https://doi.org/10.1007/s11042-016-3845-9
Dankov Y, Bontchev B. Towards a Taxonomy of Instruments for Facilitated Design and Evaluation of Video Games for Education. ACM International Conference Proceeding Series [Internet]. 2020 Jun 19 [cited 2024 Sep 27];285–92. Available from: https://doi.org/10.1145/3407982.3408010 DOI: https://doi.org/10.1145/3407982.3408010
Scolari CA 1963. Homo Videoludens 2.0: de Pacman a la gamification [Internet]. Barcelona: Universidad Mayor; Universitat de Barcelona; 2013 [cited 2024 Sep 27]. Available from: http://repositori.upf.edu/handle/10230/26009
Maheshkar Chandan, Sharma Vinod. Handbook of research on cross-cultural business education [Internet]. IGI Global; 2018 [cited 2024 Sep 27]. 463. Available from: https://books.google.com/books/about/Handbook_of_Research_on_Cross_Cultural_B.html?hl=es&id=elkatAEACAAJ DOI: https://doi.org/10.4018/978-1-5225-3776-2.ch001
Browne K, Anand C, Gosse E. Gamification and serious game approaches for adult literacy tablet software. Entertainment Computing [Internet]. 2014 Aug 1 [cited 2024 Sep 27];5(3):135–46. Available from: https://www.sciencedirect.com/science/article/pii/S1875952114000147 DOI: https://doi.org/10.1016/j.entcom.2014.04.003
Cairns P, Cox A, Imran Nordin A. Immersion in Digital Games: Review of Gaming Experience Research. Handbook of Digital Games [Internet]. 2014 Apr 7 [cited 2024 Sep 27];337–61. Available from: https://onlinelibrary.wiley.com/doi/full/10.1002/9781118796443.ch12 DOI: https://doi.org/10.1002/9781118796443.ch12
Abt CC. Serious games. UniversityPressOfAmerica [Internet]. 1987 [cited 2024 Sep 27];176. Available from: https://books.google.com.co/books?hl=es&lr=&id=axUs9HA-hF8C&oi=fnd&pg=PR13&dq=serious+games+abt+clark&ots=d0S3bey7uS&sig=zfwxfi6NEuDvLY_1exgiWemb2AE&redir_esc=y#v=onepage&q=serious%20games%20abt%20clark&f=false
Calderón A, Ruiz M. A systematic literature review on serious games evaluation: An application to software project management. Computers & Education [Internet]. 2015 Sep 1 [cited 2024 Sep 27];87:396–422. Available from: https://www.sciencedirect.com/science/article/pii/S0360131515300166 DOI: https://doi.org/10.1016/j.compedu.2015.07.011
Gebhard P, Tsovaltzi D, Schneeberger T, Nunnari F. Serious Games with SIAs. The Handbook on Socially Interactive Agents [Internet]. 2022 Oct 27 [cited 2024 Sep 27];527–46. Available from: https://dl.acm.org/doi/10.1145/3563659.3563676 DOI: https://doi.org/10.1145/3563659.3563676
Zichermann G, Cunningham C. Gamification by design: Implementing game mechanics in web and mobile apps [Internet]. 1st ed. books.google.com. O´Reilly_Media; 2011 [cited 2024 Sep 27]. Available from: https://books.google.com/s?hl=es&lr=&id=Hw9X1miVMMwC&oi=fnd&pg=PR7&dq=Gamification+by+design:+Implementing+game+mechanics+in+web+and+mobile+apps&ots=0tjkbq4von&sig=VH4TQbWDLcvpHNGqy0oTkdUb5eY
Plaisant C, Shneiderman B. Show me! guidelines for producing recorded demonstrations. Proceedings - 2005 IEEE Symposium on Visual Languages and Human-Centric Computing [Internet]. 2005 Oct [cited 2024 Sep 27];171–8. Available from: https://ieeexplore.ieee.org/abstract/document/1509501
W3C. Web Content Accessibility Guidelines (WCAG) 2.1 [Internet]. 2023 [cited 2024 Sep 27]. Available from: https://www.w3.org/TR/WCAG21/
Center for Universal Design. The Principles of universal design. lanecc.edu [Internet]. 1997 [cited 2024 Sep 27]; Available from: https://www.lanecc.edu/sites/default/files/governance/facilities/principlesofuniversaldesign06-07.pdf
Cresswell JW, Plano-Clark VL, Gutmann ML, Hanson WE. Advanced mixed methods research designs. Handbook of Mixed Methods in Social and Behavioral Research [Internet]. 2003 [cited 2024 Sep 27];209–40. Available from: http://www.sagepub.com/upm-data/19291_Chapter_7.pdf
Kohavi R, Henne RM, Sommerfield D. Practical guide to controlled experiments on the web: Listen to your customers not to the hippo. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining [Internet]. 2007 Aug 12 [cited 2024 Sep 27];959–67. Available from: https://dl.acm.org/doi/10.1145/1281192.1281295 DOI: https://doi.org/10.1145/1281192.1281295
Dumas JF, Redish JC. A Practical Guide to Usability Testing | Guide books. Greenwood Publishing Group Inc [Internet]. 1993 Jan 1 [cited 2024 Sep 27]; 412. Available from: https://dl.acm.org/doi/abs/10.5555/573014
Rubin J, Chisnell D. Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests [Internet]. 2nd ed. Indianapolis: Wiley_Publishing_Inc; 2008 [cited 2024 Sep 28]. 348. Available from: https://books.google.com/s?hl=es&lr=&id=MjNGDgAAQBAJ&oi=fnd&pg=PA3&ots=lPwI1IoWkC&sig=RHecbKlnI01u2T8N4ki_sjd1BTE
Brooke J. SUS-A quick and dirty usability scale. 1996 [cited 2024 Sep 28];7. Available from: https://seg.nju.edu.cn/tools/smarttv/download/A%20quick%20and%20dirty%20usability%20scale.pdf
Lewis JR, Sauro J. The Factor Structure of the System Usability Scale. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) [Internet]. 2009 [cited 2024 Sep 28];5619 LNCS:94–103. Available from: https://link.springer.com/chapter/10.1007/978-3-642-02806-9_12
Sauro J, Lewis JR. Quantifying the User Experience: Practical Statistics for User Research [Internet]. 2nd ed. Morgan_Kaufmann; 2016 [cited 2024 Sep 28]. 350. Available from: https://books.google.com.co/books?hl=es&lr=&id=USPfCQAAQBAJ&oi=fnd&pg=PP1&dq=Quantifying+the+User+Experience:+Practical+Statistics+for+User+Research&ots=VzYiX-9pRj&sig=OuMXzqYeL7RquguRQQbgdhBnxbs&redir_esc=y#v=onepage&q=Quantifying%20the%20User%20Experience%3A%20Practical%20Statistics%20for%20User%20Research&f=false
Dillman DA, Smyth JD, Christian LM. Internet, Phone, Mail and Mixed-Mode Surveys: The Tailored Design Method. redalyc.org [Internet]. 2016 [cited 2024 Sep 28]; 154:161–76. Available from: https://www.redalyc.org/pdf/997/99746727009.pdf
Zichermann G, Cunningham C. Gamification By Design [Internet]. 1st ed. Vasa. O´Reilly; 2011 [cited 2024 Sep 28]. 208. Available from: https://books.google.com.co/s?hl=es&lr=&id=Hw9X1miVMMwC&oi=fnd&pg=PR7&dq=Gamification+by+design:+Implementing+game+mechanics+in+web+and+mobile+apps&ots=0tjk8o7vns&sig=egG7FmR_OjuM1GiiHhU2KXZrKis&redir_esc=y#v=onepage&q=Gamification%20by%20design%3A%20Implementing%20game%20mechanics%20in%20web%20and%20mobile%20apps&f=false
Werbach K, Hunter D. For the Win, Revised and Updated Edition. The Power of Gamification and Game Thinking in Business, Education, Government, and Social Impact [Internet]. For the Win, Revised and Updated Edition. Philadelphia: University of Pennsylvania Press; 2020 [cited 2024 Sep 28]. Available from: https://doi.org/10.9783/9781613631041 DOI: https://doi.org/10.9783/9781613631041
McGonigal Jane. Reality Is Broken: Why Games Make Us Better and How They Can Change the World [Internet]. NewYork: Penguin_Books; 2011 [cited 2024 Sep 28]. Available from: https://books.google.com.co/s?hl=es&lr=&id=yiOtN_=fnd&pg=PT11&dq=Reality+is+broken:+Why+games+make+us+better+and+how+they+can+change+the+world&ots=flgkTHa_Tv&sig=O30WMx531vsU7_2sFnkKDr1FpFI&redir_esc=y#v=onepage&q=Reality%20is%20broken%3A%20Why%20games%20make%20us%20better%20and%20how%20they%20can%20change%20the%20world&f=false
Reiners T, Wood LC. Gamification in education and business. Gamification in Education and Business [Internet]. 2015 Jan 1 [cited 2024 Sep 28];1–710. Available from: https://link.springer.com/book/10.1007/978-3-319-10208-5
Guirao Goris SJA. Utilidad y tipos de revisión de literatura. Ene [Internet]. 2015 Jan [cited 2024 Sep 28];9(2):0–0. Available from: https://scielo.isciii.es/scielo.php?script=sci_arttext&pid=S1988-348X2015000200002&lng=es&nrm=iso&tlng=es DOI: https://doi.org/10.4321/S1988-348X2015000200002
Accepted 2024-08-08
Published 2024-10-02
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Authors grant the journal and Universidad del Valle the economic rights over accepted manuscripts, but may make any reuse they deem appropriate for professional, educational, academic or scientific reasons, in accordance with the terms of the license granted by the journal to all its articles.
Articles will be published under the Creative Commons 4.0 BY-NC-SA licence (Attribution-NonCommercial-ShareAlike).