Defining the Visual Complexity of Learning Management Systems Using Image Metrics and Subjective Ratings


  • Brenda M. Stoesz University of Manitoba
  • Mehdi Niknam University of Manitoba
  • Jessica Sutton University of Manitoba



eye tracking, learning management system, user interface design, visual complexity, visual perception


Research has demonstrated that students’ learning outcomes and motivation to learn are influenced by the visual design of learning technologies (e.g., learning management systems or LMS). One aspect of LMS design that has not been thoroughly investigated is visual complexity. In two experiments, postsecondary students rated the visual complexity of images of LMS after exposure durations of 50-500 ms. Perceptions of complexity were positively correlated across timed conditions and working memory capacity was associated with complexity ratings. Low-level image metrics were also found to predict perceptions of the LMS complexity. Results demonstrate the importance of the visual complexity of learning technologies and suggest that additional research on the impact of LMS design on learning outcomes is warranted.


Attneave, F. (1957). Physical determinants of the judged complexity of shapes. Journal of Experimental Psychology, 53(4), 221–227. doi:10.1037/h0043921

Baddeley, A. (2012). Working memory: Theories, models, and controversies. Annual Review of Psychology, 63(1), 1–29.

Bates, T., Desbiends, B., Donovan, T., Martel, E., Mayer, D., Paul, R., … Seaman, J. (2017). Tracking online and distance education in Canadian universites and colleges: 2017. The National Survey of Online and Distance Education in Canadian Post-Secondary Education, Vancouver, BC. Retrieved from

Berlyne, D. E. (1960). Conflict, arousal, and curiosity. New York: McGraw Hill.

Berners-Lee, T. (n.d.). The WorldWideWeb browser. Retrieved from

Boot, W. R., Charness, N., Czaja, S. J., Sharit, J., Rogers, W. A., Fisk, A. D., … Nair, S. (2015). Computer proficiency questionnaire: Assessing low and high computer proficient seniors. Gerontologist, 55(3), 404–411. doi:10.1093/geront/gnt117

Bornstein, R. F., & D’Agostino, P. R. (1992). Stimulus recognition and the mere exposure effect. Journal of Personality and Social Psychology, 63(4), 545–552. doi:10.1037/0022-3514.63.4.545

Brunner-Sperdin, A., Scholl-Grissemann, U. S., & Stokburger-Sauer, N. E. (2014). The relevance of holistic website perception. How sense-making and exploration cues guide consumers’ emotions and behaviors. Journal of Business Research, 67(12), 2515–2522. doi:10.1016/j.jbusres.2014.02.004

Christianson, S. (1992). The handbook of emotion and memory: Research and theory. Hillsdale, NJ: Erlbaum Associates.

Coates, H., James, R., & Baldwin, G. (2005). A critical examination of the effects of learning management systems on university teaching and learning. Tertiary Education and Management, 11(1), 19–36. doi:10.1080/13583883.2005.9967137

Corchs, S. E., Ciocca, G., Bricolo, E., & Gasparini, F. (2016). Predicting complexity perception of real world images. PLoS ONE, 11(6), 1–22. doi:10.1371/journal.pone.0157986

Cyr, D., Head, M., & Larios, H. (2010). Colour appeal in website design within and across cultures: A multi-method evaluation. International Journal of Human Computer Studies, 68(1–2), 1–21. doi:10.1016/j.ijhcs.2009.08.005

Cyr, D., Head, M., Larios, H., & Pan, B. (2009). Exploring human images in website design: A multi-method approach. MIS Quarterly, 33(3), 539–566. doi:10.2307/20650308

Deng, L., & Poole, M. S. (2010). Affect in web interfaces: A study of the impacts of web page visual complexity and order. MIS Quarterly, 34(4), 711. doi:10.2307/25750702

Ellis, R. K. (Ed.). (2009). Field guide to learning management systems. American Society for Training & Development Learning Circuits Retrieved from

Eveland Jr., W. P., & Dunwoody, S. (2001). User control and structural isomorphism or disorientation and cognitive load? Learning from the web versus print. Communication Research, 28(1), 48–78. doi:10.1177/009365001028001002

Forsythe, A., Mulhern, G., & Sawey, M. (2008). Confounds in pictorial sets: The role of complexity and familiarity in basic-level picture processing. Behavior Research Methods, 40(1), 116–129. doi:10.3758/BRM.40.1.116

Gartus, A., & Leder, H. (2017). Predicting perceived visual complexity of abstract patterns using computational measures: The influence of mirror symmetry on complexity perception. PLoS ONE, 12(11), 1–29. doi:10.1371/journal.pone.0185276

Geissler, G. L., Zinkhan, G. M., & Watson, R. T. (2006). The influence of home page complexity on consumer attention, attitudes, and purchase intent. Journal of Advertising, 35(2), 69–80. doi:10.1080/00913367.2006.10639232

Gelman, A., & Park, D. K. (2008). Splitting a predictor at the upper quarter or third and the lower quarter or third. The American Statistician, 62(4), 1-8. doi:10.1198/000313008X366226

Goldberg, J. H. (2014). Measuring software screen complexity: Relating eye tracking, emotional valence, and subjective ratings. International Journal of Human-Computer Interaction, 30(7), 518–532. doi:10.1080/10447318.2014.906156

Harper, S., Michailidou, E., & Stevens, R. (2009). Toward a definition of visual complexity as an implicit measure of cognitive load. ACM Transactions on Applied Perception, 6(2), 1–18. doi:10.1145/1498700.1498704

Hasler, D., & Suesstrunk, S. E. (2003). Measuring colourfulness in natural images. Proceedings SPIE 5007 – Human Vision and Electronic Imaging VIII, 87–95. doi:10.1117/12.477378

Jacobsen, T. , & Höfel, L. (2003). Descriptive and evaluative judgment processes: Behavioral and electrophysiological indices of processing symmetry and aesthetics. Cognitive, Affective & Behavioral Neuroscience, 3(4), 289–299. doi:10.3758/CABN.3.4.289

Holmqvist, K., Nystrom, M., Andersson, R., Dewhurst, R., Jarodzka, H., & van de Weijer, J. (2015). Eye tracking: A comprehensive guide to methods and measures. Oxford, UK: Oxford University Press.

Huang, Y., Xue, X., Spelke, E., Huang, L., Zheng, W., & Peng, K. (2018). The aesthetic preference for symmetry dissociates from early-emerging attention to symmetry. Scientific Reports, 8(1), 1–8. doi:10.1038/s41598-018-24558-x

Ivory, M. Y., Sinha, R. R., & Hearst, M. A. (2001). Empirically validated web page design metrics. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI ’01, 53–60. New York, NY: Society for Computing Machinery. doi:10.1145/365024.365035

Ladyshewsky, R. K. (2004). E-learning compared with face to face: Differences in the academic achievement of postgraduate business students. Australasian Journal of Educational Technology, 20(3), 316–336. doi:10.14742/ajet.1350

Lazar, J., Bessiere, K., Ceaparu, I., Robinson, J., & Shneiderman, B. (2003). Help! I’m lost: User frustration in web navigation. IT & Society, 1(3), 18–26.

Lim, Y. M., Chee, K. N., Ayesh, A., & Stacey, M. (2013). Designing learning management system to encourage e-Learning sustainability. In: Proceedings of Innovation and Transformation in Learning and Teaching, Kauala Lumpure: Tunku Adbdul Rahman University College (pp. 76–83).

Lindgaard, G., Fernandes, G., Dudek, C., & Brown, J. (2006). Attention web designers: You have 50 milliseconds to make a good first impression! Behaviour & Information Technology, 25(2), 115–126. doi:10.1080/01449290500330448

Liu, Y. (2003). Engineering aesthetics and aesthetic ergonomics: Theoretical foundations and a dual-process research methodology. Ergonomics, 46(13–14), 1273–1292. doi:10.1080/00140130310001610829

Mabila, J., Gelderblom, H., & Ssemugabi, S. (2014). Using eye tracking to investigate first year students' digital proficiency and their use of a learning management system in an open distance environment. African Journal of Research in Mathematics, Science and Technology Education, 18(2), 151–163. doi:10.1080/10288457.2014.928449

Madan, C. R., Bayer, J., Gamer, M., Lonsdorf, T. B., & Sommer, T. (2018). Visual complexity and affect: Ratings reflect more than meets the eye. Frontiers in Psychology, 8(JAN), 1–19. doi:10.3389/fpsyg.2017.02368

McDougall, S. J., Curry, M. B., & de Bruijn, O. (1999). Measuring symbol and icon characteristics: Norms for concreteness, complexity, meaningfulness, familiarity, and semantic distance for 239 symbols. Behavior Research Methods, Instruments, & Computers: A Journal of the Psychonomic Society, Inc, 31(3), 487–519.

Means, B., Toyama, Y., Murphy, R., & Baki, M. (2013). The effectiveness of online and blended learning: A meta-analysis of the empirical literature. Teachers College Record, 115(030303), 1–47.

Michailidou, E., Harper, S., & Bechhofer, S. (2008). Visual complexity and aesthetic perception of web pages. SIGDOC 2008 - Proceedings of the 26th ACM International Conference on Design of Communication, 215–223. doi:10.1145/1456536.1456581

Moshagen, M., & Thielsch, M. T. (2010). Facets of visual aesthetics. International Journal of Human Computer Studies, 68(10), 689–709. doi:10.1016/j.ijhcs.2010.05.006

Nadal, M., Munar, E., Marty, G., & Cela-Conde, C. J. (2010). Visual complexity and beauty appreciation: Explaining the divergence of results. Empirical Studies of the Arts, 28(2), 173–191. doi:10.2190/EM.28.2.d

Nadkarni, S., & Gupta, R. (2007). A task-based model of perceived website complexity. MIS Quarterly, 31(3), 501–524. doi:10.2307/25148805

Nguyen, T. (2015). The effectiveness of online learning: Beyond no significant difference and future horizons. MERLOT Journal of Online Learning and Teaching, 11(2), 309–319. Retrieved from

Niknam, M., & Thulasiraman, P. (2020). LPR: A bio-inspired intelligent learning path recommendation system based on meaningful learning theory. Education and Information Technologies, 25, 3797–3819. doi:10.1007/s10639-020-10133-3

Ownby, R., & Czaja, S. (2008). Cognitive abilities that predict success in a computer-based training program. Gerontologist, 48(2), 170–180. doi: 10.1093/geront/48.2.170

Pecchinenda, A., Bertamini, M., Makin, A. D. J., & Ruta, N. (2014). The pleasantness of visual symmetry: Always, never or sometimes. PLoS ONE, 9(3), 1–10. doi:10.1371/journal.pone.0092685

Pomales-García, C., Liu, Y., & Mendez, D. (2005). Web-based distance learning technology: Does appearance matter? Proceedings of the Human Factors and Ergonomics Society, 49(7), 755–758. doi:10.1177/154193120504900703

Reinecke, K., Yeh, T., Miratrix, L., Mardiko, R., Zhao, Y., Liu, J., & Gajos, K. Z. (2013). Predicting users’ first impressions of website aesthetics with a quantification of perceived visual complexity and colorfulness. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems – CHI ’13, 2049–2058. doi:10.1145/2470654.2481281

Sharp, J. G., Hemmings, B., Kay, R., & Sharp, J. C. (2017). Academic boredom and the perceived course experiences of final year education studies students at university. Journal of Further and Higher Education, 2017(5), 600–627. doi:10.1080/0309877X.2017.1386287

Shen, Z., Popov, V., Delahay, A. B., & Reder, L. M. (2018). Item strength affects working memory capacity. Memory and Cognition, 46(2), 204–215. doi:10.3758/s13421-017-0758-4

Smith, A. D. (2017). Exploring the inherent growth of e-tailing via e-personalization and technological innovations. International Journal of Innovation in the Digitial Economy, 8(1), 1–28. doi:10.4018/IJIDE.2017010102

Sonderegger, A., & Sauer, J. (2010). The influence of design aesthetics in usability testing: Effects on user performance and perceived usability. Applied Ergonomics, 41(3), 403–410. doi:10.1016/j.apergo.2009.09.002

Stoesz, B. M., & Jakobson, L. S. (2014). Developmental changes in attention to faces and bodies in static and dynamic scenes. Frontiers in Psychology, 5(193), 1–9. doi:10.3389/fpsyg.2014.00193

Tallent-Runnels, M. K., Thomas, J. A., Lan, W. Y., Cooper, S., Ahern, T. C., Shaw, S. M., & Liu, X. (2006). Teaching courses online: A review of the research. Review of Educational Research, 76(1), 93–135. doi:10.3102/00346543076001093

Tharangie, K. G. D., Irfan, C. M. ., Marasinghe, C. ., & Yamada, K. (2008). Kansei engineering assessing system to enhance the usability in e-learning web interfaces: Colour basis. Supplementary Proceedings of the 16th International Conference on Computers in Education, 145–150. Taipei, Taiwan.

Tractinsky, N., Cokhavi, A., Kirschenbaum, M., & Sharfi, T. (2006). Evaluating the consistency of immediate aesthetic perceptions of web pages. International Journal of Human Computer Studies, 64(11), 1071–1083. doi:10.1016/j.ijhcs.2006.06.009

Tuch, A. N., Bargas-Avila, J. A., Opwis, K., & Wilhelm, F. H. (2009). Visual complexity of websites: Effects on users’ experience, physiology, performance, and memory. International Journal of Human Computer Studies, 67(9), 703–715. doi:10.1016/j.ijhcs.2009.04.002

Wagemans, J. (1997). Characteristics and models of human symmetry detection. Trends in Cognitive Sciences, 1(9). 346–352. doi:10.1016/S1364-6613(97)01105-4

Walhout, J., Brand-Gruwel, S., Jarodzka, H., Van Dijk, M., De Groot, R., & Kirschner, P. A. (2015). Learning and navigating in hypertext: Navigational support by hierarchical menu or tag cloud? Computers in Human Behavior, 46, 218–227. doi:10.1016/j.chb.2015.01.025

Wang, Q., Yang, S., Liu, M., Cao, Z., & Ma, Q. (2014). An eye-tracking study of website complexity from cognitive load perspective. Decision Support Systems, 62, 1–10. doi:10.1016/j.dss.2014.02.007

Wechsler, D. (2008). WAIS-IV administration and scoring manual. San Antonio, TX: Psychological Corporation.

Zain, J. M., Tey, M., & Goh, Y. (2011). Does aesthetics of web page interface matters to Mandarin learning?, International Journal of Computer Science and Network Security, 7(8), 43–51. Retrieved from arXiv:1101.1608

Zajonc, R. B. (1968). Attitudinal effects of mere exposure. Journal of Personality and Social Psychology Monographs, 2(2 Pt. 1), 1-27. doi:10.1037/h0025848

Zheng, X. S., Chakraborty, I., Lin, J. J.-W., & Rauschenberger, R. (2009). Correlating low-level image statistics with users - rapid aesthetic and affective judgments of web pages. Proceedings of the 27th International Conference on Human Factors in Computing Systems - CHI 09, 1–10. doi:10.1145/1518701.1518703