Student Perceptions of the Visual Design of Learning Management Systems

Research on the impact of the visual design of the user interface of learning management systems (LMS) on learning experience is sparse. The purpose of this study was to conduct a preliminary examination of students’ perceptions of the visual design of their postsecondary institutions’ LMS and their learning experiences using survey methodology (N= 46). Students generally agreed that the course


Introduction
Approximately 2,183,973 individuals were enrolled in courses at Canadian postsecondary institutions in the 2019-2020 academic year (Statistics Canada, 2022).Before the COVID-19 pandemic, an estimated 30% of students were completing online courses (EduConsillium, 2015), but at the height of the pandemic, 92% of students were taking at least one course online (Doreleyers & Knighton, 2020).Moreover, many courses with face-to-face instruction are typically supplemented with online components (e.g., availability of learning resources and assessments on course websites or LMSn).Thus, the development of effective e-learning technologies is important for postsecondary institutions and the large numbers of students and faculty who use them.Previous research has examined the potential value of online learning by asking administrators (Bates et al., 2017) and faculty (Schönwetter & Reynolds, 2013) about their perspectives and barriers to use.Few studies, however, have detailed postsecondary students' perspectives of online learning and even fewer have asked them about their perspectives of the visual design of the LMS they are required to use.The primary goal of this study was to examine students' perceptions of the visual design of the LMS and how these perceptions related to learning experiences, engagement, and achievement outcomes using survey methodology.

Background
Online learning has been defined as learning facilitated by technology (specifically via an LMS) in which students interact with instructors synchronously or asynchronously and where the location of study is not dependent on a particular location (Singh & Thurman, 2019).Advancements in technology associated with online learning enabled postsecondary institutions to pivot their course offerings from face-to-face to remote teaching and rapid learning in response to the COVID-19 pandemic.Prior to the pandemic, online education was already growing rapidly and was due, in part, to interest in enhancing learning outcomes and coping with decreases in resources allocated to teaching and learning in higher education (Farinella et al., 2000).Some researchers have shown that students enrolled in online courses have better outcomes in numerous areas, such as understanding the course structure, communication with instructors, increased engagement and satisfaction with course material, and higher final grades, compared to face-to-face courses (Panigrahi et al., 2018;Pei & Wu, 2019;Soffer & Nachmias, 2018).Students have reported appreciation of the flexibility and convenience of online learning (Sanford et al., 2017;Song et al., 2004) and the deeper reflection that is often required prior to engaging in learning activities (Dumford & Miller, 2018).Students have also valued online courses as they are often more student-centered than traditional face-to-face lecture-based courses (Petrides, 2002;Richardson et al., 2017) as well as offering unique opportunities for instructor-student and student-student engagement (Martin & Bolliger, 2018).
Despite the documented strengths of online learning, students have reported that delays in feedback from other students and instructors (Petrides, 2002), lack of community and feelings of isolation (Kebritchi et al., 2017;Vanslambrouck et al., 2018;Vonderwell, 2003), and technological problems are limitations of online learning (Rasheed et al., 2020;Song et al., 2004).Students' levels of online learning readiness, specifically their online learning efficacy or the belief in the effectiveness of online learning vs traditional face-to-face learning, may also influence their learning outcomes significantly (Joosten & Cusatis, 2020).Moreover, instructor attitudes and dispositions toward the online modality (Cutri & Mena, 2020;Los et al., 2021) emphasized by the technological and pedagogical challenges and adaptations often necessitated by the online modality (e.g., teaching styles, time management, and content development) (Kebritchi et al., 2017) may have an impact on student perceptions of the effectiveness of online learning.Lastly, when instructors use features of the LMS to enhance the visual design of the LMS and the learning materials, students are more likely to engage with the LMS, resulting in positive benefits to their learning (Ghapanchi et al., 2020).
Visual design of a product/interface encompasses both functionality and aesthetics.Effective designs require that designers consider the nature of the users, how users will interact with the product, and other variables that influence user experience (Bader & Lowenthal, 2018).Functionality and aesthetics are viewed as intertwined yet aesthetics can sometimes be regarded as less important in online teaching and learning spaces.Aesthetics may be lower priority for some LMS developers because this aspect of design is a "wicked problem".Such problems are difficult to define, solutions cannot be tested immediately, many solutions are possible, and each solution is uniquely experienced by each user (Ritter & Webber, 1973).Without paying attention to the visual aspects of LMS interfaces, however, "LMS designs become impersonal, insipid, and uninspiring" to the learner and the instructor (Bader & Lowenthal, 2018, p. 28).Such impressions could lead to lack of satisfaction and a downward trend in the use of an LMS by teachers and learners.
Of relevance to the present study is the research examining the impact of visual complexity on students' online learning experiences.Poor visual design (including inappropriate degree of visual complexity and colourfulness) can overload learners' cognitive resources resulting in disorientation and reduced learning (Christianson, 1992;Eveland Jr. & Dunwoody, 2001;Sharp et al., 2017).Stoesz et al. (2020) reported negative correlations between a measure of working memory capacity (i.e., Digit Span subtest of the Wechsler Adult Intelligence Scale, Fourth Edition; Wechsler, 2008) and participants' ratings of the visual complexity of LMS images.These findings suggested that limiting the quantity of information and simplifying structure and colour when designing LMS interfaces and subject-matter content are important.There were two limitations in this work, however.Stoesz et al. had examined the relationship between subjective complexity ratings of LMS images and working memory capacity in a laboratory setting; further, the participants may have had limited experience with the various LMS' that were presented to them.In addition, Stoesz et al. did not examine whether students perceived LMS colour and other visual design dimensions as facilitators of or barriers to their learning.
The presence of colour information supports encoding and retrieval processes (Cortese et al., 2019;Dzulkifli & Mustafar, 2013;Gegenfurtner & Rieger, 2000).Using a delayed match-to-sample task, Gegenfurtner and Rieger (2000) found that recognition accuracy was higher for color images than for luminance-matched black and white images of natural scenes that were presented to participants for very brief durations.The authors attributed this finding to the "enriched representation of the colour images in short-term memory" (p.806), meaning that colour provided viewers with additional sensory cues that might be useful later when the images need to be remembered.Colours also draw attention and generate emotional responses (e.g., enjoyment, satisfaction, frustration), thereby enhancing (or impeding) learning (Ashrafi et al., 2020;Tharangie et al., 2008).Positive perceptions of LMS presentation and navigation (e.g., graphic design, colour, and layout) also appeared to be associated with the perceived usefulness of the LMS, which in turn may be a significant predictor of the intention to continue to use the LMS (Ashrafi et al., 2020).

Present Study
The purpose of this study was to examine students' perceptions of the visual design of the LMS vis-à-vis their learning engagement and achievement outcomes, using survey methodology.We were particularly interested in exploring the relationships between educational experience (e.g., terms or years of experience using an LMS), familiarity and satisfaction with an LMS, visual design (including colour) of the LMS, and satisfaction with the learning experience.The findings contribute to knowledge of the students' perceptions of the importance of the design of online teaching and learning environments considering their overall learning experiences in Canadian postsecondary institutions.

Participants
Participants enrolled at postsecondary educational institutions across Canada were eligible to take part in this study.Recruitment through various Canadian students' associations was attempted, however, only one association agreed to distribute the study information to its members.Thus, social media advertisement in Canada and snowball sampling were also employed as recruitment strategies.Snowball sampling involved distributing the study information (including the link to the survey) via an email contact list maintained by the first author.Contacts included collaborators, educators, and previous research participants who had consented to be contacted for opportunities to take part in future studies.These contacts were asked to share the recruitment email.Clicking on the link in the email directed students to the online survey administered via Qualtrics.The study protocol was approved by the research ethics board at the University of Manitoba and from several other postsecondary institutions across Canada.

Materials and Procedure
We developed the four-section Use and Perceptions of Learning Management Systems Survey adapting items from the literature focused on online learning and web usability (Lavie & Tractinsky, 2004;Schönwetter & Reynolds, 2013;Waheed et al., 2016).We have formal training and practical experience in teaching and learning, online learning, and LMS implementation, training others to use the LMS, and examining user experiences.As such, items were selected by the first two authors for the present study based on each item's face validity.Each of the four sections of the survey are described below.
The General Use of Your LMS section consisted of 7 items about routine use of the LMS (e.g., How many terms have you used the LMS?) and 12 items designed to collect information about the ability to perform certain tasks within the LMS rated on a 5-point scale (1 = Never tried, 2 = Not at all, 3 = Not very easily, 4 = Somewhat easily, 5 = Very easily).Responses to the latter 12 items were summed to produce LMS Proficiency scores (Range = 12 -60), with scores ≥ 48 indicating ability to perform more tasks within the LMS with ease.The Cronbach's alpha for the 12 items in the LMS Proficiency subscale was .76.
The Perceptions of the Visual Design of Your LMS section assessed perceptions of the LMS' visual appearance (11 items) and the influence of colours on learning (5 items), rated on a 7-point scale (1 = Strongly disagree to 7 = Strongly agree); and the importance of LMS design dimensions to the learning experience (8 items), rated on a 5-point scale (1 = Not important at all, 2 = Of little importance, 3 = Of average importance, 4 = Very important, 5 = Absolutely essential).Items in each of these three areas were first reverse scored (if necessary) then summed to produce three scores: Perceptions of Visual Appearance (Range: 11 -77), Colours' Influence on Learning (Range: 5 -35); and Importance of Visual Design (Range: 8 -40).Higher scores on these measures suggest that visual appearance of the LMS is pleasing, colours have a greater influence on learning, and visual design dimensions are important to the learning experience.Cronbach's α for each of these three subscales was .88,.90, and .54,respectively.
The LMS Learning Experience section consisted of items about the satisfaction and importance of the LMS to learning and overall learning experiences (after Deng & Poole, 2010;Eristi et al., 2010;Palmer & Hold, 2010).Fourteen items asked students to rate on a 7-point scale (1 = Extremely dissatisfied to 7 = Extremely satisfied, or 0 = Not applicable) their level of satisfaction of the contribution of various LMS features (e.g., accessing course materials, viewing grades) to their learning.Responses to these items were summed to produce Contribution of LMS to Learning scores (Range: 0 -98), with higher scores reflecting greater satisfaction with more features of the LMS and the contribution of these features to learning.Thirteen items asked students to rate their agreement on items related to engagement with and flexibility of the LMS (i.e., "My institution's LMS is engaging," "I can decide where I want to learn when using my institution's LMS (e.g., at home, on campus)") on a 7-point scale (1 = Strongly disagree to 7 = Strongly Agree).Responses to these 13 items were summed to produce LMS Engagement and Flexibility scores (Range: 13 -91), with higher scores reflecting greater engagement with and perceptions of flexibility associated with the LMS.Cronbach's α for the LMS Learning Experience, Contribution of LMS to Learning, and LMS Engagement and Flexibility subscales were .84,.84,and .79,respectively.
Items in the General Information section collected participant age, gender, and cultural and educational background.Participants also indicated their expected grade point average (GPA) at the end of the current term [0.0 (0 -49%) to 4.5 (90 -100%)] and their level of satisfaction with their grades (1 = Extremely dissatisfied to 7 = Extremely satisfied).
A pilot survey was administered to 10 university students recruited through our participant registry.Pilot survey participants were invited to the laboratory to take part in a think aloud procedure as they completed the online survey.They received $10 gift cards for taking part in the pilot survey.Based on their verbalized responses and reasons for their responses, minor changes to the survey (e.g., rephrasing of questions or instructions) were made to improve clarity.

Data Analysis
SPSS Version 27 was used for data analysis.Descriptive statistics (frequency, median [Mdn], range) and non-parametric (Spearman Rho) correlations were computed.

General LMS Use and Expertise
Thirty-four (89.5%) respondents reported that D2L Brightspace was their institutions' LMS and four (10.5%) did not know.Twelve (32.4%) respondents indicated having used their institutions' LMS for one or two terms and 25 (67.6%) reported three or more terms of LMS use.As this study was completed prior to the COVID-19 pandemic, it is not surprising that most respondents (n = 23, 60.5%) reported not being enrolled in any strictly online courses during the regular academic year and most (n = 34, 91.9%) were enrolled in at least one face-to-face course.
Respondents reported logging on to their LMS at least 1 to 4 times per day (n = 7, 18.9%), 5 or 6 times per day (n = 7, 18.9%), and 7 or more times per day (n = 23, 62.2%).Respondents indicated that they could easily login to the LMS (n = 37, 97.3%), upload an assignment file to a submission folder (n = 30, 81.0%), and navigate from one course to another (n = 35, 94.6%).There were also respondents who had never tried sending messages to other students using the email or chat tools (48.6%), setting up automatic notification (32.4%), or using the calendar tool (35.1%) (Table 2 ).
As expected, LMS Proficiency scores (Mdn = 43, Range = 28 -60) were correlated with the number of terms of LMS use [r s (34) = .44,p = .007]and years enrolled in postsecondary education [r s (30) = .38,p = .03],suggesting that more opportunities to experience educational technology increase competence for its use.LMS Proficiency scores were not correlated with expected end of term GPA [r s (28) = .10,p = .60]or satisfaction with grades [r s (30) = -.10,p = .94].Note.Bolded entries indicate the three LMS activities that most respondents could complete easily.
When asked about the importance of the design dimensions to their learning experience, functionality, navigation, and page layout were viewed as very and extremely important (100%, 97.2%, and 88.9% of respondents, respectively), whereas most participants rated colour and animations as not at all important or slightly/moderately important (Table 4).

Learning Management System Learning Experience
Respondents were generally satisfied that accessing and interacting with course materials, completing quizzes/self-assessments, and submitting and receiving feedback on assignments contributed to their learning.Neutral or dissatisfied responses were common for items related to contacting peers via the LMS, the calendar, and the discussion tools.Fewer students were satisfied with the contribution of the LMS towards working collaboratively with peers (Table 5).Contribution of LMS to Learning scores (Mdn = 53, Range = 15 -90) were positively correlated with the number of terms of LMS use [r s (32) = .34,p = .05],LMS Proficiency [r s (33) = .47,p = .004],and Visual Appearance scores [r s (33) = .40,p = .02].Most respondents also agreed with the statements, "I feel that my institution's LMS enhances my learning experience;" "I prefer when my instructor integrates the LMS as a large component of the course;" and "I can decide where I want to learn when using my institution's LMS (e.g., at home, on campus)" (Table 6).LMS Engagement and Flexibility scores were positively correlated with Contribution of LMS to Learning scores [r s (30) = .45,p = .01]and Visual Appearance scores [r s (30) = .67,p < .001].LMS Engagement and Flexibility scores were not correlated with expected end of term GPA [r s (28) = -.24,p = .21]or satisfaction with grades [r s (30) = .16,p = .39].Together, these results suggest that students appreciate the flexibility and accessibility to learning materials that learning technologies provide, despite the finding that their grades and satisfaction with their grades were not linked to their engagement nor perceived flexibility to learn when the LMS is utilized.

Discussion
The goals of the present study were to understand how postsecondary students use the LMS at their educational institutions, determine what students think about the visual design of the LMS and its ease of use, and describe students' perceptions of the effect that colour has on their learning.To meet these goals, we developed and implemented an online survey for students enrolled in Canadian postsecondary institutions.We found several interesting results related to visual design, including colour, which have not been previously reported to our knowledge.Specifically, we found that most students felt that the design of their LMS was aesthetically pleasing.The perception that colours influenced their learning was negatively correlated with end of term GPA; the perception that the LMS contributed positively to their learning was associated with several factors: greater proficiency with the LMS; perceptions of flexibility and engagement in learning when using an LMS and the visual appearance of their LMS.We discuss these findings in detail.
When asked about the importance of the design dimensions to their learning experience, respondents in our study indicated that navigation and functionality were extremely important, whereas colour and typography were viewed as moderately or slightly important.Further, respondents indicated that colours did not enhance their learning experience or increase their ability to remember course content.Interestingly, students who perceived that colours enhanced learning reported lower end of term GPA.These results are inconsistent with previous research showing that colour in teaching and learning activities enhances memory thereby increasing learning (Finn & McLachlan, 2010), and students often prefer the colour versions of learning materials over black and white or monochrome versions (see Pelet & Papadopoulou, 2011;Pert & Wilson, 1996 for reviews).For students with various levels of visual impairment, high colour contrast may not only be preferred but may be necessary for facilitating viewing and reading of learning materials.Indeed, it has been strongly recommended that instructional designers incorporate colour in ways that will enhance readability (see Mancilla & Frey, 2021).
The presence of colour (especially when used poorly) to emphasize information in learning materials, however, may create barriers for students with colour vision deficiencies (Klooster, 2016) and such use should be avoided (see Mancilla & Frey, 2021).Additional studies using survey and experimental methodology that involve postsecondary students with visual impairments, particularly colour blindness, are needed to improve our understanding of how the colour and complexity of the LMS and other learning materials impede or enhance task completion, recall, and higher order learning (e.g., analysis, synthesis).In the 2006 Participation and Activity Limitation Survey (PALS), a national survey of Canadians with activity limiting conditions, 5.4% (or 24,280) of the 5-to 64-year-olds surveyed were students with a visual impairment (Statistics Canada, 2009).Of these students, 32.9% reported that it took them longer to complete their programs of study and 34.5% discontinued their education.With increased LMS and other online educational tools, it is important to consider how students with conditions related to vision are impacted by the design of educational technologies.
Postsecondary students in our study who expected lower end of term grades were more likely to indicate that colours enhanced their learning experience.Why this would be the case is not clear; however, the literature on elementary school children may provide some insight.Hannus and Hyönä (1999) examined the effects of illustrations on learning content from textbooks among 10-year-old children.Recall and comprehension of text was enhanced for children with higher intellectual ability but not for children with lower intellectual ability.Further examination using eye-tracking technology revealed that children with higher intellectual ability divided their attention between narrative passages and illustrations more strategically than did those with lower intellectual ability.More colour also contributes to the complexity of an image (Michailidou et al., 2008;Reinecke et al., 2013;Stoesz et al., 2020).Stoesz et al. (2020) observed that participants who perceived screenshots of LMS interfaces as more complex also scored lower on a test of working memory capacity.For postsecondary students viewing course material within an LMS, colour may capture attention giving the impression of its importance but may not enhance learning especially when processing is not strategic.With further investigation, utilizing eye-tracking technology and assessing various participant characteristics may help to elucidate the nature of the observed association between lower GPA and the perception that the colour scheme of the LMS enhances learning.
For respondents to our survey, the perception that the LMS design was visually pleasing was associated with greater LMS use and proficiency, greater satisfaction that the LMS features contributed to learning, increased engagement with the LMS, and perceptions that the LMS allowed more flexibility.Many students preferred that instructors use the LMS consistently to deliver content as this can enhance perceived flexibility in learning.These results are in the line with the work of Sanford et al. (2017), Song et al. (2004), andWu (2016).Song et al. (2004) found that students appreciated the convenience and flexibility associated with being enrolled in online courses, as they did not have to travel to campus to attend face-to-face classes.Wu (2016) showed that the perception of high quality design of course content (including its visual aesthetics and utility) in an online learning experience was associated with increased outcome behaviour (as measured by the content analysis of essay responses).Wu suggests that high quality design of online courses encourages extended engagement with course content and increases enjoyment and motivation.Moreover, good design of the online learning environment can help to meet students' basic psychological needs, thereby enhancing selfregulated motivation (Hsu et al., 2019), perceived and actual learning (Bolliger & Halupa, 2018), and engagement (Cole et al., 2019;Ghapanchi et al., 2020).Further research is needed to determine the degree of influence that the visual appearance of the LMS has on an online course and its effects on engagement, motivation, GPA and satisfaction with GPA, and other learning outcomes (Chen & Jang, 2010).Research into these topics must also consider the impact that the pandemic has had on students' expectations and perceptions about online learning and the visual design of LMS', given that many more students have been exposed to this course delivery mode than ever before.

Limitations and Future Research
Although our study provides some insights into the importance of the visual design of the LMS to student learning experiences, we acknowledge several limitations.First, our sample of participants was small, limiting the ability to generalize our findings.We suspect that students may have been overwhelmed with their online learning experiences and may have also experienced screen and survey fatigue during the pandemic.As such, we consider this research exploratory and suggest that further research involving larger populations of students from several postsecondary institutions using a variety of LMS is warranted to confirm and build on our results.Second, we did not examine the validity of the Use and Perceptions of Learning Management Systems Survey that we developed in this study due to our small sample.In future research it would be interesting to examine the factor structure of our survey, particularly those sections designed to capture information related to the learning experiences of students, and to gather evidence for convergent and divergent validity with our measure.Third, as mentioned above, we did not ask postsecondary students about accessibility issues that may have impacted the results of this study.This is a critical area of future research considering that the educational outcomes of a considerable proportion of students with visual impairments in Canada are negatively impacted (Statistics Canada, 2009).Finally, students in the current study were not asked about the degree to which their instructors and other students interacted with them through the LMS.A community of learning is viewed as very important to students enrolled in online courses, and the lack of interpersonal interactions can lead to less satisfaction with online learning (Song et al., 2004).

Conclusion
We examined students' perceptions of the visual design of the LMS and learning experiences using survey methodology.We explored the relationships between educational experience, familiarity and satisfaction with an LMS, visual design and colour, and satisfaction with the learning experience.Our results suggested that exploring the impact of colour and various other visual design aspects of LMSs on student engagement and learning are important and have practical value for LMS developers, instructional designers, and instructors.With additional investigations involving a larger participant sample, we can examine the factor structure of our survey, consider students' accessibility issues (e.g., visual impairment) when designing studies.The findings contribute to knowledge of the design of online teaching and learning environments considering students' overall learning experiences in Canadian postsecondary institutions.

Table 1
Participant Characteristics

Table 2
Frequencies of Responses to ItemsAssessing LMS Expertise (n = 37)

Table 3
Frequencies of Responses to Items Assessing Perceptions of the Visual Design of the LMS (n = 36)Note.The responses Strongly disagree, Disagree, and Somewhat disagree were collapsed into the category Disagree.The responses Strongly agree, Agree, and Somewhat agree were collapsed into the category Agree.Bolded entries indicate the three most common responses.*Items were reverse coded when creating Visual Appearance scores.

Table 4
Frequencies of Responses to Items Assessing Perceived Importance of LMS Design Dimensions to Learning Experience (n = 36)Note.Bolded entries indicate the three most common responses of Very and Extremely important.

Table 6
Frequencies of Responses to Items Assessing Engagement with and Flexibility of the LMS

can decide where I want to learn when using my institution's LMS (e.g., at home, on campus). 4 14.3 24 85.7 28
The responses Strongly disagree, Disagree, and Somewhat disagree were collapsed into one category Disagree.The responses Strongly agree, Agree, and Somewhat agree were collapsed into one category Agree.No responses were recorded for "Neutral".*Items were reverse coded when creating LMS Engagement and Flexibility scores.Bolded entries indicate the three statements where the most common responses were "Agree."