Pedagogical Design: Bridging Learning Theory and Learning Analytics Conception pédagogique : Rapprocher la théorie de l'apprentissage et l'analyse de l'apprentissage

Which learning analytics (LA) approach might be the best choice for your teaching and learning context? Learning analytics as a field of research and application seeks to collect, analyze, report, and interpret educational data with the goal of improving teaching and learning. However, hasty adoption of learning analytics tools and methods that are simply convenient, promoted, or available risks allowing learning analytics to “drive the pedagogical bus.” In this paper, we propose that careful reflection on pedagogical design choices and the learning theory that underpins them can and should inform selection of relevant learning analytics tools and approaches. We broadly review established learning theories and the implications of each for pedagogical design; for each design approach, we offer examples of learning analytics most clearly aligned with the theoretical perspectives on learning and knowledge that have shaped it. Moreover, we argue that careful consideration of the learning theory underpinning the pragmatics of pedagogical design choices should guide LA implementation, and help educators and designers avoid the risk of gathering data on, and measuring outcomes for, activities that are not relevant to their pedagogical design or goals.


Introduction
Internet technologies and digital tools have allowed innovative educators to experiment with new approaches to facilitating learning and empowering learners. Investigators report, for instance, that digital technologies can facilitate learner engagement (Chen et al., 2010), empower learning (Prasertsilp, 2013) or assist students with disabilities (Perelmutter et al., 2017;Stauter et al., 2019). Technology-enhanced learning is now a core organizing framework in education, allowing learners with an Internet connection to learn anytime and from anywhere. Moreover, in online learning environments, every action generates data which can be collected, stored, and analyzed in pursuit of new understandings of learner behaviours and learning (Clow, 2013). As investigators began to realize the value of these critical insights for teaching and learning, the new field of learning analytics (LA) emerged: the collection, measurement, analysis, and reporting of data about learners and their learning contexts, with the goal of understanding and optimizing learning (Long & Siemens, 2011). As an interdisciplinary field of study, LA has borrowed concepts, methods, and ideas from statistics, machine learning, business intelligence, educational psychology, learning sciences, and computer science (Banihashem et al., 2018). As with many new fields of study, however, a gap remains between theory and practice (Stewart, 2017), or, one might argue, between research findings reported in the LA literature, and meaningful educational implementations.
In this paper, we seek to alert educators and designers to the risk of LA-related technological determinism, which can occur when LA tools and methods are implemented without consideration of theoretical foundations, underlying perspectives on learning or achievement, or the particular learning goals of a given pedagogical design. To do so, we explore the theoretical foundations of different approaches to and uses of learning analytics, with a particular focus on the concept of learning, as envisioned by different learning theories and manifested in different approaches to pedagogical design.

Learning Analytics and Learning Theory
Learning analytics is focussed on learning (Gašević et al., 2015). It can provide insights into learners and learning processes, and help instructors make design and teaching decisions based on evidence (Banihashem et al., 2018). Different fields of study conceive of learning differently, and although scholars from diverse disciplines, such as cognitive science, computer science, educational psychology, applied linguistics, and anthropology, have contributed to our understanding of learning (Sawyer, 2014), it remains a complex and contested concept (Barron et al., 2015). Nonetheless, educators, and even institutional administrators, need a clear understanding of what they mean by learning, if they hope to meaningfully specify learning outcomes or assess learner achievement. Similarly, designers and others involved in selecting and implementing learning analytics options need a clear understanding of learning (and what constitutes achievement), so that they can make informed decisions about what kind of data they need to collect and analyze to be able to predict or monitor progress towards desired learning goals.
In the field of education, learning theories seek to explain how and why learning is happening -how learners absorb, process, and retain knowledge during learning --and illuminate perceptions of learning as a process or product (Bell, 2011;Smith, 1999Smith, -2020. Banihashem et al. 2019 collected and analyzed LA expert perspectives on the relationship between learning theories and learning analytics, and reported their belief that consideration of learning theories can support effective implementation and use of learning analytics in three important ways: • Learning theory can underpin design and selection of appropriate learning analytics tools or approaches.
• Learning theory can guide learning analytics use, by clarifying assumptions about what learning mean, how individuals learn, how learners process information, and which factors are important in the teaching and learning process.
• Learning theory can assist with sense-making of data, because it can explain learning, behaviour, experiences, and outcomes.
In other words, consideration of learning theories can help educators plan how and where to make best use of learning analytics in educational contexts, decide which types of data need to be collected, and make appropriate decisions about how data should be represented and reported. Importantly, learning theory can help us to translate data into knowledge, guiding our interpretations of data that may point to effective interventions. Clarifying our conception of learning in each context demands that we clearly understand the learning theory that underpins teaching and learning in that context.

Design for Learning: Connecting Learning Analytics with Learning Theory
Learning theories play a foundational role in different approaches to designing for teaching and learning. Unsurprisingly, different learning theories dictate different approaches. Each offers its own prescription for a pedagogical frameworks that can shape how teaching and learning happen in a given context, specifying resources, teaching guidelines, and learning values that are designed to provide productive teaching and facilitate learning (Halttunen, 2011;Romiszowski, 2016). Below, we review the best-known learning theories and the approaches to design for learning that have emerged from each perspective. For each, we then offer examples of LA applications or approaches that may meaningfully offer insight into learning as understood within that design framework.

Instructional Design, Objectivist Learning Theories, and Behaviourist Learning Analytics
The field of instructional design (also called instructional systems design) has largely been shaped by behaviourist and cognitivist perspectives on learning. While behaviourism and cognitivism differ in their understanding of how learners perceive the world, they share an objectivist philosophical assumption. Both take the position that the world is objectively real, and not simply created in the mind, interpretatively, by each learner (Ertmer & Newby, 2013).
The perspective that the best approach for understanding the workings of the human mind is objective and scientific measurement of behaviour (rather than the "unscientific" study of consciousness) was first formalized by Watson (1913), and such scientific approaches to psychology developed throughout the first half of the 20 th century. From a behaviourist perspective, mind, thoughts, and consciousness are considered to be unobservable phenomena (Akdeniz, 2016;Siemens, 2005). Instead, the focus of behaviourist study is observable behaviour, and the role of environmental factors that influence behaviour (Watson, 1913). For educationalists, behaviourism positions learning as relatively permanent changes in behaviour (Mazur, 2016), and knowledge as something acquired from external sources (Boghossian, 2013;Mazur, 2016). Human beings are imagined to be "blank slates" at birth, and their behaviour subsequently shaped most significantly by their environment, through predictable stimulus/response processes (rather than through mental activities like memory, perception, or thought).
Pedagogical designs founded on behaviourist principles are typically prescriptive, systematic, linear, and focussed on objective goals. Learners are viewed primarily as recipients of knowledge transmitted to them by educators, driving an emphasis on teaching methods (rather than on learning strategies and processes). A perceived benefit of behaviourist design has been ease of assessment, because of its focus on measurable and observable results, which has historically resulted in a heavy focus on quantitative and summative assessment of measurable outcomes.
Later in the 20 th century, cognitivism gained attention, offering a perspective on learning in which internal processes of the mind play a more important role (Brown & Green, 2019). Although cognitivist learning theory also acknowledges reinforcement and environmental factors as motivators (Mergel, 1998), it is less mechanistic in its understanding of the human response to stimuli (Yilmaz, 2011). Instead, cognitive theories highlight the importance of mental processes (perception, thought, memory, attention, problem solving, and information processing) in learning, and seek to explain how information processing (receiving, organizing, storing, and retrieving of information) is happening in the mind (Currie, 2004;Ertmer & Newby, 2013).
Pedagogical designs founded on cognitivist principles typically consider learners be more active participants in the acquisition of knowledge, with learning understood as an active process of receiving, organizing, storing, and retrieving information and knowledge. Educators are less central, and play a guiding role through design and provision of instructional supports such as scaffolding, examples, feedback, and advance organizers for new topics (Mohammadi et al., 2010), and by facilitating recall of prior knowledge (Mergel, 1998) and design of authentic situations for learning.

Behaviourist Learning Analytics (BLA)
At present, the vast majority of learning analytics solutions promoted by educational software vendors, and also the learning analytics features integrated into most learning management systems and other eLearning platforms, are not primarily focussed on how learning is achieved. Instead, they tend to prioritize the collection, analysis, and reporting of data about progression through learning activities, as well as measurement of outcomes of learning (usually grades). Because this heavy focus on measuring the products of learning, and on monitoring progress through learning activities, reflect the core foci of behaviourist/cognitivist approaches to pedagogical design, we refer to this cluster of LA approaches as BLA.
A strength of BLA is its emphasis on observable and quantitatively measurable behaviours and outcomes, making data capture easier. Behaviourist learning analytics tend to collect, analyze and report quantitative metrics (Knight et al., 2013) such as the number of logins and logouts, grades, assignment scores, number of attempts at quizzes, time spent in discussion forums, or number of discussion messages posted by learners. The typically linear and systematic structure of pedagogical designs underpinned by behaviourist (and cognitivist) approaches to instructional design align well with the systematic, pre-set and linear technical processes of many existing learning analytics applications.
Like behaviourist approaches to pedagogical design, BLA applications can also be criticized for being heavily teacher-centric: they typically offer data and insights only to educators, and not to learners themselves. Moreover, and despite the cognitivist interest in mental processes, BLA applications and processes typically give no attention to data that might offer insights into more complex data about learners and learning processes such as cognition, meta-cognition, or higher levels of thinking (e.g., critical and creative thinking). Figure 1 offers an example of a classic BLA-oriented dashboard for educators. The developers at the UK Open University describe the goals of their OUAnalyse project as "early identification of students at risk of failing" and "to significantly improve the retention of OU students." They explain that metrics and predictions of failure risk are "available weekly to the course tutors and the Student Support Teams." 1

Figure 1
An Example of BLA: OUAnalyse: An Educator LA Dashboard from the UK Open University It is evident that such an LA solution overwhelmingly collects and displays metrics on progression through a sequence of prepared learning activities and grade achievement in assessments as the primary indicators of learning or "success." Moreover, analytics of this kind offer little insight to educators or instructional designers that could inform effective redesign of a learning environment to optimize learning outcomes. While it is conceivable that future work in educational psychology may offer deeper understanding of the learning processes in behaviourist learning contexts that may direct future selection and analysis of relevant data on learning, we are not aware of learning analytics of this kind at the present time.

Learning Design, Subjectivism, and Constructivist Learning Analytics
By contrast, the field of learning design, also called learning environment design (LED), has been profoundly shaped by constructivist perspectives on learning (Mor et al., 2015). By the late 20 th century, some theorists had begun to question objectivist theories of learning, and instead advanced perspectives, drawing on earlier work by theorists such as Piaget and Vygotsky (Weegar & Pacis, 2012), that seek to explain how learners construct knowledge (Jonassen, 1991).
Constructivism argues that what people know about the world depends primarily on their own interpretation of their experiences (Ertmer & Newby, 2013). Learning is therefore understood as a process of personal construction of knowledge from their experiences and building on prior knowledge (Altun & Büyükduman, 2007;Mergel, 1998;Merrill, 1991;Parker, 2009;Weegar & Pacis, 2012), rather than acquisition of knowledge transmitted from educators to learners. Social constructivist theories go even further, asserting that collaboration, social interaction, thought sharing, and meaning negotiation offer significant routes to conceptual development and understanding (Altun & Büyükduman, 2007;Parker, 2009). It is important to note that while cognitivism and constructivism both centre the learner, in the cognitivist view, the learner is a "processor of information," while in the constructivist view the learner not only processes information, but is also responsible for interpreting this information and building personal knowledge (Ertmer & Newby, 2013). Constructivism is an interpretivist paradigm with a relativist ontology, and a subjectivist epistemology (Levers, 2013). It holds the subjective experiences of the learner to be determining of their individual understanding of reality.
Pedagogical designs rooted in constructivist principles are therefore learner-centred (Gagnon & Collay, 2005) and consider learners to be responsible for their learning, with educators responsible for designing and facilitating authentic, challenging and problem-based learning activities and environments to effectively achieve learning outcomes (Conole, 2012;Gagnon & Collay, 2005;Mor & Craft, 2012). Typically, these evolve as a sequence of learning activities designed to engage learners in interactive and collaborative learning environments (see for example Bybee et al., 2006;Heinich et al., 1999;Jonassen, 1999;Merrill, 2002;Papadakis, 2012). There is an emphasis on subjectivity, and on how something is learned rather than what is learned, with instructional goals and objectives "negotiated rather than set, with no one best way of sequencing instruction" (Cooper, 1993, p. 17). Qualitative approaches to both formative and summative evaluation and assessment are promoted (although assessment remains a challenge in constructivist learning environments).

Constructivist Learning Analytics (CsLA)
While instructional design and learning design often embrace similar design structures and a common goal (helping learners achieve the desired learning outcomes), they follow different paths. Learning design focuses mainly on what learners do (Mor et al., 2015;Seel et al., 2017) This perspective is motivated by questions such as: What will be learned? How will it be learned? How will learning be assessed? (Ifenthaler et al., 2018). Berland et al. (2014) propose that learning analytics can help constructivist educators provide a rich learning environment for learners. With the learner at the heart of the pedagogical process (Gagnon & Collay, 2005), and the learning process given greater importance than learning outcomes, we have classified as constructivist learning analytics (ConLA) those LA tools and approaches that focus on capturing, analyzing, and reporting data that can provide insights into learning process (Knight et al., 2014). Data that answer questions about how learning is progressing, and the degree to which learners are engaging in desired learning behaviours, are more important than quantitative and summative performance metrics (Altun & Büyükduman, 2007;Merrill, 1991). Moreover, since formative assessment is a feature of constructivist learning design (Jonassen, 1991), CsLA trace data about learning activities rather than teaching activities, and typically give feedback to learners as well as educators about how they are learning. Since constructivist approaches to education focus (in principle, at least) on learning context, social communications, meta-cognition and higher levels of thinking, constructivist learning analytics concentrate on collecting, analyzing, and reporting more complex and qualitative data.
Social network analysis (SNA) offers one of the better-known approaches to analyzing and visualizing learning processes documented in the LA literature (see for example Dawson, 2009). Network metrics (centrality measures) generated by SNA offer deeper insights into evolving learning networks, by illuminating, for example, which learners are the most active, which learners are the most connected, and which learners may be brokering information flow between sub-groups (Buckingham et al., 2011). Arguably, SNA can illuminate the degree to which a learning design is facilitating the desired social constructivist learner activity. Increasingly, developers are building plug-ins and tools that allow easy network analysis of online learning network communications (Figure 2), and which can reveal network participation to learners as well as educators.

Learning Network Visualization of Learner Communications in an Online Course Discussion, Visualized with the Threadz 2 Plugin for the Canvas LMS
In a different area of work, a range of text mining and natural language processing methods are being employed to assess relevance, quality or evolving complexity, and sophistication or rhetorical structure of learner work compiled in ePortfolios or submitted over time (McNamara et al., 2017). Figure 3, for example, offers a screenshot of the AcaWriter tool 3 , recently developed and piloted at the University of Technology, Sydney. Explicitly developed as learner-facing tool, AcaWriter natural language processing (NLP) software identifies features of a learner's writing such as key concepts, people, and places, and the degree to which the learner is learning to make scholarly knowledge claims.
It offers learners automated feedback to help them develop their academic and reflective writing (Shibani et al., 2019).
The sometimes nonlinear and often qualitative and interpretive nature of constructivist learning designs are often seen as a challenge in relation to data collection. A recent review gives grounds for optimism, however, reporting that in recent years LA research has begun to move away from predictive modelling, and towards a deeper understanding of learning experiences (Viberg et al., 2018). Siemens (2005) has argued that traditional learning theories, such as those described above, fail to meet the learning needs of the digital age. Such theories, he contends, are limited in their capacity to accommodate interpersonal perspectives on learning, are unable to address the learning that is located within and supported by technology and organizations, and give no insight into the processes of value judgement that are needed for learning in knowledge-rich environments (Bell, 2009;Siemens, 2005). Instead, Siemens has elaborated a connectivist learning theory, epistemologically grounded in the notion of connective knowledge (also called distributed knowledge) (Downes, 2008), which he argues is more relevant to learning needs in the current era -one in which technology shapes our lives, an explosion of knowledge is occurring, and rapid evaluation of knowledge is important. Connectivism offers an approach to accommodating in our design and pedagogy the new understandings of chaos, complexity, and networks that have been driven by the technology explosion (Dunaway, 2011;Siemens, 2005).

Learning Ecosystem Design, Connectivism, and Connectivist Learning Analytics
As outlined by Siemens (2005), connectivism asserts that: • Learning and knowledge rest in diversity of opinions.
• Learning is a process of connecting specialized nodes (also called entities) or information sources into networks.
• Learning may reside in non-human appliances.
• Capacity to know more is more critical than what is currently known.
• Nurturing and maintenance of connections is needed to facilitate continual learning.
• Ability to see connections between fields, ideas, and concepts is a core skill.
• Currency (accurate, up-to-date knowledge) is the intent of all connectivist learning activities.
• Decision-making is itself a learning process. (p. 7) In contrast with theories that view knowledge as constructed internally by individuals, or collaboratively through social negotiation, connectivist theory posits that knowledge is "distributed across a network of connections, and therefore that learning consists of the ability to continuously construct and traverse those networks" (Downes, 2008;Duke et al., 2013). Learning is understood to take place when learners make connections with other nodes (individuals, ideas, organizations, websites, journals, computers, etc.) (Dunaway, 2011;Siemens, 2005). Rapid evaluation of knowledge, and more importantly, preservation of the capacity to learn more, is therefore more important than what is learned now.
Connectivism holds some specific educational implications for educators and learners. Learning is viewed as a process of creating and navigating networks, with pedagogical design focused on facilitating interaction, information flow, and networked learning. Gathering and distributing diverse and current knowledge (Kizito, 2016;Siemens, 2005;Siemens & Tittenberger, 2009) from networks are understood to be intrinsic features of learning, and the learning environment is viewed as an ecosystem -virtually a living organism.
Design for learning and knowledge in connectivist-inspired learning contexts remains learnercentred but requires facilitating the creation of current and dynamic knowledge-sharing networks. Learners are viewed as nodes in an ecosystem of interconnected networks (Banihashem & Aliabadi, 2017) and they make use of the ecosystem to build their own personal connections and networks (Richardson, 2002). Such learning ecosystems are envisioned as rich, dynamic, and fluid environments that facilitate complex interactions, connections, and flow of information and knowledge, and have the capacity to grow, self-organize and constantly evolve (Pradhan, 2016, February 19;Richardson, 2002). They are open, dynamic, independent, and adaptive systems that include both interpersonal interactions and interactions with non-human components such as technologies and organizations (Brown, 2000;Gütl & Chang, 2008;Saadatmand, 2017).
While the concept of a learning ecosystem aligns well with our understanding of the chaotic, rapidly growing and connected nature of knowledge in the current era, it is nonetheless challenging to design a learning environment which is pedagogically well-matched with evolving and complex nature of connectivism (Bell, 2011;Kop & Hill, 2008). Some have suggested that the LEDs implied by connectivist learning theory are thus far poorly understood (see for example Chatti, 2010).

Connectivist Learning Analytics (ConLA)
In principle, connectivist learning analytics should be heavily geared towards analyzing how well a learner is creating networks, how the learner is communicating with other learners, and how effectively information and knowledge are flowing in a learning network. Siemens (2008) has argued that connectivist learning analytics may be best suited for understanding complex learning processes in contexts of rapid change and diverse knowledge sources, and should focus on collecting data about nodes, networks, and their inter-relationships.
Methodologically, social network analysis offers some analytic potential for connectivist learning ecosystems, but a challenge is the risk of over-valuing (or over-interpreting) simple network membership or participation, and of overlooking the complexity of fine-grained network activity (Knight et al., 2014). To provide meaningful connectivist learning analytics, analysis must go beyond straightforward reporting of "who is connected with whom" and must also seek to integrate data on "who is connected with what" (such as ideas, devices, publications, or organizations). Analysis of information flow, diversity, dynamism, and knowledge currency all seem pertinent in a connectivist learning ecosystem, as well as interaction within learning networks, though examples are few. Rosé et al. (2015) and Crosslin et al. (2018) describe an experimental effort to design an explicitly connectivist multi-level MOOC, which integrated multiple and optional innovative forms of support for discussion-based learning, social learning, and self-regulated learning. These authors outline their technical efforts to integrate learner contributions from multiple channels, but to date, and perhaps reflecting the significant pedagogical challenge of connectivist learning environment design (Bell, 2011;Kop & Hill, 2008), no vision for meaningful ConLA to support learners or educators in that context has emerged.
One potential analytic approach that may valuably contribute to connectivist LA is epistemic network analysis (ENA). As Shaffer et al. (2016) explain, ENA "is a set of techniques that identifies and measures connections among elements in coded data and represents them in dynamic network models" (pp. 9-10). They highlight that ENA allows modelling of network change over timeincluding changes in composition and strength of connections -and allows comparison of networks, "so that it can be used to explore a wide range of qualitative and quantitative research questions in situations where patterns of association in data are hypothesized to be meaningful" (p. 10). Shaffer and Ruis (2017) offer a worked example of ENA applied to educational data from a learning environment to illustrate the potential of this approach. Development of meaningful connectivist learning analytics associated with effective connectivist learning ecosystems nevertheless remains an open challenge.

Where Learning Theory and Learning Analytics Meet
Above, we have sought to illuminate the connections between underlying learning theory (beliefs about what constitutes learning), choice of design approach for learning, and the LA tools or methods that are most likely to offer insights into the desired learning. Learning analytics are, first and foremost, about learning (Gašević et al., 2015). If learning analytics seeks to understand and optimize learning in a selected learning environment, then it is vital to acknowledge how learning is believed to take place in that environment, how we believe our design strategy is supporting learning (Wong et al., 2019), and what kinds of data will meaningfully offer insight into the learning of individuals.
Pragmatically, pedagogical design (shaped by learning theory and underlying epistemology) offers guidance as to how learning analytics might best be used to support that design and the learning goals it specifies. If we have espoused a constructivist learning design underpinned by a commitment to social negotiation of meaning, for example, then a behaviourist LA application designed to monitor task completion will be unlikely to offer meaningful understanding of how our learning design is influencing learning or learner achievement. Effective and successful application of learning analytics in educational contexts demands that we give attention to the theoretical foundations of the learning contexts we design (Gašević et al., 2016;Gašević et al., 2015;Knight et al., 2013;Koh et al., 2016;Stewart, 2017;Wise, 2014).
In Figure 4, we build on earlier work by Knight et al. (2014) and summarize schematically the three categories of learning analytics proposed in this paper, separated conceptually at the level of relevance to learning theory. While these categories may each make use of some of the same underlying data (for example, learner demographic data, performance data, or learning process data), each is employed for very different pedagogical and interpretive purposes, supporting the assertion by Gašević et al. (2016) and others that "one size" of learning analytics definitively cannot meet the needs of all pedagogical design frameworks. The role of theory in learning analytics application, then, is that it gives different meaning to data (Wong et al., 2019).
Furthermore, failure to consider theoretical foundations risks allowing learning analytics to drive the pedagogical bus, generating data on and measuring outcomes for activities that are not relevant in the pedagogical design. Technological and mechanical determinism, resulting from the poorly thought-out application of convenient, promoted or available learning analytics tools, is a real danger (Knight et al., 2014;Siemens et al., 2013). We cannot and must not design learning environments and learning outcomes to fit available learning analytics tools.

Conceptual Framework of Different Learning Analytics Approaches
Wise predicted in 2014 that LA would not meaningfully influence teaching and learning until it was designed into the larger pattern of instruction, but to date most LA literature continues to address researchers, and not pedagogical designers, practitioners or educators (Ferguson & Clow, 2017;Viberg et al., 2018). These same authors highlight the slow pace of meaningful LA implementation in educational contexts, and the resultant lack of evidence of impact of LA in authentic settings. The pragmatic next step calls for embedding and testing relevant learning analytics approaches in different and appropriate pedagogical designs. It is our hope that ongoing theoretical and empirical work investigating the interconnections between learning analytics, learning theory, and learning design will engage educators and pedagogical designers as collaborators in this work, and assist with selection of the best possible learning analytic approaches to support and illuminate learning outcomes of their pedagogical projects.

Author Note
Correspondence concerning this article should be addressed to Leah P. Macfadyen, Associate Director, Master of Educational Technology Program, UBC Faculty of Education, 305D-2125 Main Mall, Vancouver, BC, V6T 1Z4, Canada. Email: leah.macfadyen@ubc.ca