An Extended Systematic Review of Canadian Policy Documents on e-Learning : What We ’ re Doing and Not Doing Étude systématique de longue durée des documents d ’ orientation

This systematic review builds upon the work of Abrami et al. (2006) and McGreal and Anderson (2007). It seeks to provide a synthesis and discussion of publicly available government policy documents with regard to e-learning in Canada. In total, 138 policy documents from Canadian provinces and territories and several federal agencies, dated from 2000 to 2010, were identified, retrieved and analyzed using prescriptive and emergent coding approaches with the purpose of uncovering and describing areas of commonality and inconsistency and, hence, determining where discussions about e-learning are lacking. The review confirmed that Canadian policy makers view technology as offering various potential benefits to learners, but also revealed a troubling lack of specific details, consistency and coordination in facilitating the development of e-learning to fulfill these optimistic expectations. CJLT/RCAT Vol. 37(3) Systematic Review of e-Learning Policies in Canada 2 Résumé Cette étude systématique repose sur les travaux Abrami et al. (2006) et de McGreal et Anderson (2007). Elle vise à fournir une synthèse et une analyse des documents publics d’orientation gouvernementale qui traitent de l’apprentissage en ligne au Canada. Au total, 138 documents d’orientation publiés, de 2000 à 2010, par les provinces et les territoires canadiens ainsi que par plusieurs agences fédérales ont été identifiés, récupérés et analysés selon des méthodes de codage nouvelles et consacrées par l’usage, dans le but de découvrir et de décrire des zones de points communs et d’hétérogénéité, et, du coup, de déterminer des déficiences dans les analyses de l’apprentissage en ligne. L’étude a confirmé que, pour les responsables politiques canadiens, l’informatique offre des avantages potentiels aux apprenants; toutefois, l’étude a aussi révélé un manque préoccupant d’explication, de constance et de coordination quand il a fallu faciliter l’implantation de l’apprentissage en ligne afin de répondre à ces attentes optimistes. Introduction There is widespread agreement on the importance of technology in education, and much of the research that has been carried out over the past few decades supports this notion. For better or worse, the term e-learning has come to broadly signify various uses of learning technologies that in earlier times might have been referred to separately as CAI, CAL, CSCL, CBI, CMC, etc. Even distance education (DE), online learning (OL) and blended learning (BL) are now considered to be subsets of e-learning. For the purposes of this article, e-learning refers to the range of electronic and digital technologies used for teaching and learning. It encompasses all levels of formal education, including classroom uses, distance and online learning and blended learning (i.e., partly online and partly classroom-based). Given the variety of terminology used for e-learning, this definition includes other related terms (e.g., ICT, technology integration in education, Web-based distributed learning, electronic/digital learning tools/objects, and technologybased learning). Information and communication technologies (ICTs) bring advantages to learning processes that are not readily or easily available through other means, the most important being greater access to learning, better allocation of teaching resources, shared learning content, deeper learning and the social components of learning. In particular, e-learning can improve the flexibility and quality of learning by providing access to a high-quality, evidence-based range of multimedia resources and interactive courseware; enabling students to control the timing, location and pace of their studies; supporting educators in providing high-quality instruction; tailoring the learning environment to the learning needs of individual students; supporting increased communications among educators and learners; providing frequent and timely individual feedback and assessment; and supporting reuse of high-quality learning resources (Canadian Council on Learning, 2009). Because of these characteristics, e-learning is increasingly recognized as a fundamental tool for fostering a lifelong learning society. Member countries of the Organisation for Economic Cooperation and Development (OECD) have long recognized the importance of CJLT/RCAT Vol. 37(3) Systematic Review of e-Learning Policies in Canada 3 lifelong learning for social and economic development. Lifelong learning provides benefits such as better health, more job opportunities and a higher quality of life (CCL, 2009). Studies of technology integration in classrooms and technology-supported distance education have demonstrated the importance and benefits of technology as a way to equip learners for the future. As a second-order meta-analysis of the effects of computer technology on student achievement outcomes by Tamim, Bernard, Borokhovski, Abrami, & Schmid (2011) noted, there is a small to moderate average effect size (ES ≈ +0.30) in favour of technology versus no technology integration in classrooms (over the 40 years that it has been examined empirically). Similarly, Schmid, Bernard, Borokhovski, Tamim, Abrami, et al. (2009) found an average effect of approximately +0.28 when classrooms with a high level of technology use were compared to those with lesser use of technology. These findings were also echoed in a recent meta-analysis of all forms of distance education (DE) versus classroom instruction (CI). The study found an average effect size of around 0.05 (Bernard, Abrami, Lou, Borokhovski, Wade, et al., 2004) for asynchronous DE (i.e. DE and CI were about equal), and meta-analyses of online learning versus regular classroom instruction have found effect sizes ranging from +0.12 to +0.15 (see Cook et al., 2008; Sitzmann, Kraiger, Stewart, & Wisher, 2006; U.S. Department of Education, 2009). Studies of blended classrooms (DOE, 2009) have achieved even better results (ES = 0.24), i.e., technology-enabled elements of distance education can be at least equal to classroom instruction and in some cases offer stronger learning outcomes. With all of this encouraging news from the research front, one would think that provincial and federal Canadian policy makers would be falling all over each other to design and develop policies, strategies, plans and actions to capture the moment by putting e-learning upfront in Canadian education. But are they doing this? Many OECD countries, as well as the European Union, have recognized the importance and potential of e-learning and are implementing aggressive national/supranational e-learning strategies to support their policies and programs. Indeed, e-learning strategies and action plans in most countries are government-initiated—through ministries/departments, public funding councils or multi-ministerial committees—and translate into initiatives with significant public funding. To determine whether these positive results yielded action by Canadian policy makers, we conducted a systematic review of policy documents concerning e-learning. This project is an extension of two relatively recent publications (Abrami, Bernard, Wade, Schmid, Borokhovski, et al., 2006; McGreal & Anderson, 2007) that describe e-learning in Canada in two distinct ways. One approach is empirical and the other is non-empirical, and they both make useful contributions to the ongoing dialogue among Canadians on how to increase adoption of e-learning practices. The current work attempts to further our understanding by providing a more comprehensive and systematic review of e-learning policy documents which goes beyond Abrami et al. (2006) and McGreal and Anderson. This article is not directly comparable to these works, as differences in intent and method make comparison impossible; however, taken together they offer a more robust description and analysis of e-learning-related policies in Canada. CJLT/RCAT Vol. 37(3) Systematic Review of e-Learning Policies in Canada 4 Abrami et al. (2006) Abrami et al. (2006) study used a systematic review methodology to construct an “argument catalogue,” the results of which were touted as constituting “a rough sketch of the evidence, gaps and promising directions in e-learning from 2000 to 2005, with a particular focus on Canada” (p. 1). The research group from the Centre for the Study of Learning and Performance (CSLP) at Concordia University searched for documents in five domains: 1) general public opinion; 2) trade/practitioners’ opinions; 3) national and provincial policy documents; 4) reviews of research; and 5) primary empirical evidence. In total they located 2,042 documents, including 88 policy documents. We will focus only on the policy findings, since this part of Abrami et al. (2006) relates most closely to this article. Below is a quotation that summarizes the findings of the search for and analysis of Canadian policy documents: The overall perceptions of technology use in education by the policy-making community are favourable. The HORIZON Report (New Media Consortium, 2004) contains prognoses of what technologies will become important in higher education in the next one to five years. Several papers emphasize the role Canada should assume in implementing technology-based approaches to education (Industry Canada, Advisory Committee for Online Learning, 2001; Rossiter, 2002) and why it is both possible and necessary. Finally, Davis and Carlsen (2004) named four major reasons to invest in ICT in education: 1) ICT is an imperative for economic competitiveness; 2) ICT increases educational attainment; 3) ICT increases access to education; and 4) ICT is a catalyst promoting changes in education. One of the major messages from the policy documents is about the need to bridge the gap between theory, research and practice (Kinuthia, 2004; Mühlhäuser, 2004; Pittard, 2004). There is a concern, however, about how exactly it is done. According to Oliver and Conole (2003), the use of evidence-based prac


Introduction
There is widespread agreement on the importance of technology in education, and much of the research that has been carried out over the past few decades supports this notion.For better or worse, the term e-learning has come to broadly signify various uses of learning technologies that in earlier times might have been referred to separately as CAI, CAL, CSCL, CBI, CMC, etc.Even distance education (DE), online learning (OL) and blended learning (BL) are now considered to be subsets of e-learning.
For the purposes of this article, e-learning refers to the range of electronic and digital technologies used for teaching and learning.It encompasses all levels of formal education, including classroom uses, distance and online learning and blended learning (i.e., partly online and partly classroom-based).Given the variety of terminology used for e-learning, this definition includes other related terms (e.g., ICT, technology integration in education, Web-based distributed learning, electronic/digital learning tools/objects, and technologybased learning).
Information and communication technologies (ICTs) bring advantages to learning processes that are not readily or easily available through other means, the most important being greater access to learning, better allocation of teaching resources, shared learning content, deeper learning and the social components of learning.In particular, e-learning can improve the flexibility and quality of learning by providing access to a high-quality, evidence-based range of multimedia resources and interactive courseware; enabling students to control the timing, location and pace of their studies; supporting educators in providing high-quality instruction; tailoring the learning environment to the learning needs of individual students; supporting increased communications among educators and learners; providing frequent and timely individual feedback and assessment; and supporting reuse of high-quality learning resources (Canadian Council on Learning, 2009).
Because of these characteristics, e-learning is increasingly recognized as a fundamental tool for fostering a lifelong learning society.Member countries of the Organisation for Economic Cooperation and Development (OECD) have long recognized the importance of lifelong learning for social and economic development.Lifelong learning provides benefits such as better health, more job opportunities and a higher quality of life (CCL, 2009).
Studies of technology integration in classrooms and technology-supported distance education have demonstrated the importance and benefits of technology as a way to equip learners for the future.As a second-order meta-analysis of the effects of computer technology on student achievement outcomes by Tamim, Bernard, Borokhovski, Abrami, & Schmid (2011) noted, there is a small to moderate average effect size (ES ≈ +0.30) in favour of technology versus no technology integration in classrooms (over the 40 years that it has been examined empirically).Similarly, Schmid, Bernard, Borokhovski, Tamim, Abrami, et al. (2009) found an average effect of approximately +0.28 when classrooms with a high level of technology use were compared to those with lesser use of technology.These findings were also echoed in a recent meta-analysis of all forms of distance education (DE) versus classroom instruction (CI).The study found an average effect size of around 0.05 (Bernard, Abrami, Lou, Borokhovski, Wade, et al., 2004) for asynchronous DE (i.e.DE and CI were about equal), and meta-analyses of online learning versus regular classroom instruction have found effect sizes ranging from +0.12 to +0.15 (see Cook et al., 2008;Sitzmann, Kraiger, Stewart, & Wisher, 2006; U.S. Department of Education, 2009).Studies of blended classrooms (DOE, 2009) have achieved even better results (ES = 0.24), i.e., technology-enabled elements of distance education can be at least equal to classroom instruction and in some cases offer stronger learning outcomes.With all of this encouraging news from the research front, one would think that provincial and federal Canadian policy makers would be falling all over each other to design and develop policies, strategies, plans and actions to capture the moment by putting e-learning upfront in Canadian education.But are they doing this?Many OECD countries, as well as the European Union, have recognized the importance and potential of e-learning and are implementing aggressive national/supranational e-learning strategies to support their policies and programs.Indeed, e-learning strategies and action plans in most countries are government-initiated-through ministries/departments, public funding councils or multi-ministerial committees-and translate into initiatives with significant public funding.
To determine whether these positive results yielded action by Canadian policy makers, we conducted a systematic review of policy documents concerning e-learning.This project is an extension of two relatively recent publications (Abrami, Bernard, Wade, Schmid, Borokhovski, et al., 2006;McGreal & Anderson, 2007) that describe e-learning in Canada in two distinct ways.One approach is empirical and the other is non-empirical, and they both make useful contributions to the ongoing dialogue among Canadians on how to increase adoption of e-learning practices.
The current work attempts to further our understanding by providing a more comprehensive and systematic review of e-learning policy documents which goes beyond Abrami et al. (2006) and McGreal and Anderson.This article is not directly comparable to these works, as differences in intent and method make comparison impossible; however, taken together they offer a more robust description and analysis of e-learning-related policies in Canada.Abrami et al. (2006) study used a systematic review methodology to construct an "argument catalogue," the results of which were touted as constituting "a rough sketch of the evidence, gaps and promising directions in e-learning from 2000 to 2005, with a particular focus on Canada" (p. 1).The research group from the Centre for the Study of Learning and Performance (CSLP) at Concordia University searched for documents in five domains: 1) general public opinion; 2) trade/practitioners' opinions; 3) national and provincial policy documents; 4) reviews of research; and 5) primary empirical evidence.In total they located 2,042 documents, including 88 policy documents.We will focus only on the policy findings, since this part of Abrami et al. (2006) relates most closely to this article.Below is a quotation that summarizes the findings of the search for and analysis of Canadian policy documents:

Abrami et al. (2006)
The overall perceptions of technology use in education by the policy-making community are favourable.The HORIZON Report (New Media Consortium, 2004) contains prognoses of what technologies will become important in higher education in the next one to five years.Several papers emphasize the role Canada should assume in implementing technology-based approaches to education (Industry Canada, Advisory Committee for Online Learning, 2001;Rossiter, 2002) and why it is both possible and necessary.Finally, Davis and Carlsen (2004) named four major reasons to invest in ICT in education: 1) ICT is an imperative for economic competitiveness; 2) ICT increases educational attainment; 3) ICT increases access to education; and 4) ICT is a catalyst promoting changes in education.
One of the major messages from the policy documents is about the need to bridge the gap between theory, research and practice (Kinuthia, 2004;Mühlhäuser, 2004;Pittard, 2004).There is a concern, however, about how exactly it is done.According to Oliver and Conole (2003), the use of evidence-based practice in policy making can be criticized on methodological, epistemological, and moral grounds: "While the aim of evidence-based practice-linking research, practice and policy-is valuable, its methods are questionable."Among the problems with this ideology they mentioned "inappropriate dismissal of qualitative approaches," and "disempowering practitioners" by forcing them into a tradition "that blends evaluation, research and practice together."(p.394)

McGreal & Anderson (2007)
McGreal and Anderson paper, "E-Learning in Canada," is a narrative description addressing essentially the same issues described empirically by Abrami et al. (2006).It provides an interesting, insightful view of the uniqueness of Canada's response to the rapid rise of e-learning.The summary effectively characterizes the message addressed throughout the article: It is evident that any view of e-learning in Canada must be informed by the uniquely Canadian feature of provincial jurisdiction over education.As was noted in the introduction, Canada is the only country that does not have a national department/ministry of education.Therefore, any investigation of e-learning in Canada must focus more on specific provincial initiatives in technologically enhanced learning rather than a Canadian overview.A distinctive "Canadian" model cannot exist (unless one views disparate models as evidence of a uniquely Canadian archetype).The provinciality of Canadian e-learning serves to highlight the inability of Canada to sustain national strategies and focus, such as those implemented in many other countries, due to the fractious nature of federal and provincial relations, particularly in education.(p. 5)

The Purpose of This Article
The current review aims to provide a synthesis of policy statements related to e-learning in Canada and builds on the work of Abrami et al. (2006) and McGreal and Anderson (2007).Using Canadian federal, provincial and territorial government policy documents as the primary data source, this research offers a description of areas of commonality, intersection and inconsistency and indicates where discussions of e-learning were omitted from government policy documents.While not an exhaustive discussion, our paper will take ideas from these two previous works and combine them with a systematic review of provincial and federal policy documents.It also delves more deeply than Abrami et al. (2006) into the policy literature.Our overall approach is to develop themes from the literature dated from 2000 through 2010.It is important to emphasize that this review does not interpret policies; rather we attempt to systematically reflect the degree of representation of different topics that were originally identified and categorized in Abrami et al. (2006), as they appeared in the reviewed documents.

Search Strategy and Document Selection
This study used a two-stage approach to data collection.The data were initially collected in 2008 as part of a study commissioned by the Canadian Council on Learning and expanded in 2010 by the same research team as part of an independent study that attempted to increase the scope of the research and monitor any progress being made.Both stages of data collection explored publicly available provincial and federal policy documents that addressed e-learning.Data from both collection periods were combined.
Education and training department Web sites and the Council of Ministers of Education, Canada (CMEC) Web site were the primary sources for documents representing 10 provinces and 3 territories.In addition, the review targeted six selected federal departments: Canadian Heritage, National Defence, Health Canada, Industry Canada, Human Resources and Skills Development Canada, and Indian and Northern Affairs Canada.
Each Web site was reviewed for relevant documents using a number of keyword searches.Given the variety of terminology used for e-learning, the search included other related terms (e.g., "ICT," "technology integration in education," "Web-based distributed learning," "electronic/digital learning tools/objects" and "technology-based learning").Recent documents (i.e., from 2000) were included for analysis, where possible.
After extensive search efforts, the first round of data collection (covering the period 2000-2007) yielded 183 documents.All documents were reviewed for relevance and retained for coding if they belonged to one or more of the following categories: policies, vision statements, initiatives, and strategic plans and guidelines.Annual reports and standards and guidelines were included if they offered insights about e-learning policies.A total of 107 documents were retained for review.
The second round of data collection was conducted in August 2010 (covering the period from 2008-2010).Initially, 79 documents were retrieved; however, the number was reduced to 67 after the reviewers screened for duplicates in the existing database.After further screening by two reviewers, 36 documents were rejected for bearing little relevance to the main focus of the review.In short, 31 additional documents were added to the data.
Taken together, 138 documents were analyzed and presented in this report.The complete bibliography is available in Appendix A.

Validation and Verification
To ensure accuracy, the 2000-2008 data were verified and validated through a two-stage approach.During the first stage, selected knowledgeable representatives from provincial and territorial departments of education and training and selected federal agencies were asked to verify the relevance of documents identified through a search of government Web sites.Representatives were also asked to identify any reports missing from the initial search.During the second stage, the same group of representatives was asked to review the analysis of policy documents and to confirm whether this analysis appropriately reflected the e-learning policy currently in place in their respective jurisdictions.This process is the equivalent of "member checking" in qualitative empirical research to improve the accuracy and comprehensiveness of the data (Cresswell, 2008).

Coding Procedures
Two complementary approaches were used in coding the included policy documents.One approach used a set of predefined, prescriptive codes, while the second employed an emergent coding scheme allowing the categories to form as documents were read.In both cases, the unit of analysis was an entire document, regardless how often a particular topic emerged within it.The reviewers, working independently, looked for policy-related statements, such as descriptions of goals to achieve (expectations to meet), action plans (means of implementation), or conditions that enable or complicate goal achievements with respect to e-learning.In the instance such information was found during the prescriptive coding, the reviewers decided what pre-established category (Abrami et al., 2006) it reflected, then counted the total number of documents that mentioned the topic.For the emergent coding, reviewers grouped similar topics into clusters to be defined once the coding was completed.Disagreements between individual reviewers were resolved through group discussions.
There are various methods for systematically summarizing non-quantitative data (e.g., Abrami, Borokhovski, Bernard, Wade, Persson, et al., 2010;Bethel & Bernard, 2010;Suri & Clark, 2009).Among them is the Argument Catalogue approach (Abrami, Bernard, & Wade, 2006) that seemed to be the best fitting tool for the purposes of the current review.We realize that the interpretation of results based on a frequency count has its limits, at best reflecting the amount of attention given by policy makers to particular e-learning-related topics and issues, but neither identifying specific policies in place, nor evaluating the quality of their implementation.Also, because many of the documents included imprecise statements and descriptions, it was often difficult to decipher concrete policies or plans.There is, however, a distinct possibility that topics more frequently reflected in the reviewed documents were deemed more important by policy makers.By the same token, missing or rarely mentioned topics may have been of lesser concern to them.

Prescriptive coding
The prescriptive categories (including relevant sub-categories) used in the coding process were determined through a larger-scale review of the literature undertaken by Abrami and colleagues in 2006, which included the following: (1) Presumed benefits of e-learning.These are reports of expected or evaluated positive outcomes of employing e-learning technologies, tools and activities.What are the expected benefits of e-learning?
The purported benefits of e-learning used in this category included better academic achievement; higher motivation for and satisfaction with the learning process; increases in communication and collaboration among all participants in the educational process; decreases in both the direct and indirect costs of formal education (including reductions in the dropout rate); provision of more flexible and accessible learning environments; and fulfillment of social demands, such as the need for professionals who are literate in modern informational technologies and well prepared for the ICT challenges of the global economy.In the policy documents, these benefits can also be perceived as potential goals to be attained as e-learning practices develop and progress.
(2) Support for Implementation.These items described conditions/pre-requisites. What is necessary for e-learning to succeed?Support for implementation is understood here as the creation of environments that are perceived by policy makers as necessary to achieve the e-learning goals.The policy documents often contained " a substantial number of 'conditional' messages, specifying what conditions should be met, what factors taken into account, and what approaches implemented in order to make e-learning really efficient" (Abrami et al., 2006, p. 20).
Conditions favourable to e-learning included the following: provision of professional training for educators, e-learning-oriented course (curricula) development, planning and provision of logistical support and development of the necessary infrastructure, development and availability of print-based resources, development and availability of Web-based resources, development and availability of learning object repositories, and need for empirical research in the area of e-learning or actual use of previous research findings in e-learning practices.
(3) Collaboration.What kind of collaboration has been suggested or exists among the parties involved?
This category also took into account whether the document contained any explicit link (reference) to other policy documents within the same province, across provinces and pan-Canadian agencies or initiatives, or on the federal or international level.
Statements from policy documents falling under this category represent one of three major aspects of planned or actual collaboration among parties involved in e-learning, specifically: coordination of efforts across agencies and institutions; resource sharing by institutions/agencies involved in e-learning practices; and involvement of other parties (e.g., parents and communities).
These were also categorized by level of collaboration reflecting whether the document under review contained any explicit reference to other policy documents within the same province, across provinces and/or involving pan-Canadian agencies and initiatives, or on the federal or international level.
(4) Major areas of e-learning application.What are the major purposes for which e-learning technologies, tools, and approaches have been/should be used?
This category captures areas of public education considered by policy documents to be appropriate for the use of e-learning.The review process identified references to such areas as literacy, numeracy, scientific reasoning, second language learning, connectivity to remote schools/learners, home schooling, and technology integration.
(5) Other (financial and organizational) issues.This broad category subsumes other topics often emerging in the documents, including the following: Economic issues (the two items below distinguish between situations in which the cost of implementation of e-learning practices is given or estimated and those that explicitly assign responsibilities for covering these expenses): Members of the research team reviewed all documents independently.Disagreements in individual coding were discussed and resolved at team meetings.The overall agreement rate for prescriptive coding was 0.62 (Cohen's kappa).

Emergent coding
The emergent coding procedure was conducted in two phases: a review and extraction phase and a coding phase.In the first phase, policy statements were identified through a systematic approach, then specific policy, vision, initiative, plan, report and/or guideline statements were extracted.In the second phase, an emergent coding scheme was developed using a grounded theory approach (Strauss & Corbin, 1998).Key points repeatedly emerging in the reviewed documents were combined into meaningful categories as the review progressed.
Two members of the research team independently reviewed, extracted statements from and coded a representative subset of the included documents.Following the initial coding, the data were compared to establish consistency between the two coders.There was a high level of agreement.For the remaining documents, one researcher identified and extracted relevant statements and the other coded them.The integrity of the coding process was maintained through random checks of consensus regarding the extraction of relevant statements and the identification of codes.Cohen's kappa at this stage of the review was 0.56.

Results
The presentation of the key findings of the review of Canadian policy documents with regard to e-learning both highlights commonalities found in the reviewed documents and identifies gaps and inconsistencies.It is worth mentioning that some documents were deemed supplementary of another primary policy document within the same jurisdiction.Such documents were not coded separately, but added to the codes of the associated primary document.In total, out of the 138 documents included in the review, 98 were identified as primary and 40 as supplementary documents.

Overall Distribution of Documents
Table 1 reports the extent to which policy statements, extracted from policy documents for each jurisdiction, reflect the major themes of the prescriptive coding.In all, 1,353 policy statements were extracted from 98 separate documents, divided unevenly across the provincial and federal jurisdictions (figures in parentheses specify the number of coded documents for each jurisdiction).Generally speaking, statements citing the benefits of and support for e-learning made up between 40% and 55% of the statements.Issues relating to collaboration (i.e., consultation, sharing) ranged between an additional 13% and 29% of the statements.The total percentage of statements regarding the details of implementation in the areas of "applications," "other issues" and "learners," ranged from 18% to 42% of the total number of policy statements.
These findings (themes) are further explained and examples of relevant statements are provided below.They appear in order of importance as judged by the frequency with which they are present in the reviewed documents.

Benefits of e-Learning
Largely in agreement with previous reviews (e.g., Abrami et al., 2006), the "benefits of elearning" category was the most prominent theme in the documents, suggesting that elearning has value as a viable educational tool.Figure 1 shows the percentages of occurrence within each category of this theme.Within this category, the most frequently mentioned subcategories were flexibility/accessibility (82 references), meeting social demands (78 references), interactivity/communication (67 references), and achievement (59 references).The issues of how e-learning can help reduce attrition/improve retention or improve the cost-efficiency ratio were discussed much less frequently (9 & 13 references, respectively).
The majority of the documents contained references to several perceived benefits of elearning.For example, the Saskatchewan Learning 2005Learning -2006 Annual Report stated: "There is also increased interest in the use of technology to meet learning and information needs.The objective underscores the intent of the learning sector to provide equitable access to information and resources . .." (Saskatchewan, Saskatchewan Learning, 2006, p. 28).
Similarly, one of the goals established in the Campus Alberta policy framework was to ensure that learning opportunities for students "are available when and where they are needed and can be accessed through a variety of means" (Alberta, Alberta Learning, 2002, p. 5), referring to the benefits of accessibility and flexibility.The objective to "increase learning opportunities and career options for students" was in the mandate of the Newfoundland Centre for Distance Learning and Innovation (Newfoundland & Labrador, The Centre for Distance Education and Learning, 2003).Please, note that in the following figures the diagrams' percentages represent how often every particular topic mentioned in all documents.They do not add up to 100% as each topic is completely independent across documents -a single document could focus on a single topic, address a few or many.100 % could be achieved for any given topic if all documents under review addressed it.

Support for Implementation
The "support for implementation" theme was second in frequency of occurrence.Within this theme, the use of online resources, naturally, was cited most often (68 references).The subcategory of professional development was mentioned 63 times, indicating that policy makers acknowledge and value teachers' expertise and qualifications.The subcategory of logistic/infrastructure received the same number of references ( 63).These subcategories were followed by course development (50 references).Need for research or actual use of research findings was mentioned in only 32 documents.The two following forms of support for e-learning were the least frequently mentioned: creation of learning object repositories (22 references) and print-based resources (12 references).Figure 2 shows the percentages of occurrence within each category of this theme.
The following examples illustrate a policy statement indicative of e-learning implementation: Manitoba Education, Citizenship and Youth has supported action research on Literacy with ICT Across the Curriculum by providing, over three years, six days of 'coach the mentors' professional learning . . .Manitoba Education, Citizenship and Youth will also provide the following support and resources for implementation: online professional learning opportunities for use within school divisions/schools; online teacher self-assessment rubrics to facilitate goal setting for professional growth; summer institutes for additional divisional/school implementation team members . . .(Manitoba Education, Citizenship and Youth, 2007, p. 31) Likewise, "the e-learning strategy provides initial face-to-face professional development, as well as ongoing professional development on-line, for teachers and other school board staff . .." (Ontario Ministry of Education,p. 2).The same document referred to the issues of course development and online resources for e-learning: The Ontario Educational Resource Bank Access to the Resource Bank provides: a growing collection of learning resources for Kindergarten through Grade 12, all linked to the Ontario curriculum, and most developed by Ontario teachers; an online location for Ontario teachers to share classroom resources that they have developed with other teachers in Ontario schools; search tools for finding resources by grade, subject, strand, overall expectations, and keywords.(p. 2) The need for research was also articulated: ". . .a provincial Centre for Innovation in Education . . .would coordinate activity and support research, innovation and best practices in online learning."(Alberta, Alberta Learning, 2002, April, Appendix I, p. ii), and also "Literacy with ICT Across the Curriculum builds on the framework of Technology As a Foundation Skill Area: A Journey toward Information Technology Literacy.It incorporates promising practices from the current work of . . .educational researchers . .." (Manitoba Education, Citizenship and Youth, 2007, p. 7).Another illustration refers to infrastructure and technological resources for e-learning found in Industry Canada (2005, March): "Canada's world-class Telecommunications infrastructure has enabled Canadian training service providers and multimedia companies to work with content suppliers to build leading-edge e-learning applications.Learners can access training through technologies that use the Internet, e-mail, CD ROMs, DVDs, and satellite and cable TV . .." (p.2).

Collaboration
Collaboration, another pre-condition for enhancing and supporting e-learning implementation, was the third most commonly cited theme in the policy documents.Within this theme, the majority of the documents emphasized the importance of the coordination of efforts involved in the preparation for and implementation of e-learning.Figure 3 shows the percentages of occurrence within each category of this theme.While this was true within jurisdictions, there was inconsistency in educational policies with regard to elearning across provinces.In some documents it was easier to find an explicit link to international experience or collaboration than to find policies about inter-provincial or federal collaboration.More specifically, coordination of efforts across agencies and institutions was the most prominent topic (74 references), followed by sharing resources (54 references) and involvement of other parties (44 references).Cross-referencing between policy documents within the same province was mentioned 43 times, while across provinces it appeared only 26 times, and even less attention given to collaboration on the federal (19 references) and international (13 references) levels.
The following statement is an example of what was coded as evidence of cross-provincial collaboration: "[the goal is to] . . .develop educational products and services that can be exported to other jurisdictions, in particular, the other school systems in Atlantic Canada" (Newfoundland & Labrador, The Centre for Distance Education and Learning, 2003).Similarly, 0% 10% 20% 30% 40% 50% 60% 70% 80% statements were found that referred to resource sharing and collaboration among different institutions: "By March 31, 2007, the Department of Education will have increased the resources and support in K-12 and public post-secondary, and increased cooperation between the public post-secondary institutions to improve quality" (Newfoundland & Labrador, Department of Education. Strategic Plan, 2006, p. 1).There were also policies regarding involving students' parents in e-learning practices: "All parents will be provided with evidence of their child's literacy with ICT so they can support their children" (Manitoba Education, Citizenship and Youth, 2007, p. 10).

Figure 3. Percentage of occurrence of sub-categories of "Collaboration." E-Learning Applications
There appeared to be strong consensus in the policy documents reviewed that learning with technology has great potential in a variety of areas.Connectivity with remote learners was most often described as the major objective for learning with technology (mentioned in 63 documents), whereas there were fewer references for literacy (43).Furthermore, numeracy and scientific reasoning were only mentioned 22 and 19 times, respectively.Finally, references to second language learning and home schooling were virtually nonexistent-10 and 5 times, respectively.Figure 4 shows the percentages of occurrence within each category of this theme.Some documents, such as RÉCIT Quebec (2000), contained references to several major areas of application for e-learning, including literacy, numeracy, scientific reasoning, second language learning, and connectivity to remote schools and learners, while others concentrated solely on one of them (e.g., connectivity to remote schools and learners) (see Government of Nunavut, Department of Education, Nunavut Adult Learning Strategy, 2005;or Clarke, Imrich, Surgenor & Wells, 2003).One illustrative example of a strategy using elearning for literacy and numeracy, scientific reasoning, and in rural communities is the Ontario E-Learning Strategy:

Other Issues
Other issues, such as financial and organizational issues, are also essential for the theme of "support for implementation."This category was not frequently addressed in the reviewed documents.Within this category, the issue of financial responsibilities clearly dominated (47 references), followed by content and course standards (30 references) and more general financial considerations (25 references).By comparison, issues of ethics and security, certification and copyright, and enrollment eligibility received much less attention (24, 21, and 10 references, respectively).Figure 5 shows the percentages of occurrence of each subcategory within this theme.Specific examples of statements illustrating these categories include the following: Funding and responsibilities: The province provides funding to the K-12 and post-secondary systems for developing and establishing an array of online courses and learning resources.Regulations with regard to ethics, copyright and intellectual property: "Ethics, responsibility, and safety: School leaders ensure that their divisional ICT acceptable-use policy is understood and adhered to by staff, students, and parents" and "School leaders ensure that teachers establish rules and procedures related to… plagiarism and copyright of intellectual property" (Manitoba Education, Citizenship and Youth, 2007, p. 33).

Type of Learners
This category, supposedly reflecting the major group of e-learning recipients and targeted educational settings, was the least represented topic in the reviewed documents (only 70 references, in total).Within this category rural and Aboriginal schools and special needs students were mentioned most often (26, 19, and 14 references, respectively), while 0% 10% 20% 30% 40% 50% 60%

Certification and copyright
Ethics & security

Content and standards
Fiscal responsibilities references to gifted students, for example, only appeared 2 times.The virtually nonexistent percentage of references to the average population of learners and urban schools could be attributable to their perception by policy makers as major recipients of e-learning activities by default, though it does not mean that e-learning implementation for this category of users does not require special attention, preparation, and regulation.Figure 6 shows the corresponding percentages within this category.

Findings Summary and Discussion
The intention of this review was not to evaluate the quality of current e-learning initiatives, which far exceeds its scope.Rather, the review is intended to help open a discussion of to what extent Canadian policies with respect to e-learning meet the expectations of its benefits by identifying areas of consensus, collaboration and inconsistency in the reviewed documents.
As the review revealed, There is little doubt that e-learning holds great potential for society.In this review we were primarily interested in exploring the extent to which policy documents reflected plans and action intended to guide the fulfillment of this potential.
Overall, the data suggest that there is a consensus among policy makers in their positive perception of e-learning and its potential to benefit learners.It is clear from this review that Canadian provincial and federal policy makers view technology as offering a variety of benefits to learners.These benefits include flexibility/accessibility, meeting social demands, interactivity/communication, and learner achievement.The extent of attention paid to support of e-learning (i.e., the conditions that are thought to optimize the chances for successful implementation) is also fairly strong, unlike the other coded categories.More specifically, support for implementation received a high level of attention with emphasis on professional training, Web-based resources, and logistic infrastructure.However, the use of research to guide and support e-learning implementation was noticeably underrepresented.Within jurisdictions, there was a high level of attention paid to coordination of efforts across agencies and institutions with an emphasis on sharing resources.Acknowledgment of the importance of collaboration on various levels remains strong, while actual explicit links to other documents (within or across provinces/agencies) seem to be relatively infrequent.The problem of the lack of specifics is also evident with respect to application areas (i.e., in what ways and how much e-learning would benefit the development of literacy, numeracy, scientific reasoning, etc.).These issues were seldom addressed.Also, the types of learners who are targeted by and would potentially benefit the most from e-learning were addressed only minimally by the majority of reviewed documents.Finally, it was observed that little attention was paid to regulation and economic issues pertaining to e-learning.
In brief, the data suggest that policy makers have some confidence and largely agree in understanding what is necessary to make e-learning functional, but are less certain about how, in collaboration with whom, and under what conditions e-learning is most effective.
The relative lack of attention to the issue of how to plan, guide and effectively use relevant research can only aggravate the problem in the future.
Following what McGreal and Anderson (2007) describe as "significant trends in Canadian e-learning that should be watched" (p.4), this study found a number of areas requiring greater attention, such as the development of common standards (either technical or content-related) for establishing learning object repositories, blueprints for policies about credit exchange and acceptance, etc.Interestingly, a much earlier review conducted by Rossiter (2002) identified the steps necessary to implement a Canadian action plan on elearning, including encouraging research in the area of e-learning; identifying best practices; and developing comprehensive standards.Despite these recommendations, the majority of policy documents do not contain adequate insight on these issues.
It is possible, however, that the policy documents at our disposal do not reflect the entire picture, and that there are initiatives and programs in place that pay e-learning more balanced attention.If so, there is a need for higher transparency and better vocalization of educational policies.Given the professional qualifications and expertise of our research team, if some documents escaped our review, it is unlikely that other stakeholders interested in e-learning could locate the necessary information to make informed decisions.
It seems that provincial and federal governments could do better and design and publicize more coherent and thoughtful strategies towards e-learning Ideally, policies are intended to translate insights and consensuses based on research findings, public opinion and the balanced evaluation of needs and resources into realistic action plans.This review is just the first step in attempting to understand the current state of e-learning policy in Canada and how much of it has been lost in translation.
David Pickup, Concordia University responsibility (who is paying for what) Regulations:  Ethics & security  Content standards  Certification & copyright  Eligibility (6) Type of learners and settings.This last category depicts what policy documents describe as the recipients of e-learning activities-what categories of students and what types of formal educational settings would be targeted by e-learning.

Figure 1 .
Figure 1.Percentage of occurrence of sub-categories of "Benefits of e-learning" This strategy is giving teachers access to more resources to help them engage more of their students in learning.It will provide significant support for meeting the government's targets of 75 per cent of students achieving at the provincial standard in literacy and numeracy by 2008 and of 85 per cent of students graduating from high school by 2010-2011.(Ontario, Ministry of Education.E-Learning Ontario, p. 2)    Other examples include the following statements: "School leaders ensure that teachers have access to professional learning opportunities to explore literacy with ICT for . . .higher-level thinking such as comparing, classifying, analyzing, inquiring, problem solving, justifying, and evaluating sources . .."(ManitobaEducation, Citizenship and Youth, 2007, p.  33)  and "Promote IT skills and expand the use of computer technology to help improve education, training and information sharing in rural communities . .."(Newfoundland &  Labrador, Department of Education, Strategic Plan, 2006, p. 27).

Figure 4 .
Figure 4. Percentage of occurrence of sub-categories of "Applications of e-learning"

Figure 6 .
Figure 6.Percentage of occurrence of sub-categories of "Types of learners"Summaries of the review findings organized by jurisdiction-province, territories, and federal agency-are available in Appendix A and include the results of both prescriptive and emergent coding procedures.
Michael A. Surkes, Concordia University This work is licensed under a Creative Commons Attribution 3.0 License.

Home schooling Second language learning Science reasoning Numeracy Technology integration Literacy Connectivity to remote schools / learners are
delivered from the available array.(Saskatchewan,SaskatchewanLearning, 2006,  p. 30)There is no cost for boards to use the learning management system, the Resource Bank, or the technical help desk or to receive professional development.Participating boards must agree to certain terms and conditions, and provide their own equipment and network setups.(Ontario,Ministry of Education, p. 2)