Course Design and Delivery Specifications 1 as a Tool for ensuring Quality in an Online Training Program

This case discusses the design, implementation, and regulation of a hybrid training program (60 credits over two years) organised by three business schools in Europe, and stretching over a five-year period. Following an incremental design process, the design team faced multiple challenges, from finding the added value of hybridization to choosing the technological environment that would be key to the instructional design of the 17 program modules. Subtle interpersonal skills were required to develop a program that suited every professor (n = approximately 40) and every institutional context (3).

six-year European funding package and continued over several years.The challenge of the "Online Manager's Project" was to design and implement a two-year, 60-credit online interuniversity program as a springboard to a Master's degree.The result was a formalized project that targeted professional adults in managerial positions as well as graduates in fields other than management.

Institutional setting and stakeholders
Project team (see figure 1):  Frédérique: senior instructional designer in charge of instructional and technological assistance, University of Lac-Ville (ULV)  Carlos: a junior instructional designer also at ULV  Doriane, project manager, and assistants Sylvie and Andrée, University of Belle-Ville (UBV);  Forty faculty members (professors and teaching assistants/tutors) from the three partner universities (ULV, UBV and University of Grande-Ville, UGV).

Figure 1 a schematic representation of stakeholders and their institutional interconnections
Frédérique became involved in this project that involved several participants.She and Carlos worked at the University of Lac-Ville (ULV) in the Teaching and Learning Center.This center provides both instructional and technological assistance to all faculties (schools).As part of the Online Managers Project (OMP), Frédérique and Carlos were asked to devote half their time providing assistance to faculty, which was why their Teaching and Learning Center was a project partner.This case focuses mainly on the assistance provided by Frédérique, as senior instructional designer.About 40 faculty members (professors and teaching assistants/tutors) from the three partner universities were involved in the training program.The project was led by an executive committee (EC) headed by a professor from each of the three partner universities.
The project funding allowed the EC to put together a course team, which was responsible for the coordination and development of the project.The instructional and technological assistance provided by Frédérique and Carlos was one of the main components of the team.
The team had a workforce of three full-time equivalents (FTEs) divided among five people.

Meta-reflection
Frédérique realised that her and Carlos' roles within the project team would be defined and redefined throughout the project's development.Although they were official members of the project team, the institutional context increased their risk of finding their place in the sun, so to speak.Not being housed within the university that was coordinating the project, they were not directly linked to any of the Business Schools nor did they have as great an investment in the project as did their three colleagues from UBV. However they possessed important expertise in online learning, which the others did not.

Specificities of this case
This training program represented four specific innovations for the 40 faculty members involved (including regular and adjunct professors):  Designing a hybrid course within an online program, a kind of teaching innovation with which none of the partner universities had any experience  Training professional, working adults rather than traditional university students  Organising inter-university training within the same program  Implementing a program approach (Prégent, 2009), that is, a collective project assisted by a team of faculty, embodied 1) by an exit profile as an achievement marker (student learning outcomes at the end of the program) and 2) by educational leadership in order to coordinate the faculty team activities in achieving shared objectives.
The instructional assistance provided by the advisors addressed these four dimensions.

Meta-reflection
Frédérique thought that this assistance dovetailed well with the incremental design approach (Quintin, 2011), wherein project members had to navigate iteratively between the project's macro-level (integration of instructional and technological innovation in three institutions), its mid-level (design of a hybrid training course, face-to-face and online component articulation, technological environment) and its micro-level (the instructional design of every course in the project).

Development of the overall training program
Designing the online managers training program accelerated after European funding for a sixyear project was awarded.The first students were expected within the year so the project team only had a few months to create an online inter-university training program that must:  be attractive to professional adults (managers)  be accepted by each of the partner schools, given the fact that these schools already The first question that Frédérique submitted for discussion was: "What is the added value of organizing an online training program?"The goal is to offer a training program which is flexible in terms of geography (minimizing travel) and in terms of time management (allowing learners to work whenever they wish).However, the suggestion of an entirely distance formula that would maximize these two flexibilities scared the Committee members.

Their comments included:
 "Professors will never want that!They love the face-to-face contact with students too much.That is why they teach!" "Students like contact; this is one of the aspects they are looking for when they enroll in a continuing education."  "An entirely distance training program cannot work at the university level.How can we stimulate student thinking; how can we develop critical thinking remotely?"Meta-reflection In Frédérique's opinion, these comments indicated a lack of awareness about the educational and technological possibilities of managing interaction at a distance.Members of the Executive Committee had never taken a virtual class or webinar, or had interacted online in a structured forum.While the EC members wanted to innovate, this felt like too big a change: everyone wanted to start, but didn't want to commit to too much all at once! Remembering that the distance challenge was not the only one Frédérique had to face, she and her colleagues proposed a hybrid formula: one day a month of face-to-face sessions (on Saturday) and everything else online, a formula which was more flexible than the usual training offered by the three schools (consisting of two or three evening courses a week with an occasional offering on Saturday), while maintaining some face-to-face contact.Thus, the program adopted a blended learning formula, offering 80% of the workload online.
Another critical choice occurred as a result of the required cooperation between schools: European funding was granted to the three schools to work as a consortium.In addition, the inter-university character of the training program was expected to be attractive to the target audience.But how could we make this cooperation effective for students?Frédérique opened the debate with the Executive Committee by reminding them of the original intent of the project -to have each course jointly developed by three professors, one from each school: "The founders of that project believed in sharing practices among professors from the same disciplinary area.But experience has shown us the difficulties with this type of cooperation: professors have extremely tight schedules, complicated by the time devoted to getting to meetings.And then there is competition among these three schools, something that is never totally forgotten, even on teams where there is good will… Remember, we decided to drop the policy of cooperation after two or three attempts at jointly developing courses.Do we want to try it again?Otherwise, what are we going to call this current inter-university training program?" The debate led to the following decisions:  The Executive Committee will assure that professors from each school are in charge of course development.Each institution will be in charge of a third of the courses in the training program.
 Face-to-face sessions will rotate among the three universities;  The project team will plan meetings and share activities among all of the professors involved (from all of the schools) in order to create a "program spirit" worthy of interuniversity cooperation.
"What should our technological environment be?" continued Frédérique.In order to benefit from what was learned from previous attempts, it was agreed that they would continue using the same LMS, one which was well known by professors already involved in the project and easily implemented by new ones.However, the online learning resources previously developed, such as hypertext pages developed one at a time (with the help of HTML software), and a few animations programmed by computer technicians, seemed too amateurish to use in the new training program..

"Our websites are currently developed with and hosted on a CMS (content management system), allowing us to harmonise graphics and to make content updating easier," explained
Frédérique. "If we put all of the hypertexts into one CMS, we could suggest that faculty manage their online contents themselves, once the structure of their online courses has been integrated into the CMS.What do you think?" Frédérique asked.

Self-reflection
Frédérique is convinced she is guiding the Executive Committee in the right direction by proposing the evolution from an LMS to a CMS.However, in the back of her mind, she cannot help but wonder what she is going to think about her choice in a few years' time.
Technology evolves so fast, choices that seem relevant at one moment look outdated shortly afterwards.
This step led to the following set-up: a 60-credit training program, spread over two calendar years, packaged in a hybrid format, using an LMS and hypertext pages developed in a CMS.
From this basic set-up, the program was composed of 17 training modules (14 courses, one final thesis, two residential seminars) as determined by the Executive Committee.No more than two or three courses would be offered simultaneously.The face-to-face sessions would occur alternately on the three universities campuses (see figure 2).Frédérique draws the attention of her colleagues to some safeguards that must be put in place to guide the instructional design of each course:  Design limits: a course spread over two or three months (depending on the number of assigned credits) should be "entitled" to two, three, or four face-to-face sessions (because there is one on campus class a month); its position within the program will be chosen depending on content and on the learning sequencing established over the length of the program.
 Learning outcomes.Each faculty member has to (re)define the learning outcomes based on one fundamental question: "What skills and knowledge does a general manager need to have if s/he is not an expert in finance, marketing, human resources or strategy development but must be able to lead his/her company and interact with these specialists?" Existing courses.Professors have one or more existing courses that could be used as the basis for subsequent course redesign.That means either courses developed as part of the OMP or regular, non-hybrid courses.
 Course design and delivery specifications (i.e.terms of reference) for the new OMP (see below).

Course design and delivery specifications for Online Managers (initial version)
A few years earlier, Frédérique and Doriane, together with the founding professors, had written the Instructional specifications for OMP (see table 1).This reference document defined fundamental educational choices for any course developed as part of the project.It determined the type of e-learning that online managers would like to implement: e-learning implementing interaction between learners (in contrast to self-learning) and interaction with university professors (tutored e-learning) balanced between theory and practice.IDs and project team managers agreed to check periodically if the specifications were being observed in the courses for which they were in charge.

Means
The instructional means used to implement the ends will be the following: 1.
The web resources and the online courses used shall be university level and shall aim at developing knowledge and skills relevant for this educational level.

2.
The online courses shall consist of a series of learning activities designed to make students active learners and not simply observers.

3.
The activities shall be numerous and diversified so that they build enthusiasm and instill a desire to learn.They should allow for greater student autonomy in learning.

4.
The online resources should offer the students opportunities for self-assessment of their progress (exercises, self-correcting tests, submitted tests etc.). 5.
When courses are implemented online, professors shall organise means of assisting and encouraging students during the distance learning phase: online tutoring.

6.
When courses are implemented online, professors shall enhance motivation through interactive activities among students (team work, discussion forums…).

Difficulties of techno-pedagogical support
The educational design process turned out to be difficult for some professors.For example:  Gilbert Leroy really wanted learners to do advanced exercises, requiring advanced skills in calculation and software procedures, because "that is what you have to learn when you study management." Marthe Durant could not imagine doing anything else than lecturing during the rare moments of face-to-face contact she had with her students.As there are fewer face-toface hours than in an ordinary course, she wanted to maximise these hours to the utmost and talk as much as possible! Laurent Dumortier wanted to keep using the hypertext learning aids that he developed previously for his first-year undergraduate students.
These positions led to some intensive discussions within the design team: Andrée: "Can I make Gilbert Leroy change the learning goals in his course?They just aren't relevant for general managers!The design team realised that having faculty change their habits was difficult for some, especially when a course is already online (for instance, the Online Managers course was already online but was not aimed at the audience currently targeted).Frédérique and her colleagues reacted in different ways to these problems.Doriane, Sylvie, and Andrée had serious concerns about achieving a quality instructional structure as quickly as possible (the first version if possible) and, in pursuit of that goal, they took responsibilities upon themselves that were not being completed by faculty: Frédérique and Carlos are reluctant to become substitutes for the professors.Indeed, the mission of the Teaching and Learning Center is to assist professors in their professional development (help them, think with them if they ask) but not to "do things in their stead:" Frédérique: "No, I will not write evaluation criteria instead of the professor.It is his responsibility.However, I suggest calling him every day to make sure he provides them to his students as soon as possible.He has to realise that students need these criteria to get motivated to complete the requested homework."

Metareflection
Frédérique reflects on all of this after the meeting with her colleagues.Her doubts regarding what attitude to adopt in supporting faculty depicts the tension existing between three loyalties of the advisor (Lebrun, Bachy, Docq et al., 2010): loyalty to the faculty member being supported (a desire to meet his/her demands), loyalty to the project (a desire to meet with success) and loyalty to the institution (given the missions and goals assigned to the ID as an employee).Poumay (2006) also highlights the diversity of attitudes that an ID can assume.
Frédérique, reflects on the situation of her three colleagues from Belle-Ville, who were hired by the School of Management on the basis of this funding project and who are not working on any other project at their university, which impacts how they prioritise loyalty to the project.
They want to implement everything they can in order to develop the project as fast as possible while aiming at high quality.They express major concerns about success: first, the graduate students must be satisfied and promote the program, even if the IDs have to assume some of the tasks that faculty do not perform or for which they do not do enough.However, Frédérique is feeling increasingly confident: of course the success of the project matters to her but she believes that her mission, conferred on her by her university, is to stimulate faculty materials production and educational development.For this to occur, faculty must be autonomous and assume the entire responsibility for their actions, even if it means that some trial and error will occur from time to time.Being aware of these attitudes and loyalties enables Frédérique to better understand her colleagues' reactions, to situate them contextually and to discuss with them the limits of their roles and missions as members of the project team.

Implementation of the program and regulations
At the end of the year, the first courses of the program were ready.Students of the first cohort were coming.
First cohort of learners: pioneers "How are we going to find out if the program is satisfactory and efficient for both students and teachers?"Doriane asked Frédérique."Let's think about an assessment plan that will help us bring necessary regulations during the training and assess educational efficiency of the training over the years.I will make a few proposals to this effect," replied Frédérique.
During the first training year, Frédérique and Carlos arranged for three separate student surveys (one focus group during a face-to-face session and two online surveys).These surveys highlighted important disparities between courses:  Some courses were very theoretical whereas others used numerous case studies.
 Some used a textbook, others used only online resources (hypertexts created by the professor).
 Some required substantial homework and on-going assessment, whereas others included only a final exam for assessment.
 Some prescribed assessments in the form of knowledge-based exams, while others used application projects within a professional context.
These various practices were a source of uncertainty and stress for students.Time management also appeared to be a source of stress: most students wanted to receive instructions and deadlines as early as possible, whereas professors tended to think about their face-to-face session only a few days prior to the session and then would send off a number of  the steps to take into account when developing and delivering the course  the required quality criteria for each of these steps (see table 2).

Table 2 An excerpt from the Online Managers Instructional Charter" (second version)
Working stages Quality criteria Building your online course 1. Writing learning objectives  Learning objectives in each course are directly linked to program learning outcomes  Learning objectives are designed so that their mastery can be observed using prescribed assessments 2. Choosing learning activities that allow students to meet the identified objectives  The activities are diversified (reading, exercises and drill, writing papers, tests, case studies…)  Active methods (rather than transmission ones) are favoured  Activities are relevant for the university level: complex problem-solving, scientific literature analysis, critical thinking, creativity and innovation…  Links are made to professional fields (managerial explanations that help students deepen their understanding of content 6. Preparing a printable textbook  There is a printable textbook available to students  Important concepts are explained and referenced in the textbook This instructional charter, sent out to all professors and tutors, highlighted the following aspects:  Level of expected details during course preparation.The following elements must be provided when the course starts: instructions for all assignments and homework, quality criteria to be used for learning assessment; a workload estimation for each scheduled activity.  Types of learning resources to be prioritised: Online resources should exploit the added value of web and multimedia delivery modes (webography, videos, animations, simulations, interactive exercises…).Offline resources, such as student reading material, should also be provided.

Self-reflection:
Frédérique identified dissatisfaction among students regarding available learning resources.
Professors tended to suggest web links and slides for students to print out, but students did not find them sufficiently relevant or helpful.Students asked for access to resources with a structure and an appropriate writing level that would allow for autonomous reading, even years after the courses when they would no longer have access to the platform and would not remember the lectures based on these slides.This request was a surprise for professors, considering that an online course should provide online resources.Thus Frédérique suggested that, in addition to online resources with Web-specific added value, each course should use a course textbook and/or a readings portfolio and/or handouts provided by the professor.
 Type of instructional sequencing scenario to be prioritised: The team recommended starting the course online and ending it online (Figure 3) in which inclass sessions supported online activities (rather than the opposite); individual and group activities were alternated; the final exam was based on individual production linked to students' workplace experiences.Links should be made between theory and practice (exchanges dealing with student experiences, experiences sharing with faculty, feedback from faculty on professional dilemmas experienced by students…).In this way, pure lecturing could be avoided.
Frédérique: "We have to make an effort to use this instructional charter in every meeting with faculty, for each question asked, for each hesitation about course design… These guidelines have to become our reference, our habit!,We should always carry them with us in our pockets!If we ensure that everybody applies these guidelines to the greatest extent possible when we work with faculty, our courses would be more homogeneous, more efficient, and more satisfactory for students."Doriane: "How can we dare ask faculty to meet so many quality criteria?" Carlos: "Faculty are not used to receiving instructions about how to organise their courses.
Won't they argue that we are interfering with their academic freedom?" Frédérique: "We do have an argument to justify these guidelines.It is based on the assessment of our first-year courses; the guidelines are good practices highlighted and requested by students.Furthermore, some faculty members asked for more guidance in the design of their online courses, as they don't really feel comfortable in this new context.So let's challenge them to design quality courses!Innovation requires it!"

Online tutor training
Second-year courses in the program were designed according to these guidelines and the first-  On the other hand, Frédérique discovered that, during this project, the critical need for organisational and administrative support was mainly carried out by Doriane and her Belle-Ville team, namely establishing a collaboration agreement between the three universities, managing copyright issues for online resources developed by faculty, student

Frédérique
and Carlos thus had three colleagues: Doriane, Sylvie, and Andrée.These team members were all hired by the Business School at University Belle-Ville (UBV) as administrative staff.Their mandate included:  general program coordination, both administrative and logistical  administration and monitoring of enrolled students  program promotion and recruitment of future student  provision of a portion of the instructional and technological support for the courses and program.
had a catalogue of on campus lifelong training programs (need to avoid cannibalising the existing training programs being offered)  implement the added value of online learning  use existing resources (online courses offered recently by professors already engaged in the OMP (learner support, tried and true instructional tools, tutoring models developed, etc.).Frédérique invited her colleagues to conduct an in-depth analysis of the context and constraints in order to submit several program alternatives to the Executive Committee."What type of training do we want to offer?Via distance education or something hybrid?How many courses, which ones, and in what order?What are the learning outcomes?And what will be our technological environment?We have our work cut out for us if we want this all to be ready in less than a year!"

Figure 2
Figure 2 An excerpt from the university calendar of the first-year program showing the length of each course (between 8 and 12 weeks long) and Saturday face-to-face meetings (dates in yellow) messages at the last minute.Eventually, a major question arose: How can one optimise the small amount of time available during face-to-face sessions?What do face-to-face sessions add to the learner experience?To understand the added value of e-learning, one had to consider the face-to-face activities."Disparity between courses is problematic," emphasized Frédérique."Let's identify good practices and try to apply them to all of the courses."Coursedesign and delivery specifications (second version)Frédérique had to face the facts: "We did not succeed in applying our specifications satisfactorily to each course.Or they are no longer adapted to our target audience and to our new context.We may have to review them.What is a quality course in this training program?"Frédérique coordinated an in-depth analysis of 1) student feedback collected during course evaluations, 2) course syllabi, 3) course debriefings with professors and tutors.She summarised good practices identified during these exchanges and integrated them into a new version of the specifications which was more detailed and accurate.To better highlight the necessity of a real commitment from professors and tutors in harmonising practices (to a degree), this new version was renamed an Instructional Charter.This charter detailed the design and delivery phases of the course: responsibilities)  Interactions between students are fostered  Students participate in and complete individual and team projects 3. Designing the learning path in the course  Face-to-face meetings and distance activities follow a logical sequence: face-to-face meetings, open and/or closed distance activities  The learning workload is balanced throughout the course  Due dates and course workloads are balanced with those of other courses delivered at the same time  Students are briefed on the expected workload for each learning activity 4. Writing instructions for assignments and homework  Instructions inform students what they are expected to do in terms of production, format, due dates, quality criteria, etc.  Instructions are available one month before the course starts 5. Preparing online learning resources  Learning resources offered online have added value compared to the same offline resources: multimedia (video, audio…), quality of illustrations, hypertext, personalised learning paths, simulations, webography  Information and resources are updated each time the course is offered  When multiple-choice tests are used, they include

Figure 3 
Figure 3 An example of instructional sequencing scenario recommended for the "Online Managers" course year courses were reviewed by taking them into account.Special attention was paid to tutors supporting learners.Not surprisingly for the IDs (although faculty were surprised), student tutoring during the online learning segments of the courses turned out to be a critical aspect, in terms of efficiency, of the first-year course.Thus Frédérique added tutor expectations to the new specs.The kind of tutoring recommended was proactive: tutors should take the initiative in contacting students, either as a group or individually to assist them throughout the course.Students expected qualitative feedback for every assignment and not just a score.This feedback had to allow students to deepen their learning and professional thinking (i.e.going beyond simply telling them what was right and what was wrong).

Figure 4 A
Figure 4 A conceptual map describing roles of different actors taking place in the online This is tricky because he is an expert in his field… And after all, what would we do if he decided to withdraw from the project".Doriane: "This is touchy.Let me think about it."Sylvie: "Can I tell Marthe Durant she should facilitate a case study during her face-toface session?Students can learn all the content on their own by reading or listening to podcasts.But what if she refuses?What should I do?Should I go so far as to facilitate the case study myself?"Carlos: "How do I make it clear that the hypertext developed by Laurent Dumortier is not adapted to an adult enrolled in professional training?He is very enthusiastic about using it even though, just between us, his first year students don't use it a lot…" Frédérique: "These are excellent questions.We have to discuss them." They will never have to do these calculations by themselves!Doriane, could you, as Project Coordinator, make him do that?