Touch Screen Technology: Implementing a Technologically - Enhanced Profiling System for Student Sport Coaches

This exploratory case study evaluates the implementation and use by student coaches of an innovative coaching profiling system, Touch Screen Technology (TST), to assess coaching behaviours. The study was designed to evaluate the potential of this technologically-enhanced assessment system as a profiling option for gathering, storing, retrieving, and presenting data about coaching competences. The case study documents and evaluates trainee coaches’ (N=100) experiences of using TST during an internship, identifying the advantages and challenges of implementation, and evaluating the potential for coach education and development. Evidence was triangulated from questionnaires, journals, and interviews (N=20). The responses from the coaches were overwhelmingly positive. The touch screen technology, specified assessment criteria, and graphical profiling helped to integrate assessment into the program along with increased awareness and understanding of the assessment process. The students’ subsequent reflections on the components of the coaching process positively impacted their performance. The study concludes that TST has the potential to assist in enhancing the learning process and bridging the gap between education and practice. Attention is drawn to the challenges of implementation. entraîneurs. Les preuves ont été triangulées à partir de questionnaires, de journaux et d’entrevues (N=20). Les réponses des entraîneurs ont été très largement positives. La technologie des écrans tactiles, les critères d’évaluation précisés et le profilage graphique, tout comme la sensibilisation et la compréhension accrue du processus d’évaluation, ont aidé à intégrer l’évaluation au programme. Les réflexions subséquentes des élèves sur les composantes du processus d’entraînement ont eu une incidence positive sur leur rendement. L’étude conclut que la TÉT a le potentiel d’aider à améliorer le processus d’apprentissage et à combler l’écueil entre l’éducation et la pratique. L’étude attire l’attention vers les défis liés à la mise en œuvre. This exploratory study examines TST as a profiling option when used by trainee coaches (interns) as part of a 12-week Introduction to Coaching module on a Sports Science and Management undergraduate program, and a subsequent 24-week coaching internship. The study was designed to examine the potential of TST as a profiling option for gathering, storing, retrieving, and presenting data about coaching competences. The purpose was three-fold: first, to evaluate trainee coaches’ experiences of using TST; second, to identify and review the advantages and disadvantages of using TST; and third, to consider the potential impact of such technologies on coach education and development.


our les entra
neurs sportifs en formation
1644F3EEDD9456B70F421D1FD09C445410.21432/cjlt27776
This exploratory case study evaluates the implementation and use by student coaches of an innovative coaching profiling system, Touch Screen Technology (TST), to assess coaching behaviours.The study was designed to evaluate the potential of this technologically-enhanced assessment system as a profiling option for gathering, storing, retrieving, and presenting data about coaching competences.The case study documents and evaluates trainee coaches' (N=100) experiences of using TST during an internship, identifying the advantages and challenges of implementation, and evaluating the potential for coach education and development.Evidence was triangulated from questionnaires, journals, and interviews (N=20).The responses from the coaches were overwhelmingly positive.The touch screen technology, specified assessment criteria, and graphical profiling helped to integrate assessment into the program along with increased awareness and understanding of the assessment process.The students' subsequent reflections on the components of the coaching process positively impacted their performance.The study concludes that TST has the potential to assist in enhancing the learning process and bridging the gap between education and practice.Attention is drawn to the challenges of implementation.entraîneurs.Les preuves ont été triangulées à partir de questionnaires, de journaux et d'entrevues (N=20).Les réponses des entraîneurs ont été très largement positives.La technologie des écrans tactiles, les critères d'évaluation précisés et le profilage graphique, tout comme la sensibilisation et la compréhension accrue du processus d'évaluation, ont aidé à intégrer l'évaluation au programme.Les réflexions subséquentes des élèves sur les composantes du processus d' ntraînement ont eu une incidence positive sur leur rendement.L'étude conclut que la TÉT a le potentiel d'aider à améliorer le processus d'apprentissage et à combler l'écueil entre l'éducation et la pratique.L'étude attire l'attention vers les défis liés à la mise en oeuvre.

Résumé

Cette étude de cas exploratoire évalue la mise en oeuvre et l'utilisation, par les élèves entraîneurs, d'un système innovateur de profilage d'entraîneurs, la technologie des écrans tactiles (« TÉT »), pour évaluer les comportements des entraîneurs.L'étude a été conçue pour évaluer le potentiel de ce système d'évaluation technologiquement amélioré comme option de profilage pour la collecte, l'entreposage, la récupération et la présentation de données sur les compétences des entraîneurs.L'étude de cas documente et évalue les expériences d'utilisation de la TÉT par les entraîneurs stagiaires (N=100) durant un stage tout en relevant les avantages et les défis liés à la mise en oeuvre et en évaluant le potentiel pour la formation et le développement des


Introduction

Assessment of sport coaching expertise has re eived less attention than it merits (Cassidy, Jones, & Potrac, 2009;Papailiou, Strigas, Travlos, & Kipreos, 2015), but its value as a learning process is undeniable (Araya, Bennie, & O'Connor, 2015).This paper reports on the implementation of innovative technology as a method of assessment, using a newly constructed profiling system based on TST.Much of the existing feedback associated with coaching expertise and interventions has been founded on performance profiles that provide relevant information about coaching performance (Butterworth, O'Donoghue, & Cropley, 2013), and a number of validated observation systems on coaching behaviours have been developed (Cushion, Harvey, Muir & Nelson, 2012).However, these have not been incorporated into coaching education and development.Profiling provides a mechanism for coaches to capture their behaviour and practice, and thereby to test their competence and progress.Performance profiling has traditionally been used in the analysis of athlete performance as a tool to assist in the identification of appropriate goals and interventions.A key principle is establishing a selfawareness of current capacity as the basis for improvement.
he earlier work in this domain by Butler and Hardy (1992) and Jones (1993) has been revisited by Gucciardi and Gordon (2009), Weston, Greenlees, and Thelwell (2011), and Butterworth et al. (2013).Rynne, Mallett, and Rabjohns (2017) have recently acknowledged the increased attention given to profiling, what they term the assessment of relative competence against identifiable criteria.They counsel that profiles should be based on attributes for which the coach can be held accountable (meaning personal attributes rather than athlete performance), and acknowledge the need for further research.

Nevertheless, much of the greater part of academic writing on profiling has focused on the athlete's performance (e.g., Vinson & Peters, 2016), but, in this, the coach's expertise as an element of delivery has often been taken for granted, despite the fact that similar principles can be applied.In addition, research on the use of technologically assisted profiling of coaches' expertise has been conspicuous by its absence.Not surprisingly, there has been a general movement towards learning environments that are supported by the use of technology, including for sport teachers (Borchert & Schlöffel, 2018).However, research on coaches' adoption of technological assessment as a component of their coaching or pedagogical practices is limited.Many coaches are not sure how to integrate technology into their everyday coaching practice and consequently may be resistant to doing so (Fleischer, 2012).This is exacerbated by the complexity and particularity of the coaching process, which tends to militate against comparison of practice and exemplification of good practice (Lyle & Cushion, 2017).

There is little doubt that assessment and feedback are essential elements of adult learning (Race, 2014;Sambell, McDowell, & Montgomery, 2013).In the context of education and training, we interpret assessment to imply "the gathering, recording and using of information about a learner's performance/achievement in a task" (Scottish Qualifications Authority, n.d.).Assessment has a number of purposes, including its summative role in evaluating progress towards a qualification or stage in learning (Lau, 2016).However, in this study, we stress the formative nature of assessment (Bennett, 2011): motivating the learner, promoting reflection and, in particular, evaluating current status and future learning needs.Feedback is acknowledged to be crucial in learning, although its practice produces mixed results (Fyfe & Rittle-Johnson, 2015;O'Donovan, Rust, & Price, 2016).Feedback can be synonymous with formative feedback, and refers to the provision of information on performance that will allow students to monitor, evaluate, and regulate their own learning.It is recognized to be important that students engage actively in the feedback process (Molloy & Boud, 2014;Nicol, 2010).
There are a number of studies that help to scaffold the introduction of innovative technology to support learning, and these provide useful principles with which to evaluate innovation.Cook, Holley, and Andrew (2007) identify the practitioners' recognition of the need for change as a crucial factor in the early stages of the implementation of new processes.Drent and Meelissen (2008) emphasise the importance of positive attitudes by tutors and familiarity with computer technology.Lillejord, Børte, Nesje, and Ruud (2018) carried out a systematic review of the role of technology in learning and teaching in higher education.They stress the importance of not simply incorporating i

ovative technology within traditional delive
y methods, as well as the role of students' active learning in the effectiveness of new technology.In relation to research on educational technology, Chew, Cheng, Kinshuk, and Chen (2018) warn against focusing on outcomes rather than the challenges of implementation.Of particular interest is the application of modern technologies to profiling.Research suggests that the assessment process provides an opportunity to facilitate both the athlete's and the coach's capacity to integrate technology into their practice (Kirkbride, 2013).Therefore, coach education programs with a profiling tool provide a vehicle through which to enable prospective coaches to develop the necessary competencies to use information communication technology (ICT) effectively.Butterworth et al. (2013) suggest that ICT methods would be beneficial for assessing characteristics of individual coaches, identifying and developing talent, planning coaching programs, analyzing performance, evaluating training, and evaluating technological equipment.The introduction of ICT offers coaches the potential to improve the quality of coaching by achieving a more athlete-centred learning environment, developing problem solving skills, and requiring higher order thinking skills through differentiated pedagogies (Penual, 2006).

This study has been particularly influenced by the work of Race (2014;2015) on factors that underpin successful learning.Race envisions seven factors acting like "ripples on the water".At the centre are the le rners' desire to learn and recognition of the need to learn.These are followed by "learning by doing" and "learning through feedback".Learning is subsequently facilitated by the learner "making sense" of the learning, assisted by verbalising it orally, and , finally, "learning through assessing"in other words, testing the learning.These factors reinforce the earlier comments about students' active engagement in feedback.Profiling is a learning tool that offers an opportunity for identifying learning needs; it involves an active process of obtaining feedback on performance, and provides a means of assessing progress.This exploratory study examines TST as a profiling option when used by trainee coaches (interns) as part of a 12-week Introduction to Coaching module on a Sports Science and Management undergraduate program, and a subsequent 24-week coaching internship.The study was designed to examine the potential of TST as a profiling option for gathering, storing, retrieving, and presenting data about coaching competences.The purpose was three-fold: first, to evaluate trainee

aches' exper
ences of using TST; second, to identify and review the advantages and disadvantages of using TST; and third, to consider the potential impact of such technologies on coach education and development.


Touch Screen Technology -The Profiling Tool

The sport coaching competencies that comprised the performance profile were organized into three groupingsplanning, delivery, and game knowledgeand expanded into 25 headline categories (see Figure 1).They were derived from those adopted for coach accreditation purposes in coach education, more specifically, Levels 1 through 3 of World Rugby's coach education award program (http://www.coaching.worldrugby.org/index.php?page+160).For example, the construct "delivery" included items such as providing clear explanations, checking for un erstanding, and controlling the pace of the session.Observable performance outcomes were generated for coaching competencies at four levels, and captured, recorded, stored, and shared using digital assessment disks/profiles.Adopting a framework transcript developed from World Rugby's existing accreditation criteria, the profiles satisfy the requirement for content validity, denoting that they cover important aspects of coaching intervention.In addition, validation of the key coaching competencies was sought from a panel of four practicing and experienced experts, who were both renowned national and international coaches and coach educators (the panel's expertise derives from engagement with World Rugby and/or International Amateur Athletics Federation).An innovative feature of TST was that it allowed comprehensive, robust, quick analysis live or post-performance, as a consequence of the capture, storing and sharing of rated coaching performance data.

Using TST, coaching behaviour is analyzed by capturing video-recorded coaching practice on an appropriate technological device and then rating the performance within one of four levels (enthusiast, emerger, evolver, and expert).The coaching criteria, consisting of outcome-based observable behaviours, are arranged on a circular disk, in order that coaches may view their coaching performance while simultaneously referring to specific outcomes.The results for each of the 25 observable coaching competencies (see Figure 1) are presented visually, includi g total percentage scores and individual comments.TST profiles are able to represent coaching performance in three ways: as individual session profiles, as typical (stable value over a number of performances) demonstrations of competence, and as items prioritised for further development.The videos promote coach reflection and understanding, assisted by an aggregated profile.These objective data, along with clear supporting video evidence, enable a more complete interpretation of coaching practice and development decisions.


Methodology

The purpose of the study was to gain an insight into the introduction of TST profiling into student coaches' practice, for which a multiple-site interpretative case study (Yin, 1989) was considered an appropriate approach.This exploratory case study design, which is not uncommon when researching innovation in education (Hastie & Glotova, 2012), had two important characteristics: first, the focus was less about the design of the profiling tool itself, and more about its implementation and the perceived impact on both assessment practice and coaching knowledge.Second, the study was approached from an insider perspective (Eide & Kahn, 2008).The first author was the designer of the TST tool (http://www.kritique.pro)and delivered the university module in which it was used.

A qualitative methodology was adopted to enable the breadth and richness of the student coaches' responses to be explored fully (Charmez, 2000).Thematic analy

s was employed
to facilitate an in-depth exploration of responses, and multiple interpretations of data (Chesterfield, Potrac, & Jones, 2010).The insider research lens enabled a broadly constructivist stance.This acknowledged an existing reciprocity between the first author and student coaches as he was in possession of prior knowledge of the coaches as a consequence of working with them in both university and work placement contexts (De Martin-Silva, Fonseca, Jones, Morgan, & Mesquita, 2015).The benefits of undertaking insider research include gaining a greater insight into the coaching issues being raised, knowing how to approach individuals a d making them feel at ease, and using pre-formed relationships and trust, which encourages the coaches to be more honest in their responses (Cushion, Armour, & Jones, 2003).Nevertheless, this approach is not without its potential disadvantages (Anderson, Varnhagen, & Campbell, 1998), including concerns over revealing insecurities to colleagues, receiving criticism or offending individuals because of existing relationships which may result in the participants feeling that they were not able to be entirely honest in their responses.Sensitivities to the implications of adopting this method were recognised and the researcher reminded coaches to be "critical consumers" (Zhu, 2002) when using TST.

In considering the experiences of student coaches, it was hoped that there would be a generation of new connections, insights, and understanding of related issues that enhanced highquality coaching (Mallett & Dickens, 2009).When employing a case study method, it is important to produce a coherent interpretive account based on the insights, impact, and evidence gained within the study's framework (Cohen & Manion, 1994).When analyzing data, there is also a strong imperative to achieve rigour in both the collection of data and its analysis.This study embraces the persuasive power of interpretive case study research and sets out

convince the reader t
at the data are trustworthy (Bogdan & Biklen, 1992).

The implementation strategy was derived from a change model based on a theoretical framework (Fullan, 1991).This model identifies a three-step process of change.First, new or revisited materials, such as the profiling tool, are introduced.Second, attention is paid to the skills and application necessary to implement changed practice, and third, a change in attitude to its adoption is facilitated.We find it useful to conceptualise this as an introduction, operationalisation, and incorporation process, and we have been mindful of these steps in our study design and analysis of findings.It is also important to differentiate between voluntary and imposed change.It should also be acknowledged that all student coaches volunteered to take part in the study.

Participants.Of the 138 final-year coaching students on a four-year BSc Sports Science program, 100 agreed to participate in the study.The course aims to foster coaching knowledge and practical development through a posi

ve learning environmen
with assessable components that are related to real coaching experiences (Denison, 2010).The course comprises athlete development, teaching for coaching (e.g., communication for coaches, planning and organization of training sessions), coaching assessment (e.g., technology for coaching), and contemporary trends in sports and the practical operation of coaching (e.g., effective coaching,

eadership).

Th
sample consisted of 68 males and 32 females, aged between 21-34, all of whom were currently coaching in schools or clubs.Participants received a detailed information sheet prior to volunteering to participate in the study.They were informed and continually reminded that they were free to withdraw from the study without penalty, and were repeatedly assured that their decision about whether or not to participate in the study would not affect their eventual grade or subsequent tr atment by the course tutor.Strict confidentiality during reporting was assured.Ethical approval was obtained from the University's Internal Research Board.Permission was sought and given confirming investigator access, and analysis of coaching diaries.During the module itself, attention was drawn to the processes and recent innovative developments in the assessment of coaching expertise, and the TST tool was introduced in this context.

Parti ipants were required to plan and coach for 24 weeks during their internship, reflect regularly on sessions, revise coaching pedagogy and plans accordingly, analyze their practice through self-reflection and provide commentary on videotapes of their coaching using TST disks, and reflect on which coaching strategies were successful, which were not, and their perceived reasons for these outcomes.

Participant training.Prior to a 24-week internship, 100 participant coaches had been introduced to TST as a means of assessing coaching behaviour during their 12-week undergraduate coaching course, comprising 12 lectures (60 mins) and 12 practical workshops (180 mins; Appendix A).The first author modelled the TST profiling tool during the lecture component of the module but avoided expressing a personal opinion about the efficacy of such a self-assessment process.TST was introduced as an analytical tool to assess, provide retrievable data, and track coaching performance.This involved becoming familiar with the technology involved and the practicalities of using the system and incorporating it into the coaches' practice.Thus, attention was given to creating a profile, capturing performance on video, identifying performance cues for each criterion, populating the profile, analyzing and interpreting the profile, amending practice, and re-assessing.The practice data were saved and made accessible to peers or used as a self-assessment tool.

Resear h instruments.Evidence of the coaches' responses to using the TST profiling tool was collected by questionnaires, journals, and interviews.Data were triangulated to provide a robust interpretation of these responses, whilst avoiding the potential pitfalls of researcher bias.Coaches had confirmed their understanding of the procedures and given permission for access to the data and its dissemination.

Questionnaire.A two-part questionnaire was developed (Appendix B).The first part consisted of short open-ended and closed-response questions designed to gather evidence about the student coaches' previous experience of coaching assessment.The second part investigated their experience of using ICT generally, and TST specifically.The questionnaire was completed twice: the first occasion at the beginning of the coaching module, and the seco

post in
ernship.There was a 100% response rate.

Coaches' Journals.Participants maintained daily journals for the duration of the study, including the 12-week lecture module and

he 24-week internship in their various sports
lub environments.Coaches were encouraged to reflect regularly and report their use of TST profiling.In general, journals were comprehensively and insightfully maintained in electronic format.Guidance for journal writing was provided during the course (Appendix C).

Interviews.It was also thought to be important to hear the coaches' "voices" in reporting the use of the profiling tool.A purposive selection of interviewees (Robinson, 2014) was based on comprehensiveness of journal maintenance and insightful responses to questionnaire items.Twenty coaches (15 males and 5 females) were selected and interviewed using a semi-structured interview schedule post internship (Appendix D).Interviews focussed on participants' experiences when using TST.Information was used to scaffold findings, discussion, recommendations, and conclusions about the coaches use of TST.Bogdan and Biklen's (1992) advice was followed and descriptive data gathered in the c aches own words, allowing the development of insights into how coaches interpreted their world.

Data Analysis.The study's purpose was to gain insight into student coaches' experiences of using the TST profiling tool to assess and develop their coaching behaviour.It is acknowledged that a substantial amount of data was generated.However, the nature and focus of this initial exploratory case study resulted in limited treatment of the extensive research information.As this was an exploratory study, it was considered appropriate to use simple descriptive statistics to evidence the weight of student coaches' opinions and enhance the qualitative data.These were aggregated with documentary analysis of journals and interviews to provide a narrative summary.

Inductive and constant comparison methods (Patton, 2002) were employed to analyze journals and interview scripts (Appendix E) in which categories of phenomena, relationships, and interpretative meaning were developed.Using this process meaning is attributed by the researchers to passages of the journal text corresponding to and i

ntifying the challenges and advantages of the assessment prof
ling.Thereafter, these are evolved into more general themes (Ritchie & Lewis, 2003).Reliable interpretation of the data was sought by having four assessment and coaching research experts view the data analysis and check for bias in interpretation.Evidence was triangulated by comparing and contrasting data from questionnaires, journals, and interviews.Pseudonyms were used for coaches when reporting the research findings to ensure anonymity.


Results

The results are presented in three sections: student coaches' experiences preand postinternship, the emerging challenges coaches faced, and the advantages of using TST.


Section One: Pre-and Post-Internship Opinions

It is important to acknowledge both the starting points and the changes in the coaches' opinions from pre-to post-internship, as expressed in the questionnaires.Given the student coaches' relative inexperience, it might be expected that, over time, they would feel more confident about their familiarity with the coaching context and the specificity of the ICT.Perhaps not surprisingly, the training received (and, to some extent, the absence of previous exposure to ICT) resulted in the percentage of coaches feeling appropriately prepared increasing from 26% to 74%.Those who responded very positively to questions about their perceived competence had risen from 20% to 44%.A lesser increase in those expressing confidence in the system's use (from 40% to 52%) perhaps reflected the time necessary to feel comfortab e with new technology.However, the proportion of coaches willing to use TST in their future practice had risen from 36% to 80%.

Comments from coaches supported this positive change.Planning was now "careful and thoughtful" (Asha, journal, p. 8).Coaches reported that they developed "strategies to accommodate all 25 assessment criteria when coaching" (Jon,interview,p. 11).Although coaches considered themselves to be more confident and competent, they acknowledged that "we need continued professional development" (Jemma, journal, p. 2).Coaches recognised that adopting TST changed their approach to their coaching behaviour as a result of applying the assessment criteria.They expressed increased incentives to be reflective about their coaching and to be aware of the required l

dership capacities.Finally, coaches held strong opinions, p
st-internship, about the opportunity to use TST in the future: "the assessment criteria set excellent coaching directives and improved communication and [there were better] relationships among our coaching group" (Ben, journal, p. 15).Coaches adopting TST could envisage it as a vehicle for development: "planning how progress was to be made" (Jon,interview,p. 11).


Section Two: Emerging Challenges for Touch Screen Technology

C aches identified several challenges for the implementation of TST, including, time for training (87%), finance (72%), increased exposure to the use of technology (58%), concerns about the reliability of rating scores (53%), and lack of confidence in using technology (47%).Although ICT training was delivered during the coaching course itself, this was generic and nonsport specific.Coaches identified this introduction to TST as their first coaching-specific application, and, not surprisingly, 87% of the coaches identified a need for more formal and coaching-focused technology training as a component of their education."There is so much to cover in our training, we don't get encouraged to use ICT." (Paul,journal,p. 14).Not surprisingly, coaches would welcome increased time for training in the use of TST: "the more we experience TST the easier it becomes and the more proficient we became" (Simon, journal, p. 6).A significant proportion of coaches (68%) highlighted the need for improved time management on their part.The availability of technological support in sports clubs was also an issue: "Not all clubs have devices for us to use and outlay might be considered as prohibitive" (Simon, journal, p. 8); "I used my own iPad because there was not one at the club -if we think assessment is important then we need to invest in it" (Veronica, interview, p. 5).Coaches thought that, in training courses, increased budgeting should be in place to support the use of TST.Some concern was expressed about the element of subjectivity in the process of rating the observed coaching behaviours within each level: "[Rating is] complicated by subjectivity . . .levels 1 and 4 are easy to rate . . .levels 2 and 3 are more difficult" (Sylvia, journal, p. 8); "the consistency of the score so that test and retest provide similar results.I am confident that scorer reliability will improve with constant use and or training" (Adam, journal, p. 5).As might be expected, some coaches were initially rather wary of using the techn logy, although this diminished over time.The fear of losing assessment data was highlighted by 65% of coaches: "It was always in the back of my mind that I am not a 'techie', that I will lose my results" (Sally, journal, p. 12).


Section Three: Advantages of Using Touch Screen Technology

The generally positive response to the use of TST for assessment purposes resulted in coaches identifying a range of advantages.Those highlighted by more than 80% of the coaches were improved coaching performance (96%), greater familiarity with coaching concepts and profiling (93%), greater understanding of the assessment process (90%), data and trends presented visually (87%), and increased interaction between coaches (84%).

Most significantly, coaches identified a number of ways in which the use of TST had helped their coaching: "My understanding of coaching and consequent ability improved . . .assessment criteria guided me" (Adam,journal,p. 14).Coaches attributed performance improvement to be a direct result of developing " common language for assessment" (Sylvia, journal, p. 2); "Clarity of assessment criteria and coaching competencies vastly improved my coaching" (June, journal, p. 3); "TST assessment promoted motivation because of clear articulated and identified performance indicators and competencies" (Sally, journal p. 9].Interestingly, 66% of coaches reported that TST simplified the coaching assessment process, "TST saved time when assessing" (Simon,journal,p. 3); "The saving of tons of record keeping time" (June,journal,p. 8).Coaches were involved in assessment training as they were given regular individual guidance on how to use TS to rate their coaching using performance cues and assessment criteria.Coaches reported that the TST appeared to be effective as it led to permanent, easily stored, and retrievable assessment data.One coach commented "Constant assessment led to reflection on my coaching . . .my coaching became more focussed because I followed the plan, deliver and reflect process" (Simon,interview,p. 4).

Coaches reported that TST helped them to create more meaningful assessment, as the requirements for coach assessment were clearly defined and coaches had improved knowledge of what was to be assessed: "TST gave us clearer coaching expectations" (Jennifer, journal, p. 4); "I coached with better understanding" (Amy, interview, p. 4); "I coached with improved clarity because of assessment" (Ronald,journal,p. 8).In evaluating their competencies, the "TST criteria were simple and impactful" (Paula, journal, p. 8) and these were used to align intended outcomes with observable behaviour: "Coaching competencies, performance cues and assessment criteria were matched" (Leo, journal, p. 2).

Most significantly, 92% of coaches reported that assessment allowed and encouraged them to reflect on their coaching practice: "Assessment meant more to me and I understood the coaching process much better ….Assessment theory was practically applied" (Peter,journal,p. 3].The educational merits of assessment and the value of feedback for their learning and improvement were also identified by th coaches.As a consequence of using TST, they developed an increased familiarity with assessment discourse.86% of coaches reported that they acquired an improved appreciation of the potential power of assessment tools: "[we developed] ownership of assessment" (David, interview, p. 12), "assessment was on the agenda" (June, journal, p. 5).One coach expressed the opinion that "We started to talk regularly about our coaching using TST competencies" (Peter, journal, p. 3).Coaches reported that TST "provided a benchmark for planning coaching competencies, performance doubled up as assessment criteria."(Ben,interview,p. 13).

TST involved the coaches in both self-and peer-assessment, and this led to more purposeful interaction between coaches, including face-to-face and online sharing of comments about assessment criteria: "Talking in groups made coaching easier" (Ben, interview, p. 4); "TST created opportunities for us to work together, share responsibility and allow interaction" (Jennifer, interview, p. 9); "we discussed our coaching using assessment competencies, swopped stories and experiences about how we used TST (Kharil,journal,p. 11).A significant outcome of the study was "the development of a coach network" (Sylvia, journal, p. 7).

In addition to the above, coaches pointed out many practical advantages: "Assessing was easier, more convenient and useful to track what I was good at but also where I needed to improve . . .we responded to simple and constant feedback" (Siti, journal, p. 8).There were many comments about efficient application: "After initial investment it became obvious that assessment was going to improve because it was no longer laborious and once constructed, once made up, the profile can be replic

ed and used time and time ag
in."(Joelle, interview p. 4); "I quickly got to use TST after a few minutes training.It was great to look at and use the video to correct where I was going with my coaching" (Asha, journal p. 11); "Gone are the day when we worry about losing pen and paper reports TST is stored in the cloud and can be retrieved and shared much easier than traditional assessment at the push of a key . . .much better" (Sam, interview, p. 14); "I liked the fact that assessment was easy to do, either peer or self-assessment, there was no need for special set up.We coached as usual and used the disk to assess our performance" (Siti, journal, p. 3).The challenge of allocating time to assess coaching appeared to diminish by combining the assessment criteria and performance cues.Planning is fundamental to the coaching process: "I liked the fact that prior to coaching I could look at the disk and see what I needed to focus on in my coaching" (Ron, interview, p. 5).

The combination of video footage, touch screen visual presentation, and identified criteria was an evident advantage: "The half screen function allowed evidence-based asse sment" (Ruth, journal, p. 5).Video evidence proved to be a highly robust and useful source of performance evidence.The process of collecting evidence using TST was not overly demanding, nor resource intensive: "Cool, I could get an overview of my coaching immediately I saw what I was good atgreen, and what I needed to focus onred, and those in between.I want to coach so that I get all greens" (Steve, interview, p. 5).Coaches stated that the TST summaries proved useful for auditing their coaching performance, with subsequent development potential for learning and accountability, and as a valuable foundation for future practice."The disk categories of plan, deliver and content certainly open my eyes to some coaching option to promote inclusive participation" (Samantha, interview, p. 8).Coaches became more aware of the competencies as they rated their coaching behaviours, although some coaches acknowledged some inconsistencies in scoring.

In summary, there was significant support for using a well-designed analytical profiling and assessment tool to assess coaching, and the introduction of the TST had a marked impact on the coaches' attitude to this form of assessment.Nevertheless, the trainee coaches identified a number of implementation challenges; these related to the time available for training, modelling of the assessment technology, and issues of validity and reliability.The advantages identified included collaborative working practices, developing assessment rhetoric, and employing observable assessment criteria as instructional cues.


Discussions and Conclusions

The purpose of the study was to examine the potential of TST as a profiling option for gathering, storing, retrieving, and presenting data about coaching competencies.This was done by evaluating the use of the profiling tool by student coaches during a sport coaching internship.There is no doubt that this approach to assessing coaching expertise was well received by the participants.Those who engaged with it were better informed about the assessment process and had an increased awareness of the coaching process as a result of being exposed to these competences, as well as from rating themselves on the criteria provided.It should be borne in mind, however, that the subjects were young, relatively inexperienced coaches, and many will have been keen to experiment with the system as part of their coaching practice module.In other words, they may have had relatively limited experience of alternative means of assessing coaching behaviour.In addition, many of the student coaches were familiar with modern communications technology, and will have been relatively c mfortable with a system such as TST profiling.

The coaching process for these student coaches might be most appropriately termed participation or development coaching (Lyle & Cushion, 2017), for most it meant teaching sport to young people.In such a situation, the emphasis is on the quality of the coaches' behaviour for both constructing suitable learning opportunities and for maintaining participants' interest and enthusiasm.The TST profiling system highlighted the competences that were appropriate to these circumstances.The coaches' responses indicate that they found that the assessment process became much more than a "summative add-on" to the coaching experience.The use of TST provided coaches with op ortunities to embed assessment in their coaching practice.

Not surprisingly, the coaches identified a number of limitations in the operationalization of TST, these were mainly concerned with the practicalities involved in using it.This is important, however, as the supportive context within which the student coaches were operating may not always be available for coaches more widely.Further work is required to examine how best the obvious advantages of TST could most usefully be realized for practicing coaches, particularly more experienced performance sport coaches.Providing access to ICT is not sufficient for successful implementation (Cushion, 2008); there is a need for on-going provision of training if it is to be used effectively.The need for familiarization, technological back-up, and a collegial support network suggests that the TST approach may be most applicable in coach education and development.In this context, coach educators can create awareness of innovative and appropriate use of ICT and act as exemplars for coaches by modelling its application.However, a significant recommendation is that the TST assessment tool is not a stand-alone instrument; it should be complemented by clear fe dback and a well-structured remediation/development program (Redelius & Hay, 2009).It can be argued that using an existing set of coaching competences provided an appropriate measure of validity for the TST instrument.However, a significant issue is the question of reliability in the rating of coaching behaviours.The four-level visual rating scheme was found to be useful by the coaches, but a number of them raised the issue of consistency of ratings.Some further work is required to ensure an inter-rater consistency.However, as a development tool, the TST profile serves its purpose as a baseline for development by individual coaches.The problem of reliability was partly ameliorated by the coaches working in pairs and sharing their ideas through social media.

Key to the effectiveness of TST is that it is an analytical tool that incorporates clear assessment criteria (Williams & Penney, 2011), with a graphical display that allows coaches to prioritize strong and weak competences within an overall profile.The study suggests that TST generated evidence with which to make valid assessment judgements, and served the purposes of verification of competence and reporting of standa ds of coaching.This integrated approach to assessment supported changes in three dimensions of coaching: the textualdocumentation, material, assessment disks; the perceptualcoaches pedagogical thoughts and assumptions; and the operationalcoaching interventions and strategies (Choi, 1992).

In relation to the contextual factors identified by Race (2014;2015) as being likely to lead to successful learning, the case study both reinforced these principles and demonstrated their application.It was evident that the student coaches had become convinced of the need for assessment criteria and for the identification of coaching competencies.The impact of learning by doing was clear, and the student coaches' active involvement in creating feedback on performance and using this to inform their practice facilitated and reinforced the need for selfappraisal.Perhaps the most interesting finding was the contribution that peer discussion made to learning.The opportunity to verbalize their learning helped student coaches to make sense of their evolving learning.Fullan's (1991) simple implementation model helped to structure the findings.The study focused on the first two stagesintroduction and operationalization.In the preparation stage, the need for developing appropriate skills and a period of familiarization was emphasized, and this was expressed clearly by the participants.The operationalisation stage helped to identify a number of implementation obstacles related to contextual ma ters.

This was an exploratory study, in which the first author was significantly involved as the designer of the assessment system and as the module tutor, and might reasonably be expected to drive the module delivery with some nthusiasm.Nevertheless, the student coaches' response was overwhelmingly positive, and the findings provide sufficient evidence of the need for continued refinement and application of technologically-enhanced assessment methods in education and development.The application of new technologies in education seems inevitable.This study demonstrates that an appropriate perf rmance profile that can generate user-friendly information about coaching performance is a viable option.The coaches' responses were unequivocal that their coaching performance had been impacted po itively by the use of TST.The most likely explanation for this is that the identification of coaching competences and performance criteria enabled and encouraged the coaches to evaluate coaching interventions and encouraged an element of reflection about their coaching behaviours.The continuous collation of assessment data and the maintenance of performance records were clear advantages.

The TST profiling tool, and its use in assessment, created a valuable synergy between the need for assessment criteria and the reinforc ment that this had for the components of the coaching process at this stage of the coaches' development.Most significant was the provision of a language and concepts with which to discourse about assessment and c aching behaviour.The rapid proliferation of technology-supported coaching pedagogy and assessment has the potential to change the coach education landscape.It is difficult to imagine future coaching and coach education that is not supported by technology.It is essential, therefore, for those involved in coaching to adapt to the continually changing and evolving technological landscape.This study demonstrates that TST reinforced the components of the coaching process, highlight d coaching competences, proved to be a useful mechanism for rating performance, and was a basis for future development.The combination of video footage, touch screen technology, and identified criteria was an evident advantage.A pressing concern in coach ed cation and development is bridging the gap between education and practice.We conclude that the use of a technologically- nhanced assessment syst m such as TST made it more likely that assessment would impact practice.
ppendix A: Participant Training

All participants underwent exte sive training as a component element of their undergraduate module, Introduction to Coaching.A summary of the stages and content is as follows:

Stage 1: Clarifying performance outcomes to be mastered: defini g learning, using disks to identify significant learning outcomes, expre sing what it is that performers should learn using l nguage that is hel