Designing Knowledge Dissemination in a Digital Era – Analysing TED Talk’s Multimodal Orchestration

Jingxin Jiang; Fei Victor Lim

doi:10.21432/cjlt28263

Authors

Jingxin Jiang Chongqing Open University
Fei Victor Lim Nanyang Technological University

DOI:

https://doi.org/10.21432/cjlt28263

Keywords:

online learning, TED talks, speech, images, gestures, multimodal discourse analysis

Abstract

Online learning has gained increasing attention during the COVID-19 pandemic. Teachers face social exigencies to design ways of knowledge dissemination in online instruction. We posit that understanding how knowledge can be represented in successful online academic genres can inform teachers on how they can design students’ online learning experiences. This study examined how scientific knowledge is disseminated in one of the most widespread academic genres, TED Talks, which share discoursal similarities with other academic genres such as online lectures. This study adopted a systemic functional multimodal discourse analysis approach to explore how a presenter used speech, images, and gestures to disseminate knowledge. The analysis shows that a presenter orchestrates speech, images, and gestures strategically to clarify the scientific ideas and engage the audience. Based on understanding how the three semiotic modes are used to disseminate scientific knowledge in accessible and engaging ways, this paper discusses how insights on multimodal orchestration can function as a heuristic tool to inform design in online learning.

Author Biographies

Jingxin Jiang, Chongqing Open University

Jingxin Jiang is an Associate Professor at Chongqing Open University, China, and Ph.D. candidate at the National Institute of Education, Nanyang Technological University, Singapore. Her research interest is multimodal discourse analysis, multiliteracy, and online learning.

Fei Victor Lim, Nanyang Technological University

Fei Victor Lim is an Assistant Professor at the National Institute of Education, Nanyang Technological University, Singapore. He researches and teaches on multiliteracies, multimodal discourse analyses, and digital learning.

References

Adedoyin, O. B., & Soykan, E. (2020). Covid-19 pandemic and online learning: The challenges and opportunities. Interactive Learning Environments. Advance online publication. 1-13. https://doi.org/10.1080/10494820.2020.1813180

Ali, W. (2020). Online and remote learning in higher education institutes: A necessity in light of COVID-19 pandemic. Higher Education Studies, 10(3), 16-25. https://doi.org/10.5539/hes.v10n3p16

Alibali, M. W., Nathan, M. J., Wolfgram, M. S., Church, R. B., Jacobs, S. A., Johnson Martinez, C., & Knuth, E. J. (2014). How teachers link ideas in mathematics instruction using speech and gesture: A corpus analysis. Cognition and Instruction, 32(1), 65-100. http://dx.doi.org/10.1080/07370008.2013.858161

Allo, M. D. G. (2020). Is the online learning good in the midst of Covid-19 pandemic? The case of EFL learners. Jurnal Sinestesia, 10(1), 1-10. https://sinestesia.pustaka.my.id/journal/article/view/24

Ally, M. (2008). Foundations of educational theory for online learning. In T. Anderson & F. Elloumi (Eds.), Theory and practice of online learning (pp. 3-31). Athabasca University Press.

Bannert, M. (2002). Managing cognitive load—Recent trends in cognitive load theory. Learning and Instruction, 12(1), 139-146. https://psycnet.apa.org/doi/10.1016/S0959-4752(01)00021-4

Bucchi, M., & Saracino, B. (2016). “Visual Science Literacy” images and public understanding of science in the digital age. Science Communication, 38(6), 812-819. https://doi.org/10.1177/1075547016677833

Chang, Y.-J., & Huang, H.-T. (2015). Exploring TED Talks as a pedagogical resource for oral presentations: A corpus-based move analysis. English Teaching & Learning, 39(4), 29-62. http://dx.doi.org/10.6330/ETL.2015.39.4.02

Chue, S., Lee, Y.-J., & Tan, K. C. D. (2015). Iconic gestures as undervalued representations during science teaching. Cogent Education, 2(1), Article 1021554. https://doi.org/10.1080/2331186X.2015.1021554

D’avanzo, S. (2015). Speaker identity vs. speaker diversity: The case of TED Talks corpus. In G. Balirano & M. C. Nisco (Eds.), Languaging diversity: Identities, genres, discourses (pp. 279-296). Cambridge Scholars Publishing.

Feng, D., & O’Halloran, K. L. (2013). The visual representation of metaphor: A social semiotic approach. Review of Cognitive Linguistics. Published under the auspices of the Spanish Cognitive Linguistics Association, 11(2), 320-335. https://doi.org/10.1075/rcl.11.2.07fen

Halliday, M. A. K. (1994). An introduction to functional grammar (2nd ed.). Edward Arnold.

Hamid, R., Sentryo, I., & Hasan, S. (2020). Online learning and its problems in the Covid-19 emergency period. Jurnal Prima Edukasia, 8(1), 86-95. https://doi.org/10.21831/jpe.v8i1.32165

Harrison, C. (2003). Visual social semiotics: Understanding how still images make meaning. Technical Communication, 50(1), 46-60.

Harrison, S. (2021). Showing as sense-making in oral presentations: The speech-gesture-slide interplay in TED Talks by Professor Brian Cox. Journal of English for Academic Purposes, Article 101002. https://doi.org/10.1016/j.jeap.2021.101002

Jewitt, C., Bezemer, J., & O’Halloran, K. (2016). Introducing multimodality. Routledge.

Kalantzis, M., & Cope, B. (2020). After the COVID-19 crisis: Why higher education may (and perhaps should) never be the same. Access: Contemporary Issues in Education, 40(1), 51-55. https://doi.org/10.46786/ac20.9496

Kress, G., & van Leeuwen, T. (2006). Reading images: The grammar of visual design (2nd ed.). Routledge.

Li, Y., Gao, Y., & Zhang, D. (2016). To speak like a TED speaker—A case study of TED motivated English public speaking study in EFL teaching. Higher Education Studies, 6(1), 53-59. https://doi.org/10.5539/HES.V6N1P53

Lim, F. V. (2019). Analysing the teachers’ use of gestures in the classroom: A systemic functional multimodal discourse analysis approach. Social Semiotics, 29(1), 83-111. https://doi.org/10.1080%2F10350330.2017.1412168

Lim, F. V. (2020). Designing learning with embodied teaching: Perspectives from multimodality. Routledge.

Lim, F. V. (2021). Investigating intersemiosis: A systemic functional multimodal discourse analysis of the relationship between language and gesture in classroom discourse. Visual Communication, 20(1), 34-58. https://doi.org/10.1177/1470357218820695

Lorés, R. (2020). Science on the web: The exploration of European research websites of energy-related projects as digital genres for the promotion of values. Discourse, Context & Media, 35, Article 100389. https://doi.org/10.1016/j.dcm.2020.100389

Maqableh, M., & Alia, M. (2021). Evaluation online learning of undergraduate students under lockdown amidst COVID-19 Pandemic: The online learning experience and students’ satisfaction. Children and Youth Services Review, 128, Article 106160. https://doi.org/10.1016/j.childyouth.2021.106160

Martin, B. A. (2021). Teachers perceptions of Google Classroom: Revealing urgency for teacher professional learning. Canadian Journal of Learning and Technology, 47(1). https://doi.org/10.21432/cjlt27873

Martinec, R. (2000). Construction of identity in Michael Jackson’s Jam. Social Semiotics, 10(3), 313-329. https://doi.org/10.1080/10350330050136370

Masi, S. (2016). Gestures in motion in TED Talks: Towards multimodal literacy. In V. Bonsignori & B. C. Camiciottoli (Eds.), Multimodality Across Communicative Settings, Discourse Domains and Genres (pp. 146-165). Cambridge Scholars Publishing.

Masi, S. (2020). Exploring meaning-making practices via co-speech gestures in TED Talks. Journal of Visual Literacy, 39(3-4), 201-219. https://doi.org/10.1080/1051144X.2020.1826223

Mattiello, E. (2019). A corpus-based analysis of scientific TED Talks: Explaining cancer-related topics to non-experts. Discourse, Context & Media, 28, 60-68. http://dx.doi.org/10.1016/j.dcm.2018.09.004

Mayer, R. E. (2002). Multimedia learning. Psychology of Learning and Motivation, 41, 85-139. https://doi.org/10.1016/S0079-7421(02)80005-6

Mayer, R. E. (2019). Thirty years of research on online learning. Applied Cognitive Psychology, 33(2), 152-159. https://doi.org/10.1002/acp.3482

Mayer, R. E., & Moreno, R. (2002). Aids to computer-based multimedia learning. Learning and Instruction, 12(1), 107-119. http://dx.doi.org/10.1016/S0959-4752(01)00018-4

Miller, T., MacLaren, K., & Xu, H. (2020). Online learning: Practices, perceptions, and technology. Canadian Journal of Learning and Technology/La revue canadienne de l’apprentissage et de la technologie, 46(1). https://doi.org/10.21432/cjlt27894

Niess, M. L. (2012). Teacher knowledge for teaching with technology: A TPACK lens. In R. N. Ronau, C. R. Rakes, & L. N. Margaret (Eds.), Educational technology, teacher knowledge, and classroom impact: A research handbook on frameworks and approaches (pp. 1-15). IGI Global. https://doi.org/10.4018/978-1-60960-750-0.ch001

Ogbonna, C. G., Ibezim, N. E., & Obi, C. A. (2019). Synchronous versus asynchronous e-learning in teaching word processing: An experimental approach. South African Journal of Education, 39(2), 1-15. https://doi.org/10.15700/saje.v39n2a1383

O’Halloran, K. L. (2008). Systemic functional-multimodal discourse analysis (SF-MDA): Constructing ideational meaning using language and visual imagery. Visual Communication, 7(4), 443-475. https://doi.org/10.1177/1470357208096210

Oyedotun, T. D. (2020). Sudden change of pedagogy in education driven by COVID-19: Perspectives and evaluation from a developing country. Research in Globalization, 2, Article 100029. https://doi.org/10.1016/j.resglo.2020.100029

Pei, L., & Wu, H. (2019). Does online learning work better than offline learning in undergraduate medical education? A systematic review and meta-analysis. Medical Education Online, 24(1), Article 1666538. https:// doi.org/10.1080/10872981.2019.1666538

Rowley-Jolivet, E. (2012). Oralising text slides in scientific conference presentations: A multimodal corpus analysis. In A. Boulton, S. Carter-Thomas, & E. Rowley-Jolivet (Eds.), Corpus-informed research and learning in ESP (pp. 135-166). John Benjamins.

Salem, A. A. (2019). A sage on a stage, to express and impress: TED Talks for improving oral presentation skills, vocabulary retention and its impact on reducing speaking anxiety in ESP settings. English Language Teaching, 12(6), 146-160. https://doi.org/10.5539/ELT.V12N6P146

Shalev-Shwartz, S. (2011). Online learning and online convex optimization. Foundations and Trends in Machine Learning, 4(2), 107-194. https://doi.org/10.1561/2200000018

Swales, J. (1990). Genre analysis: English in academic and research settings. Cambridge University Press.

Sweller, J. (2005). Implications of cognitive load theory for multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 19-30). Cambridge University Press. https://doi.org/10.1017/CBO9780511816819.003

Tanis, C. J. (2020). The seven principles of online learning: Feedback from faculty and alumni on its importance for teaching and learning. Research in Learning Technology, 28, Article 2319. https://doi.org/10.25304/rlt.v28.2319

Tardy, C. M. (2005). Expressions of disciplinarity and individuality in a multimodal genre. Computers and Composition, 22(3), 319-336. https://doi.org/10.1016/j.compcom.2005.05.004

Teo, C. L., Tan, S. C., & Chan, C. K. (2021). Pedagogical transformation and teacher learning for knowledge building: Turning COVID-19 challenges into opportunities. Canadian Journal of Learning and Technology, 47(4). https://doi.org/10.21432/cjlt28057

Towndrow, P. A., Nelson, M. E., & Yusuf, W. F. B. M. (2013). Squaring literacy assessment with multimodal design: An analytic case for semiotic awareness. Journal of Literacy Research, 45(4), 327-355. https://doi.org/10.1177/1086296X13504155

Valeiras-Jurado, J., Ruiz-Madrid, M. N., & Jacobs, G. (2018). Revisiting persuasion in oral academic and professional genres: Towards a methodological framework for multimodal discourse analysis of research dissemination talks. Ibérica: Revista de la Asociación Europea de Lenguas para Fines Específicos (AELFE), 35, 93-118.

Wingrove, P. (2017). How suitable are TED Talks for academic listening? Journal of English for Academic Purposes, 30, 79-95. https://doi.org/10.1016/j.jeap.2017.10.010

Zhang, Y., Machin, D., & Song, T. (2015). Visual forms of address in social media discourse: The case of a science communication website. Journal of Multicultural Discourses, 10(2), 236-252. http://dx.doi.org/10.1080/17447143.2015.1042384