Designing Knowledge Dissemination in a Digital Era – Analysing TED Talk’s Multimodal Orchestration


  • Jingxin Jiang Chongqing Open University
  • Fei Victor Lim Nanyang Technological University



online learning, TED talks, speech, images, gestures, multimodal discourse analysis


Online learning has gained increasing attention during the COVID-19 pandemic. Teachers face social exigencies to design ways of knowledge dissemination in online instruction. We posit that understanding how knowledge can be represented in successful online academic genres can inform teachers on how they can design students’ online learning experiences. This study examined how scientific knowledge is disseminated in one of the most widespread academic genres, TED Talks, which share discoursal similarities with other academic genres such as online lectures. This study adopted a systemic functional multimodal discourse analysis approach to explore how a presenter used speech, images, and gestures to disseminate knowledge. The analysis shows that a presenter orchestrates speech, images, and gestures strategically to clarify the scientific ideas and engage the audience. Based on understanding how the three semiotic modes are used to disseminate scientific knowledge in accessible and engaging ways, this paper discusses how insights on multimodal orchestration can function as a heuristic tool to inform design in online learning.

Author Biographies

Jingxin Jiang, Chongqing Open University

Jingxin Jiang is an Associate Professor at Chongqing Open University, China, and Ph.D. candidate at the National Institute of Education, Nanyang Technological University, Singapore. Her research interest is multimodal discourse analysis, multiliteracy, and online learning.

Fei Victor Lim, Nanyang Technological University

Fei Victor Lim is an Assistant Professor at the National Institute of Education, Nanyang Technological University, Singapore. He researches and teaches on multiliteracies, multimodal discourse analyses, and digital learning.


Adedoyin, O. B., & Soykan, E. (2020). Covid-19 pandemic and online learning: The challenges and opportunities. Interactive Learning Environments. Advance online publication. 1-13.

Ali, W. (2020). Online and remote learning in higher education institutes: A necessity in light of COVID-19 pandemic. Higher Education Studies, 10(3), 16-25.

Alibali, M. W., Nathan, M. J., Wolfgram, M. S., Church, R. B., Jacobs, S. A., Johnson Martinez, C., & Knuth, E. J. (2014). How teachers link ideas in mathematics instruction using speech and gesture: A corpus analysis. Cognition and Instruction, 32(1), 65-100.

Allo, M. D. G. (2020). Is the online learning good in the midst of Covid-19 pandemic? The case of EFL learners. Jurnal Sinestesia, 10(1), 1-10.

Ally, M. (2008). Foundations of educational theory for online learning. In T. Anderson & F. Elloumi (Eds.), Theory and practice of online learning (pp. 3-31). Athabasca University Press.

Bannert, M. (2002). Managing cognitive load—Recent trends in cognitive load theory. Learning and Instruction, 12(1), 139-146.

Bucchi, M., & Saracino, B. (2016). “Visual Science Literacy” images and public understanding of science in the digital age. Science Communication, 38(6), 812-819.

Chang, Y.-J., & Huang, H.-T. (2015). Exploring TED Talks as a pedagogical resource for oral presentations: A corpus-based move analysis. English Teaching & Learning, 39(4), 29-62.

Chue, S., Lee, Y.-J., & Tan, K. C. D. (2015). Iconic gestures as undervalued representations during science teaching. Cogent Education, 2(1), Article 1021554.

D’avanzo, S. (2015). Speaker identity vs. speaker diversity: The case of TED Talks corpus. In G. Balirano & M. C. Nisco (Eds.), Languaging diversity: Identities, genres, discourses (pp. 279-296). Cambridge Scholars Publishing.

Feng, D., & O’Halloran, K. L. (2013). The visual representation of metaphor: A social semiotic approach. Review of Cognitive Linguistics. Published under the auspices of the Spanish Cognitive Linguistics Association, 11(2), 320-335.

Halliday, M. A. K. (1994). An introduction to functional grammar (2nd ed.). Edward Arnold.

Hamid, R., Sentryo, I., & Hasan, S. (2020). Online learning and its problems in the Covid-19 emergency period. Jurnal Prima Edukasia, 8(1), 86-95.

Harrison, C. (2003). Visual social semiotics: Understanding how still images make meaning. Technical Communication, 50(1), 46-60.

Harrison, S. (2021). Showing as sense-making in oral presentations: The speech-gesture-slide interplay in TED Talks by Professor Brian Cox. Journal of English for Academic Purposes, Article 101002.

Jewitt, C., Bezemer, J., & O’Halloran, K. (2016). Introducing multimodality. Routledge.

Kalantzis, M., & Cope, B. (2020). After the COVID-19 crisis: Why higher education may (and perhaps should) never be the same. Access: Contemporary Issues in Education, 40(1), 51-55.

Kress, G., & van Leeuwen, T. (2006). Reading images: The grammar of visual design (2nd ed.). Routledge.

Li, Y., Gao, Y., & Zhang, D. (2016). To speak like a TED speaker—A case study of TED motivated English public speaking study in EFL teaching. Higher Education Studies, 6(1), 53-59.

Lim, F. V. (2019). Analysing the teachers’ use of gestures in the classroom: A systemic functional multimodal discourse analysis approach. Social Semiotics, 29(1), 83-111.

Lim, F. V. (2020). Designing learning with embodied teaching: Perspectives from multimodality. Routledge.

Lim, F. V. (2021). Investigating intersemiosis: A systemic functional multimodal discourse analysis of the relationship between language and gesture in classroom discourse. Visual Communication, 20(1), 34-58.

Lorés, R. (2020). Science on the web: The exploration of European research websites of energy-related projects as digital genres for the promotion of values. Discourse, Context & Media, 35, Article 100389.

Maqableh, M., & Alia, M. (2021). Evaluation online learning of undergraduate students under lockdown amidst COVID-19 Pandemic: The online learning experience and students’ satisfaction. Children and Youth Services Review, 128, Article 106160.

Martin, B. A. (2021). Teachers perceptions of Google Classroom: Revealing urgency for teacher professional learning. Canadian Journal of Learning and Technology, 47(1).

Martinec, R. (2000). Construction of identity in Michael Jackson’s Jam. Social Semiotics, 10(3), 313-329.

Masi, S. (2016). Gestures in motion in TED Talks: Towards multimodal literacy. In V. Bonsignori & B. C. Camiciottoli (Eds.), Multimodality Across Communicative Settings, Discourse Domains and Genres (pp. 146-165). Cambridge Scholars Publishing.

Masi, S. (2020). Exploring meaning-making practices via co-speech gestures in TED Talks. Journal of Visual Literacy, 39(3-4), 201-219.

Mattiello, E. (2019). A corpus-based analysis of scientific TED Talks: Explaining cancer-related topics to non-experts. Discourse, Context & Media, 28, 60-68.

Mayer, R. E. (2002). Multimedia learning. Psychology of Learning and Motivation, 41, 85-139.

Mayer, R. E. (2019). Thirty years of research on online learning. Applied Cognitive Psychology, 33(2), 152-159.

Mayer, R. E., & Moreno, R. (2002). Aids to computer-based multimedia learning. Learning and Instruction, 12(1), 107-119.

Miller, T., MacLaren, K., & Xu, H. (2020). Online learning: Practices, perceptions, and technology. Canadian Journal of Learning and Technology/La revue canadienne de l’apprentissage et de la technologie, 46(1).

Niess, M. L. (2012). Teacher knowledge for teaching with technology: A TPACK lens. In R. N. Ronau, C. R. Rakes, & L. N. Margaret (Eds.), Educational technology, teacher knowledge, and classroom impact: A research handbook on frameworks and approaches (pp. 1-15). IGI Global.

Ogbonna, C. G., Ibezim, N. E., & Obi, C. A. (2019). Synchronous versus asynchronous e-learning in teaching word processing: An experimental approach. South African Journal of Education, 39(2), 1-15.

O’Halloran, K. L. (2008). Systemic functional-multimodal discourse analysis (SF-MDA): Constructing ideational meaning using language and visual imagery. Visual Communication, 7(4), 443-475.

Oyedotun, T. D. (2020). Sudden change of pedagogy in education driven by COVID-19: Perspectives and evaluation from a developing country. Research in Globalization, 2, Article 100029.

Pei, L., & Wu, H. (2019). Does online learning work better than offline learning in undergraduate medical education? A systematic review and meta-analysis. Medical Education Online, 24(1), Article 1666538. https://

Rowley-Jolivet, E. (2012). Oralising text slides in scientific conference presentations: A multimodal corpus analysis. In A. Boulton, S. Carter-Thomas, & E. Rowley-Jolivet (Eds.), Corpus-informed research and learning in ESP (pp. 135-166). John Benjamins.

Salem, A. A. (2019). A sage on a stage, to express and impress: TED Talks for improving oral presentation skills, vocabulary retention and its impact on reducing speaking anxiety in ESP settings. English Language Teaching, 12(6), 146-160.

Shalev-Shwartz, S. (2011). Online learning and online convex optimization. Foundations and Trends in Machine Learning, 4(2), 107-194.

Swales, J. (1990). Genre analysis: English in academic and research settings. Cambridge University Press.

Sweller, J. (2005). Implications of cognitive load theory for multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 19-30). Cambridge University Press.

Tanis, C. J. (2020). The seven principles of online learning: Feedback from faculty and alumni on its importance for teaching and learning. Research in Learning Technology, 28, Article 2319.

Tardy, C. M. (2005). Expressions of disciplinarity and individuality in a multimodal genre. Computers and Composition, 22(3), 319-336.

Teo, C. L., Tan, S. C., & Chan, C. K. (2021). Pedagogical transformation and teacher learning for knowledge building: Turning COVID-19 challenges into opportunities. Canadian Journal of Learning and Technology, 47(4).

Towndrow, P. A., Nelson, M. E., & Yusuf, W. F. B. M. (2013). Squaring literacy assessment with multimodal design: An analytic case for semiotic awareness. Journal of Literacy Research, 45(4), 327-355.

Valeiras-Jurado, J., Ruiz-Madrid, M. N., & Jacobs, G. (2018). Revisiting persuasion in oral academic and professional genres: Towards a methodological framework for multimodal discourse analysis of research dissemination talks. Ibérica: Revista de la Asociación Europea de Lenguas para Fines Específicos (AELFE), 35, 93-118.

Wingrove, P. (2017). How suitable are TED Talks for academic listening? Journal of English for Academic Purposes, 30, 79-95.

Zhang, Y., Machin, D., & Song, T. (2015). Visual forms of address in social media discourse: The case of a science communication website. Journal of Multicultural Discourses, 10(2), 236-252.