Examining the Emotional Tone of Student Evaluations of Teaching

Derek Newman

doi:10.21432/cjlt28695

Authors

Derek Newman Cambrian College https://orcid.org/0000-0001-9146-4991

DOI:

https://doi.org/10.21432/cjlt28695

Keywords:

academia, emotional tone, student evaluation of teaching, student evaluations

Abstract

Student-written evaluation (N = 600) of professors was examined to determine the emotional tone of the words used to evaluate faculty. Using the revised Dictionary of Affect (DOA; Whissell, 2009), evaluation words (N = 26,764) uploaded to the Rate My Professors website between 2018 and October of 2023 were measured for their pleasantness, activation, and imagery. Overall, the emotional tone of the students’ written evaluation was very close to the DOA’s definition of everyday English (M = 50) for all three categories: pleasantness (M = 51.1, SD = 6.3), activation (M = 52.2, SD = 4.8), and imagery (M = 50.2, SD = 7.4). The results indicated that the written evaluations were uniform in expression and emotional tone: neither very pleasant/unpleasant, active/passive, or imagery/abstract.
While significant relationships were found with professor quality and difficulty ratings, the number of words in the evaluation, and the instructor’s gender, all associations had small correlational strengths and weak effect sizes, indicating that the variables might not be strong predictors of the emotional tone of student evaluations. If student written evaluations are not emotionally charged, then there is an opportunity to reduce any negative feelings faculty members have attached to the process.

Author Biography

Derek Newman, Cambrian College

Derek Newman, PhD, is a Psychology Professor in the Schools of Community Services, Interdisciplinary Studies & Public Safety (CIPS) at Cambrian College in Canada. His primary research activities focus on studying or learning strategies. Email: derek.newman@cambriancollege.ca ORCID: https://orcid.org/0000-0001-9146-4991

References

Abd-Elrahman, A., Andreu, M., & Abbott, T. (2010). Using text data mining techniques for understanding free-style question answers in course evaluation forms. Research in Higher Education Journal, 9, 1.

Afonso, N. M., Cardozo, L. J., Mascarenhas, O. A. J., Aranha, A. N. F., & Shah, C. (2005). Are anonymous evaluations a better assessment of faculty teaching performance? A comparative analysis of open and anonymous evaluation processes. Family Medicine, 37(1), 43–47.

Algozzine, B., Gretes, J., Flowers, C., Howley, L., Beattie, J., Spooner, F., Mohanty, G., & Bray, M. (2004). Student evaluation of college teaching: A practice in search of principles. College Teaching, 52(4), 134–141. https://doi.org/10.3200/CTCH.52.4.134-141

Arthur, L. (2009). From performativity to professionalism: Lecturers’ responses to student feedback. Teaching in Higher Education, 14(4), 441–454. https://doi.org/10.1080/13562510903050228

Balam, E. M., & Shannon, D. M. (2010). Student ratings of college teaching: A comparison of faculty and their students. Assessment & Evaluation in Higher Education, 35(2), 209–221. https://doi.org/10.1080/02602930902795901

Barnes, L. L. B., & Barnes, M. W. (1993). Academic discipline and generalizability of student evaluations of instruction. Research in Higher Education, 34, 135–149. https://doi.org/10.1007/BF00992160

Blackburn, R. T., & Clark, M. J. (1975). An assessment of faculty performance: Some correlates between administrator, colleague, student and self-ratings. Sociology of Education, 48(2), 242–256. https://doi.org/10.2307/2112478

Brown, M. J., Baillie, M., & Fraser, S. (2009). Rating RateMyProfessors.com: A comparison of online and official student evaluations of teaching. College Teaching, 57(2), 89–92. https://doi.org/10.3200/CTCH.57.2.89-92

Burdsal, C. A., & Bardo, J. W. (1986). Measuring student’s perceptions of teaching: Dimensions of evaluation. Educational and Psychological Measurement, 46(1), 63–79. https://doi.org/10.1177/0013164486461006

Carmack, H. J., & LeFebvre, L. E. (2019). “Walking on eggshells”: Traversing the emotional and meaning making processes surrounding hurtful course evaluations. Communication Education, 68(3), 350–370. https://doi.org/10.1080/03634523.2019.1608366

Clayson, D. E. (2018). Student evaluation of teaching and matters of reliability. Assessment & Evaluation in Higher Education, 43(4), 666–681. https://doi.org/10.1080/02602938.2017.1393495

Colardarci, T., & Kornfield, I. (2007). RateMyProfessors com versus formal in-class student evaluations of teaching. Practical Assessment, Research and Evaluation, 44(12), 1–15. https://doi.org/10.7275/26ke-yz55

Cook, S., Watson, D., & Webb, R. (2024). Performance evaluation in teaching: Dissecting student evaluations in higher education. Studies in Educational Evaluation, 81, 101342. https://doi.org/10.1016/j.stueduc.2024.101342

Cooper, A. K., & Coetzee, S. (2020). On the ethics of using publicly-available data. In M. Hattingh, M. Matthee, H. Smuts, I. Pappas, Y. K. Dwivedi, & M. Mäntymäki (Eds)., Responsible design, implementation and use of information and communication technology: I3E 2020. Lecture notes in computer science, 12067 (pp. 159–171). Springer International Publishing. https://doi.org/10.1007/978-3-030-45002-1_14

Dahal, K., & Rafiq, R. I. (2023, May). What makes a good course and professor: Through the lens of RateMyProfessor website. In Proceedings of the 2023 7th International Conference on Information System and Data Mining (ICISDM) (pp. 1–9). https://doi.org/10.1145/3603765.3603767

Dodou, D., & de Winter, J. C. F. (2014). Social desirability is the same in offline, online, and paper surveys: A meta-analysis. Computers in Human Behavior, 36, 487–495. https://doi.org/10.1016/j.chb.2014.04.005

Ellett, C. D., Loup, K. S., Culross, R. R., McMullen, J. H., & Rugutt, J. K. (1997). Assessing enhancement of learning, personal learning environment, and student efficacy: Alternatives to traditional faculty evaluation in higher education. Journal of Personnel Evaluation in Education, 11, 167–192. https://doi.org/10.1023/A:1007989320210

Feldman, K. A. (1989). Instructional effectiveness of college teachers as judged by teachers themselves, current and former students, colleagues, administrators, and external (neutral) observers. Research in Higher Education, 30, 137–194. https://doi.org/10.1007/BF00992716

Felton, J., Mitchell, J., & Stinson, M. (2004). Web-based student evaluations of professors: The relations between perceived quality, easiness and sexiness. Assessment & Evaluation in Higher Education, 29(1), 91–108. https://doi.org/10.1080/0260293032000158180

Foster, A. L. (2003). Picking apart Pick-A-Prof. Chronicle of Higher Education, 49(26), A33–A34.

Howell, A. J., & Symbaluk, D. G. (2001). Published student ratings of instruction: Revealing and reconciling the views of students and faculty. Journal of Educational Psychology, 93(4), 790–796. https://psycnet.apa.org/doi/10.1037/0022-0663.93.4.790

Janss, R., Rispens, S., Segers, M., & Jehn, K. A. (2012). What is happening under the surface? Power, conflict and the performance of medical teams. Medical Education, 46(9), 838–849. https://doi.org/10.1111/j.1365-2923.2012.04322.x

Kember, D., Leung, D. Y. P., & Kwan, K. P. (2002). Does the use of student feedback questionnaires improve the overall quality of teaching? Assessment & Evaluation in Higher Education, 27(5), 411–425. https://doi.org/10.1080/0260293022000009294

Kulik, J. A. (2001). Student ratings: Validity, utility, and controversy. New Directions for Institutional Research, 2001(109), 9–25. https://doi.org/10.1002/ir.1

Legg, A. M., & Wilson, J. H. (2012). RateMyProfessors.com offers biased evaluations. Assessment & Evaluation in Higher Education, 37(1), 89–97. https://doi.org/10.1080/02602938.2010.507299

Lutovac, S., Kaasila, R., Komulainen, J., & Maikkola, M. (2017). University lecturers’ emotional responses to and coping with student feedback: A Finnish case study. European Journal of Psychology of Education, 32, 235–250. https://doi.org/10.1007/s10212-016-0301-1

MacNell, L., Driscoll, A., & Hunt, A. N. (2015). What’s in a name: Exposing gender bias in student ratings of teaching. Innovative Higher Education, 40, 291–303. https://doi.org/10.1007/s10755-014-9313-4

Marsh, H. W., & Roche, L. (1993). The use of students’ evaluations and an individually structured intervention to enhance university teaching effectiveness. American Educational Research Journal, 30(1), 217–251. https://doi.org/10.3102/00028312030001217

Murray, K. B., & Zdravkovic, S. (2016). Does MTV really do a good job of evaluating professors? An empirical test of the internet site RateMyProfessors.com. Journal of Education for Business, 91(3), 138–147. https://doi.org/10.1080/08832323.2016.1140115

Olvet, D. M., Willey, J. M., Bird, J. B., Rabin, J. M., Pearlman, R. E., & Brenner, J. (2021). Third year medical students impersonalize and hedge when providing negative upward feedback to clinical faculty. Medical Teacher, 43(6), 700–708. https://doi.org/10.1080/0142159X.2021.1892619

Otto, J., Sanford Jr, D. A., & Ross, D. N. (2008). Does ratemyprofessor.com really rate my professor? Assessment & Evaluation in Higher Education, 33(4), 355–368. https://doi.org/10.1080/02602930701293405

Overall, J. U., & Marsh, H. W. (1980). Students’ evaluations of instruction: A longitudinal study of their stability. Journal of Educational Psychology, 72(3), 321–325. https://psycnet.apa.org/doi/10.1037/0022-0663.72.3.321

Penny, A. R. (2003). Changing the agenda for research into students’ views about university teaching: Four shortcomings of SRT research. Teaching in Higher Education, 8(3), 399–411. https://doi.org/10.1080/13562510309396

Quansah, F., Cobbinah, A., Asamoah-Gyimah, K., & Hagan Jr., J. E. (2024, February). Validity of student evaluation of teaching in higher education: A systematic review. Frontiers in Education, 9, 1–12. https://doi.org/10.3389/feduc.2024.1329734

Reid, L. D. (2010). The role of perceived race and gender in the evaluation of college teaching on RateMyProfessors.com. Journal of Diversity in Higher Education, 3(3), 137–152. https://psycnet.apa.org/doi/10.1037/a0019865

Robins, L., Smith, S., Kost, A., Combs, H., Kritek, P. A., & Klein, E. J. (2020). Faculty perceptions of formative feedback from medical students. Teaching and Learning in Medicine, 32(2), 168–175. https://doi.org/10.1080/10401334.2019.1657869

Shah, M., & Pabel, A. (2020). Making the student voice count: Using qualitative student feedback to enhance the student experience. Journal of Applied Research in Higher Education, 12(2), 194–209. https://doi.org/10.1108/JARHE-02-2019-0030

Shevlin, M., Banyard, P., Davies, M., & Griffiths, M. (2000). The validity of student evaluation of teaching in higher education: Love me, love my lectures? Assessment & Evaluation in Higher Education, 25(4), 397–405. https://doi.org/10.1080/713611436

Sidwell, D., Lee, D., Zimmerman, P.-A., Bentley, S., & Barton, M. (2025). Teaching faculty experiences with student evaluation of instruction: A mixed-methods study. Teaching and Learning in Nursing, 20(1), e276-e284. https://doi.org/10.1016/j.teln.2024.11.009

Silva, K. M., Silva, F. J., Quinn, M. A., Draper, J. N., Cover, K. R., & Munoff, A. A. (2008). Rate my professor: Online evaluations of psychology instructors. Teaching of Psychology, 35(2), 71–80. https://doi.org/10.1080/00986280801978434

Sonntag, M. E., Bassett, J. R., & Snyder, T. (2009). An empirical test of the validity of student evaluations of teaching made on RateMyProfessors.com. Assessment & Evaluation in Higher Education, 34(5), 499–504. https://doi.org/10.1080/02602930802079463

Spooren, P., Brockx, B., & Mortelmans, D. (2013). On the validity of student evaluation of teaching: The state of the art. Review of Educational Research, 83(4), 598–642. https://doi.org/10.3102/0034654313496870

Stroebe, W. (2020). Student evaluations of teaching encourages poor teaching and contributes to grade inflation: A theoretical and empirical analysis. Basic and Applied Social Psychology, 42(4), 276–294. https://doi.org/10.1080/01973533.2020.1756817

Stronge, J. H., Ward, T. J., & Grant, L. W. (2011). What makes good teachers good? A cross-case analysis of the connection between teacher effectiveness and student achievement. Journal of Teacher Education, 62(4), 339–355.

Stupans, I., McGuren, T., & Babey, A. M. (2016). Student evaluation of teaching: A study exploring student rating instrument free-form text comments. Innovative Higher Education, 41, 33–42. https://doi.org/10.1007/s10755-015-9328-5

Timmerman, T. (2008). On the validity of RateMyProfessors.com. Journal of Education for Business, 84(1), 55–61. https://doi.org/10.3200/JOEB.84.1.55-61

Uttl, B. (2021). Lessons learned from research on student evaluation of teaching in higher education. Student Feedback on Teaching in Schools: Using Student Perceptions for the Development of Teaching and Teachers, 13, 237–256. https://doi.org/10.1007/978-3-030-75150-0_15

Uttl, B., White, C. A., & Gonzalez, D. W. (2017). Meta-analysis of faculty’s teaching effectiveness: Student evaluation of teaching ratings and student learning are not related. Studies in Educational Evaluation, 54, 22–42. https://doi.org/10.1016/j.stueduc.2016.08.007

Wagenaar, T. C. (1995). Student evaluation of teaching: Some cautions and suggestions. Teaching Sociology, 23(1), 64–68. https://doi.org/10.2307/1319382

Wang, H., Burić, I., Chang, M., & Gross, J. J. (2023). Teachers’ emotion regulation and related environmental, personal, instructional, and well-being factors: A meta-analysis. Social Psychology of Education, 26, 1651–1696. https://doi.org/10.1007/s11218-023-09810-1

Whissell, C. (1996). Traditional and emotional stylometric analysis of the songs of Beatles Paul McCartney and John Lennon. Computers and the Humanities, 30, 257–265. https://doi.org/10.1007/BF00055109

Whissell, C. (2009). Using the Revised Dictionary of Affect in Language to quantify the emotional undertones of samples of natural language. Psychological Reports, 105(2), 509–521. https://doi.org/10.2466/PR0.105.2.509-521

Whissell, C. (2012a, January). Emotional consistency as evidence of dynamic equivalence among English translations of the Bible. Comprehensive Psychology, 1. https://doi.org/10.2466/28.49.CP.1.15

Whissell, C. (2012b). The emotionality and complexity of public political language in Canada’s Question Period. English Language and Literature Studies, 2(4), 68–76. http://dx.doi.org/10.5539/ells.v2n4p68

White, E., Whissell, C., & Dewson, M. (1989). An objective quantification of the affective tone of language in children’s television programing. Journal of Social Behavior & Personality, 4(1), 127–131.

Wright, S. L., & Jenkins-Guarnieri, M. A. (2012). Student evaluations of teaching: Combining the meta-analyses and demonstrating further evidence for effective use. Assessment & Evaluation in Higher Education, 37(6), 683–699. https://doi.org/10.1080/02602938.2011.563279

Yan, Z., & Carless, D. (2022). Self-assessment is about more than self: The enabling role of feedback literacy. Assessment & Evaluation in Higher Education, 47(7), 1116–1128. https://doi.org/10.1080/02602938.2021.2001431

Zhao, J., & Gallant, D. J. (2012). Student evaluation of instruction in higher education: Exploring issues of validity and reliability. Assessment & Evaluation in Higher Education, 37(2), 227–235. https://doi.org/10.1080/02602938.2010.523819