Major increases in teachers’ performance evaluations: Evidence from student evaluation of teaching surveys
Abstract
Purpose: This exploratory study examined major increases in teachers’ performance evaluations and their immediate impact on next year’s score for those instructors that taught the same subject for at least two years in a row. The purpose was twofold. Firstly, to identify those Student Evaluation of Teaching (SET) survey items associated with major increases in teacher evaluations. Secondly, to examine if there is evidence of the use of these SET results by instructors to improve their teaching.
Design: The sample comprised SET survey ratings from one university over a five consecutive year period, for a total sample of 13,052 teacher evaluations and 3,893 teachers-subject observations under analysis. Frequency tables and Student’s t-test were used for analysis.
Findings: The results highlighted the three SET survey items captured by the dimension of teaching methodology as those most closely related to major increases in teacher evaluations. Regarding the second objective, the results show no generalised response from teachers who experience major increases in SET ratings. This suggests that the use of SET results is either limited or does not have an immediate measurable effect on student satisfaction.
Originality/Value: To the best of our knowledge, this was the first study to specifically examine major increases in teachers’ performance evaluations and their immediate impact on next year’s score based on evidence from SET surveys.
Received: 22 December 2021
Accepted: 26 February 2023
Downloads
References
Arthur, Linet. “From performativity to professionalism: Lecturer’s responses to student feedback.” Teaching in Higher Education 14, (2009): 441-454. https://doi.org/10.1080/13562510903050228.
Avery, Rosemary J., Keith Bryant, Alan Mathios, Hyojin Kang, and Duncan Bell. “Electronic course evaluations: does an online delivery system influence student evaluations?” The Journal of Economic Education 36, no. 1 (2006): 21-37. https://doi.org/10.3200/JECE.37.1.21-37.
Bacci, Silvia. “Longitudinal data: Different approaches in the context of item- response theory models.” Journal of Applied Statistics 39, no. 9 (2012): 2047-2065. https://doi.org/10.1080/02664763.2012.700451.
Ballantyne, Roy, Jill Borthwick, and Jan Packer. “Beyond student evaluation of teaching: Identifying and addressing academic staff development needs.” Assessment & Evaluation in Higher Education 25, no. 3 (2000): 221-236. https://doi.org/10.1080/713611430.
Bedggood, Rowan E., and Jerome D. Donovan. “University performance evaluations: what are we really measuring?” Studies in Higher Education 37, no. 7 (2012): 825-842. https://doi.org/10.1080/03075079.2010.549221.
Beran, Tanya. N., and Jeniffer L. Rokosh. “Instructor’s perspectives on the utility of student ratings of instruction”. Instructional Science 37, (2009): 171-184. https://doi.org/10.1007/s11251-007-9045-2.
Bolivar, Senior. “Student teaching evaluations: Options and concerns.” Journal of Construction Education 5, no. 1 (2000): 20-29.
British Educational Research Association (BERA). Ethical Guidelines for Educational Research. (London: BERA, 2018).
Burden, Peter. “Does the end of semester evaluation forms represent teacher’s views of teaching in a tertiary education context in Japan?” Teaching and Teacher Education 24, no. 6 (2008): 1463-1475. https://doi.org/10.1016/j.tate.2007.11.012.
Burden, Peter. “Creating confusion or creative evaluation? The use of student evaluation of teaching surveys in Japanese tertiary education.” Educational Assessment, Evaluation and Accountability 22, (2010): 97-117. https://doi.org/10.1007/s11092-010-9093-z.
Capa-Aydin, Yesim. “Student evaluation of instruction: comparison between in-class and online methods.” Assessment & Evaluation in Higher Education 41, no. 1 (2016): 112-126. https://doi.org/10.1080/02602938.2014.987106.
Chan, Cecilia. K. Y., Lillian Y. Y. Luk, and Min Zeng. “Teachers’ perceptions of student evaluations of teaching.” Educational Research and Evaluation 20, no. 4 (2014): 275-289. https://doi.org/10.1080/13803611.2014.932698.
Cheng, Jacqueline. H. S., and Herbert W. Marsh. “UK National Student Survey: Are differences between universities and courses reliable and meaningful.” Oxford Review of Education 36, no. 6 (2010): 693-712. https://doi.org/10.1080/03054985.2010.491179.
Cladera, Magdalena. “Let’s ask our students what really matters to them.” Journal of Applied Research in Higher Education 13, no. 1 (2021): 112-125. https://doi.org/10.1108/JARHE-07-2019-0195.
Clayson, Dennis. E. “Student evaluations of teaching: are they related to what students learn? A meta-analysis and review of the literature.” Journal of Marketing Education 31, no. 1 (2009): 16-30. https://doi.org/10.1177/0273475308324086.
Cone, Catherine., Velliyur Viswesh, Vasudha Gupta, and Elizabeth Unni. “Motivators, barriers, and strategies to improve response rate to student evaluation of teaching.” Currents in Pharmacy Teaching and Learning 10, no. 12 (2018): 1543-1549. https://doi.org/10.1016/j.cptl.2018.08.020.
Eurydice. “Teaching and learning in Primary Education. European Commission, Education Information Network in Europe.” 2020. https://eacea.ec.europa.eu/national-policies/eurydice/content/teaching-and-learning-primary-education-42_en.
Fan, Y., L. J. Shepherd, E. Slavich, D. Waters, M. Stone, M., R. Abel, and E. L. Johnston. “Gender and cultural bias in student evaluations: Why representation matters.” PLoS ONE 14, no. 2 (2019): e0209749. https://doi.org/10.1371/journal.pone.0209749.
Hendry, Graham D., Patricia M. Lyon, and Cheryl Henderson-Smart. “Teachers’ approaches to teaching and responses to student evaluation in a problem-based medical program”. Assessment and Evaluation in Higher Education 32, no. 2 (2007): 143-157. https://doi.org/10.1080/02602930600801894.
Johnson, Rachel. “The authority of the student evaluation questionnaire.” Teaching in Higher Education 5, no. 4 (2000): 419-434. https://doi.org/10.1080/713699176.
Jones, Sandra. “Measuring the quality of higher education: Linking teaching quality measures at the delivery level to administrative measures at the university level.” Quality in Higher Education 9, no. 3 (2003): 223-229. https://doi.org/10.1080/1353832032000151094.
Kulik, James A. “Student ratings: Validity, utility and controversy”. New Directions for Institutional Research 27, (2002): 9-25. http://dx.doi.org/10.1002/ir.1.
Marsh, Herbert W. “Students’ evaluations of university teaching: Dimensionality, reliability, validity, potential biases and usefulness.” In The scholarship of teaching and learning in higher education: An evidence-based perspective, ed. R. P. Perry and J. C. Smart (New York: Springer, 2007a), 319-383.
Marsh, Herbert W. “Do university teachers become more effective with experience? A multilevel growth model of students’ evaluations of teaching over 13 years.” Journal of Educational Psychology 99, no. 4 (2007b): 775-790. https://doi.org/10.1037/0022-0663.99.4.775.
Marsh, Herbert W., and L. A. Roche. “Effects of grading leniency and low workload on students’ evaluations of teaching: Popular myth, bias, validity, or innocent bystanders?” Journal of Educational Psychology 92, no. 1 (2000): 202–228. https://doi.org/10.1037/0022-0663.92.1.202.
Moreno-Murcia, Juan Antonio, Yolanda Silveira, and Noelia Belando. “Questionnaire Evaluating Teaching Competencies in the University Environment. Evaluation of Teaching Competencies in the University.” Journal of New Approaches in Educational Research 4, no. 1, (2015), 54-61. https://doi.org/10.7821/naer.2015.1.106.
Nasser, Fadia, and Barbara Fresko. “Faculty views of student evaluation of college teaching.” Assessment & Evaluation in Higher Education 27, no. 2 (2002): 187-198. https://doi.org/10.1080/02602930220128751.
Ory, John C. “Faculty thoughts and concerns about student ratings.” New Directions for Teaching and Learning 87, (2001): 3-15. http://dx.doi.org/10.1002/tl.23.
Roche, Lawrence A., and Herbert W. Marsh. “Multiple dimensions of university teacher self-concept.” Instructional Science 28, (2000): 439-468. http://dx.doi.org/10.1023/A:1026576404113.
Spooren, Pieter, Bert Brockx, and Dimitri Mortelmans. “On the validity of student evaluation of teaching: The state of the art.” Review of Educational Research 83, no. 4 (2013): 598-642. https://doi.org/10.3102/0034654313496870.
Stein, Sarah. J., Dorothy Spiller, Stuart Terry, Trudy Harris, Lynley Deaker, and Jo Kennedy. Unlocking the impact of tertiary teachers’ perceptions of student evaluation of teaching Wellington, New Zealand: Ako Aotearoa National Centre for Tertiary Teaching Excellence, 2012.
Sulis, Isabella, Mariano Porcu., and Vincenza Capursi. “On the Use of Student Evaluation of Teaching: A Longitudinal Analysis Combining Measurement Issues and Implications of the Exercise.” Social Indicators Research 142, (2019): 1305-1331. https://doi.org/10.1007/s11205-018-1946-8.
Surgenor, P. W. G. “Obstacles and opportunities: Addressing the growing pains of summative student evaluation of teaching.” Assessment & Evaluation in Higher Education 38, (2013): 363-376. https://doi.org/10.1080/02602938.2011.635247.
Tran, Thi Thu Trang, and Truong Xuan Do. “Student evaluation of teaching: do teacher age, seniority, gender, and qualification matter?” Educational Studies, (2020). https://doi.org/10.1080/03055698.2020.1771545.
Tucker, Beatrice, Sue Jones, Leon Straker, and Joan Cole. “Course evaluation on the web: Facilitating student and teacher reflection to improve learning.” New Directions for Teaching and Learning 96, (2003): 81-94. http://dx.doi.org/10.1002/tl.125.
Winchester, Tiffany. M., and Maxwell Winchester. “A longitudinal investigation of the impact of faculty reflective practices on students’ evaluations of teaching.” British Journal of Educational Technology 45, no. 1 (2014): 11-124. https://doi.org/10.1111/bjet.12019.
Yao, Yuankun, and Marilyn Grady. “How do faculty make formative use of student evaluation feedback? A multiple case study.” Journal of Personnel Evaluation in Education 18, (2005): 107-126. https://doi.org/10.1007/s11092-006-9000-9.
Zabaleta, Francisco. “The use and misuse of student evaluation of teaching.” Teaching in Higher Education 12, no. 1 (2007): 55-76. https://doi.org/10.1080/13562510601102131.
Zhao, Jing, and Dorinda J. Gallant. “Student evaluation of instruction in higher education: Exploring issues of validity and reliability.” Assessment & Evaluation in Higher Education 37, no. 2 (2012): 227-235. https://doi.org/10.1080/02602938.2010.523819.
Authors are required to sign and submit a copyright transfer agreement after acceptance but before publication of their manuscript. To that effect, they receive, from the Managing Editor of Tuning Journal for Higher Education, a standard copyright assignment form designed along the following lines:
1. Authorship:
The author who signs the copyright transfer agreement must be the sole creator of the work or legally acting on behalf of and with the full agreement of all the contributing authors.
2. Copyright and Code of conduct:
a) Authors warrant that their work is original; has not been previously copyrighted or published in any form; is not under consideration for publication elsewhere; its submission and publication do not violate TJHE Ethical Guidelines for Publication and any codes (of conduct), privacy and confidentiality agreements, laws or any rights of any third party; and no publication payment by the Publisher (University of Deusto) is required.
b) Authors are solely liable for the consequences that may arise from third parties’ complaints about the submitted manuscript and its publication in Tuning Journal for Higher Education (TJHE).
c) Authors grant to the Publisher the worldwide, sub-licensable, and royalty-free right to exploit the work in all forms and media of expression, now known or developed in the future, for educational and scholarly purposes.
d) Authors retain the right to archive, present, display, distribute, develop, and republish their work (publisher's version) to progress their scientific career provided the original publication source (Tuning Journal) is acknowledged properly and in a way that does not suggest the Publisher endorses them or their use of the wortk.
e) Authors warrant that no permissions or licences of any kind will be granted that might infringe the rights granted to the Publisher.
3. Users:
Tuning Journal for Higher Education is an Open Access publication. Its content is free for full and immediate access, reading, search, download, distribution and reuse in any medium or format only for non-commercial purposes and in compliance with any applicable copyright legislation, without prior permission from the Publisher or the author(s). In any case, proper acknowledgement of the original publication source must be made and any changes to the original work must be indicated clearly and in a manner that does not suggest the author’s and or Publisher’s endorsement whatsoever. Any other use of its content in any medium or format, now known or developed in the future, requires prior written permission of the copyright holder.
1.jpg)
1.jpg)
.jpg)
1.jpg)
.jpg)
.jpg)
