Assessing the Teacher: In-class Surveys

This past month, I helped two staff members with their in-class surveys.  The first staff member used the standard Stop-Start-Continue and a tailored Ticket Out The Door feedback tools.  Both are examples of 'One Minute Papers', versions of which are used from Glasgow to Massachusetts.  The second staff member used a tailored tool based on the UCL Department of Philosophy lecture evaluation form.

The first staff member, 'Sam', delivered the Stop-Start-Continue form in April and was very unhappy with their initial readings of the results.  Sam understood that some data collection anomalies were present due to a mixed delivery method.  However, the teacher noted a high level of destructive criticism in the feedback they received, and this became the point of focus.  Understanding the need for an objective interpretation, I assisted Sam by conducting a content analysis of the data.  From this analysis, I concluded that the vast majority of the feedback was actually positive.  Of the positive feedback, the majority complimented Sam both personally and as a teacher.

Upon reflection, I noted that some structural issues with the Stop-Start-Continue tool.  In particular, the question order - where a Likert Scale was offered first - did not promote thoughtful responses to qualitative questions.  Further, questions that solicited complaint (Stop) and improvement (Continue) responses drew the same data.  Removal of the complaint question offered the added benefit of allowing student respondents the opportunity to understand evaluation as a constructive (versus destructive) tool.

The second staff member, 'Ashley', forwarded me a proposed tool for comment.  We had been in discussion for some time about the different feedback methods available within our institution, both formal and informal.  Ashley teaches in a practice-based discipline.  The evaluation tool under consideration was common in the practice of the discipline, but not in the teaching of it.  After reviewing the proposed tool, and reflecting on the Stop-Start-Continue weaknesses, I offered a tailored version of the UCL lecture evaluation form.

I retained the attendance question in the tailored version to help Ashley understand how much the respondents’ answers were based on their actual experience of lectures.  Tailoring also allowed an opportunity to move the Likert scales down in the question order.  As the UCL form only uses a 4-point scale, I added in a neutral response to match the scale offered in Ashley's original form.  Although there could be same debate about the validity of a neutral response in teaching evaluation, the existence of the point in disciplinary practice suggested it is likely to reflect the context of the survey more accurately.  Deleting the complaint question, I replaced it with an example question from the UCL evaluation form, and provided an opportunity for Ashley to swap it for another.  Ashley thanked me for tool and I heard nothing until after the teacher had delivered it and analysed the data.

The results for Ashley were very good, highlighting both personal and teaching strengths.  The recommendations for improvement were diverse and interesting.  I intend to recommend this tailored approach again.