Today, Temple released student evaluations from last semester. I taught two courses last semester--an upper-level undergraduate seminar on social theory and a graduate seminar on classical social theory. Reading evaluations can be tricky. The self-critical eye tends to be drawn towards particularly damning comments. The self-praising eye tends to be drawn towards comments that are especially kind. Either can lead to a distorted image of what actually happened in the classroom from the students' perspectives.The multiple choice items provide a better sense of the mean, but even this quantitative data can be misleading. My school places one of three letters next to the scores, indicating whether the number is in the upper levels (U), middle levels (M), or lower level (L). The assignment is based on the percentages in "strongly agree" (U), "disagree," and "strongly disagree" (L). But a professor who receives "agree" from 100% of his students on any particular question will receive an M, even though that sounds like a better than average situation.
The school also provides comparative averages for the department, the college, the university, and the course level.
Before reading evaluations, you need to know what you want to learn from them. An instructor who has no plans to change his course probably shouldn't read evaluations, but might need to in order to make a case that he deserves tenure or a raise.
In my case, I'm looking for two broad purposes. First, I want the feedback for my own pedagogical reasons. I'd like to know what's working in the classroom and what I can do to improve. The evaluations are just one source of data on this. My own assessment of student work is the primary source of performance information. For instance, if students say they hate the course paper, but I can see that they are writing smart papers that will benefit them down the road, then I limit how much I pay attention to that feedback. That said, I do want to hear from them about what may not be working so that I can tinker with the course and make it better.
Second, I want to use the evaluations to build a narrative about my teaching. That narrative should highlight what I am doing well and it should also document that I work successfully to improve my teaching. Since I teach the same courses from year to year, it's nice to be able to say I learned a lesson in an earlier year than paid off with a new successful teaching method in a later year.
So, I try to highlight the following items:
1. What worked well. I want to keep these aspects of the course since students report that they work. Here, I am looking for the overall pattern, not the one-off comments.
2. Major success stories. This is more than just things that generally worked. These are the quotes that are exceedingly positive, or the good comments that are made repeatedly.
3. What didn't work. These are the aspects I want to consider adjusting for the future. That doesn't mean I need to scrap them altogether, it just means I want to pay attention to them. If a reading is widely criticized, I might replace it, or I might try to give a better introduction to it. As with item 1 above, I'm looking for the overall pattern.
4. Useful suggestions. These might come from just one student, but if it's a good idea, it's worth considering.
You might argue that if I'm looking for major success stories, I should also look for major failure stories (individual students who write mean angry comments). But, to state the obvious, I apply for raises and tenure, not for demotions. If someone wants to make a case against me, it's their job to find the failure stories.
As an example, I'll review one set of evals from last semester. This course only had 7 students in the end (an 8th student had withdrawn). This was my graduate seminar in classical social theory. The number in parentheses indicates how many students made the comment.
1. What worked well?
- Applying the readings in group assignments (3)
- Lectures on key concepts and backgrounds of the theorists (4)
- Journals (4)
- Selected Topic Paper (1)
- Capitalism Paper (2)
- Papers, in general (2)
- Emphasis on making arguments (3)
- Overall structure (2)
- Focus on student research areas (1)
- Feedback on writing (1)
- Grading style (1)
- Tocqueville (1)
- Gilman (1)
- Du Bois (1)
- 6 students highlighted my sensitivity to diversity.
- "I really thought the course was valuable, but of course time consuming."
- Though most of my scores are in the middle range (M), all are 4.1 or higher on a 5 point scale.
- Two scores were in the upper range (U): 1) increased by ability to analyze and evaluate, and 2) learned a great deal in this course.
- Reading journals due on the same day as papers (reading journals were due every week, papers were additionally due on some of those weeks) (3)
- Small group assignments (1)
- Tocqueville paper (1)
- Applying concepts in group assignments (2)
- The readings (1)
4. Suggestions
- Skip journals the week a paper is due (3)
- Add Du Bois's Souls of Black Folk, in addition to The Philadelphia Negro (1)
- Give biographical info the week before the reading (1)
- More applications (1)
- Option of less detailed journals (1)
- Don't remove Gilman reading (1)
- One less paper (1)
- Remove Tocqueville paper (1)
Lots of classroom components were listed as working well but the ones listed most often include journals, lectures, the emphasis on making arguments, and group discussions/applications. It's worth noting that the small group work was also mentioned twice as something that did not work well. However, in my experience group work is often disliked by some students, even as it proves invaluable to others. I'm inclined, then, to keep using it, even though feelings about it are divided.
Additionally, I note that journals are valuable but there were a few comments about the journals being due the same day as papers. The suggestion is to skip journals that day. But if journals are valuable, why would I drop them on a regular basis? That would mean those readings are read less closely and the students would not have useful summaries for those authors that they could use later. Instead, I'm inclined to shift the due dates of papers to later in the week. Since papers are submitted electronically, I can collect them anytime. This is something I will raise with my current graduate students in class this week.
The other items that didn't work well were only mentioned once. Components of the course that are only mentioned once under worked well or didn't work well I treat as neutral items. They work fine, but I could also replace them easily as I work to improve the class.
I could have ended the quote under 'success stories' before the part about the course being time consuming, but I find that my department and college like indications that my classes are demanding. Students who attempt to complain about the demands of my classes are unwittingly making a good case for me to the administration.
Graduate courses have been a challenge for me since I started at Temple. These are actually pretty good evaluations so I will use them to build a narrative of gradual improvement in my graduate teaching.
I will blog soon about my undergrad evals. Because of the high enrollment, that will take more analysis.
Comments
Post a Comment