Faculty often have a lot to complain about when it comes to course evaluations. A significant body of literature suggests that course evaluations tell us little, as instructors, about the actual effect a course has had in achieving its goal (e.g. imparting a subject to a group of students).  However, one of the social problems that, I think, we have with course evaluations is that it’s not a dialogue or conversation; it’s a set of one-sided statements that come, all at once, around the time final grades are posted. The comments can be reflective, not of the value or rigor of the course, but rather the emotional state of the student in anticipating of the final grade they have earned in the course. They can be vacuously enthusiastic or bitter. They are often treated by the institution as definitive, without opportunity for faculty to respond to them.

I wish to turn this problem part-way around and make it, not a dialogue, but at least a response. I think it’s important to respond to student comments, especially the ones that are critical of the course in specific ways (e.g. due to another influence that soured them on the course, due to experiences with grading, etc). In speaking back to the commenters, anonymous though they may be, I hope to also provide something more resembling the “closed loop” of a real dialogue. At the very least, I can achieve one cycle of “call and response”.

General Comments on my course

Here are some basic statistics for my course:

• The semester began with 106 enrolled students.
• By final exam time, enrollment was 93 students.
• I don’t curve grades in my course. I do offer students a chance to do more work to show they’ve mastered material, but no automatic curving. I believe in an absolute scale of mastery, not one relative to peers in the course.
• The mean grade in my course was 84%, the median grade 88%. The corresponds to a B (B+) for the mean (median).
  • The grade distribution is asymmetric around either the mean or the median, with a long tail down to the low side of the distribution.

On a scale of 0-4, a mean of 84% corresponds to 3.36.

General Overview of Evaluations

At SMU, students are asked to rate the course based on 10 questions. An 11th question allows them to self-report how much time they are spending on the course outside of scheduled class time. In addition, they are asked to provide written feedback to 2 questions. When data are provided to faculty, they come in the form of numerical reports on each of the first 11 questions, with comparisons to the average of the College and the Department (not for that course, but across all courses… which makes no distinction between the challenges of a captive audience for upper-level majors courses and a reluctant and, usually, younger audience taking required introductory courses).

The 11 numerical questions are as follows:

1. The syllabus clearly explained the goals for learning, grading policy, and the schedule.
2. Class time was well-organized.
3. Course materials supported my learning of the course content.
4. Examples and/or particular readings used during class time helped me understand the course content.
5. Assignments including readings, videos, and problem sets, helped clarify my understanding of the course content.
6. Feedback on assignments improved my understanding of the course content.
7. My performance in the class was clearly communicated to me throughout the semester.
8. My interest in the subject increased as a result of taking this course.
9. My interest in the subject increased as a result of taking this course.
10. The instructor was available to answer questions outside of class.
11. How many hours per week did you spend on this course outside of class time?

The 2 written-feedback questions are as follows:

1. Did any particular aspects of this course enhance your learning?
2. Did any particular aspects of this course detract from your learning?

Let’s begin by looking at the feedback on the quantitative questions.

Quantitative Questions

On the first 10 questions, students are asked to select from the following answers, attached to which are numerical weights (I call them “points” below):

• Strongly Agree (with the statement): 4 points
• Agree: 3 points
• Disagree: 2 points
• Strongly Disagree: 1 points
• Not applicable: 0 points

Over 50% of the enrolled students at end-of-term answered the numerical questions (50/93 students).

Encouragingly, on a general level, the course scored well in the spring. In all 10 instructor-evaluation questions, the course received a 3.35 or above. By the measures the University provides, this is considered above average; but I’m not sure those measures are meaningful, given that they lump together courses at all levels. I noted that the mean of score on the question that received the lowest scores was approximately equal to the mean grade of students enrolled in my course at the end of the semester. This is not inconsistent with observations in the literature that course evaluations correlate with student GPA in the course. I should note that 3.35 was the lowest score I received; all others were above that (on the 9 other course assessment questions).

Drilling down into this a bit more, the lowest instructor assessment was on the assignment feedback question (question 6). I’ll return to that when we get to the qualitative assessments, since some students called this out in writing.

Some interesting numbers from the instructor evaluations:

• 88% felt that class time was well-organized (agree or strongly agree)
• 87% of respondents felt that class-time materials supported course learning, which is heartening as the flipped-classroom model explicitly is intended to make class time useful in specific ways that the “sage on the stage” model is not good at doing.
• 8% of students felt that their performance in the course was not communicated clearly during the semester… which, of course, is odd, because as soon as Exam 1 was concluded and a sufficient fraction (about 18-20%) of their grade was available, all of their grades were reported to them on Canvas with frequent updates, especially after each exam). They always had access to their grades after Exam 1 (1 month into the course).
• 2% felt that I wasn’t available outside of class to answer questions, which again is odd, since I had about 3 hours of open office hours every week plus I would gladly schedule individual meetings with students if they provided a written request. So, clearly, there was some miscommunication there… despite the fact that I reminded them in class constantly of my availability.

Student time outside of class

Regarding question 11, the student time self-assessment, respondents on average reported that they spent about 6-9 hours outside of class time doing work for the course – reading, videos, homework, and other studying. This is precisely in the target range I set for the course.

A 3 credit-hour course requires 3 contact hours every week, plus a minimum of 2 hours of work outside of class for every contact hour. Therefore, students are expected to spend at least 6 hours outside of class working on the course. I estimate for my students at the outset (I state it on day one of the course) that they can expect 6-9 hours per week outside of class. (Side note: this is why it’s insane that students enroll in 18 or more credit-hours in a semester – they are usually setting themselves up, or being set up by the University, for failure).

35% of respondents reported spending 3-6 hours, 41% reported 6-9 hours, and 14% reported 9-12 hours; therefore, 90% of students reported that they worked 3-12 hours outside of class. That all sounds perfectly normal to me for an introductory-level physics class. 8% reported 12+ hours, but that is not a surprise; it was clear that there were students (and about 8 students seems right) that had to work much harder then their peers to keep up with the material. I’ll respond more to that point later.

From the quantitative perspective, things were heartening. Especially on the time outside of class, that was a target I was concerned about meeting on the first pass through teaching this class.

Qualitative Questions

Let’s take a look at the qualitative questions. I want to focus on the first question, and then use the second question as a chance to respond as if fielding anonymous questions or comments (which, effectively, I am).

Of the 93 students enrolled in the class at the end of the course, only 29 provided written feedback on the first question (Did any particular aspects of this course enhance your learning?). Here are the things that were mentioned frequently as supporting learning:

• Flipped-classroom model in general: about 10 students explicitly noted this model, or a phrase locked to the model (e.g. in-class problems, demonstrations, out-of-class videos, etc.) as a strong positive for their learning.
• Lecture videos: about the same number of respondents highlighted the videos, and some explicitly expressed the fact that they could self-pace on those videos (which is precisely the point).
• Interests or passion of the instructor: it’s embarrassing to say it, but it’s important for other instructors to take note of this, especially young instructors struggling to find their footing in teaching: communicate your passion for the subject to the students. More than half of the written responses used words like “passion,” “cares,” “enthusiastic,” etc. in positive ways to describe what enhanced learning in the class.

That said, there were some veiled criticisms embedded in here. For instance, one student wished I would lecture more in-class because they felt I was good at it and that it would help keep the attention of other students more (I’ll respond to this and a related issue below). One or two students said that they learned to “tolerate pain” more as a result of this course… indeed, I had a similar experience in my introductory physics classes and, to be quite honest, that is the point: to teach you that college operates at a different level than high school.

Let’s turn our attention to the second qualitative question: did any particular aspects of the course detract from learning? Almost the same number of students responded to this question (28 instead of 29), but with a key difference I will mention below. The most frequent responses were:

• Nothing detracted from learning: 7 of the 28 respondents replied to this question that nothing detracted from their learning, so really there was no need to state that outright. That leaves 21 comments about something(s) that actually detracted from learning.
• Flipped classroom: 5 respondents expressed that they did not like the model, or significant aspects related to it. Of those, most wished I had lectured more in class (it was sort of veiled praise in the form of criticism).
• How much time they had to spend out of class: several students complained about the time they had to spend outside of class.
• The homework was more difficult than what was expected on the tests: I’ll answer that directly below. A few students commented on this.
• A few students reported getting bored in class because the in-class examples were too easy; again, I’ll address that directly in a moment.

Of the students in either question category:

• Respondents favored the flipped classroom model by at least 2:1 for those who mentioned it by name; if you include students who commented on things that are core to the flipped classroom model, it was really more like a preference of 3:1 or better for the flipped-classroom model (or videos outside of class, for in-class problem solving, etc.)

Question and Response

Let me now respond directly to questions or comments written out in the qualitative sections.

I am not a fan of the “flipped classroom” system. I spent a lot of time outside of class watching lecture videos that I would have preferred to have during class, and then had to sit through classes where I had finished the practice problem presented and my classmates would spend several more minutes working. Given that on top of lectures, homework is done out of class, the 80 minute class period seems unproductive and like an unnecessary waste of time. While I really enjoyed having Professor Sekula, and admire his passion for physics, I would encourage him to explore a more productive classroom model that does not result in students having to spend several hours out of class doing what should have been accomplished in class.

Thanks for taking the time to write so much feedback about the course. First, let me begin by addressing your first specific complaint: that you had to spend a lot of time outside of class watching lecture videos (I’ll return to the point about lectures in-class in a moment).

I made it extremely clear on day one of the course (c.f. Slide 23 of the slides on the introduction to the course) that you would be expected to spend 6-9 hours of time each week outside of our class time together. I wasn’t kidding about that. Each lecture video or video assignment (e.g. part of a video) was about 1 hour in length; some more, some less. The reading takes time, too. I estimated about 3 hours would be spent on these activities, especially because I urged you to take notes during the reading and video that could then be used to aid in taking the in-class quizzes (and thus prepping for class time in general).

The goal was to expose you to these ideas, have you learn to better and better highlight in your notes the key take-aways (which promotes concept filtering and synthesis), and then begin to exercise those ideas in class (further reinforcing the ideas). The rest of the time (3-6 hours) was to be spent on the 10-15 homework questions per week (some homeworks had more problems because you had more than a week to complete them).

I provided an example time-management strategy on the first day of class. I hope you utilized it. It provided a rhythm intended to keep you just ahead of the crest of the “work wave” you have to ride in any introductory class. I recall no professor of mine in college who provided such a detailed strategy for time management. Of course, you are free to ignore my advice, but I hope at least the reason I wrote all this down is clear: you are expected, at minimum by the federal definition of the credit-hour and at maximum by the obligations implied by enrolling in an institution of higher learning, to commit to working extremely hard to achieve your goals. My class is no different than any other;  time management and organization are your friends.

I am sorry you felt that the class period was “unproductive” and a “waste of time.” I noted that most of your peers disagree with that position, and while teaching is not a popularity contest the responses and scores this class received are in-line with the predictions of physics education research: learning is overwhelmingly efficiently and effectively achieved by providing an interactive learning environment, with emphasis on exercising ideas and concepts, rather than a “sage on the stage” approach where I talk and you write down what I say. This is tied to your last comment – that I should explore other models for the course. In fact, for this course I have pursued what is considered one of the best models for teaching physics.

I suspect that your problem was not so much with my course model, but fundamentally tied to what you said above: you finished the practice problems fast and got bored. I have two comments on this.

First, thanks: next semester,  I will provide a stack of “challenge problems” that the students who advance more quickly can access. They will be much harder than the baseline class-time problems and will provide a struggle to students who feel they already have a grasp of the subject. This will deepen learning and probe the depths of your expertise.

Second, let me note the following: you always had the option to raise your hand, flag me down in class (recall I floated around the classroom, going where I was needed), and tell me you were bored. I would have immediately found something more challenging for you to do.

For instance, I could have invented a problem for you to work on on-the-spot. I might not have known the answer off the top of my head, but I would have worked it out later and we could have compared solutions. I always have a reserve of ideas for problems bouncing around in my skull. You need only have asked to be challenged, and your request would have been honored.

A much better use of your skills, however, would have been to serve your peers as a mentor. If you were finishing problems so fast, it would have taken no effort to look around and see that other students were struggling or lost or confused. The point of class time is partly to encourage you to work together, to make each other better and stronger. You could have volunteered to help other students around you.

In fact, you should have approached me about serving as an unofficial peer mentor in each class. I would have encouraged you to sit in different seats each class period and help students who needed help. If you are bored, channel that into action: those who truly understand something can communicate it to others, and a good show of your belief in your skill would have been to serve your fellow students in that capacity.

Action items I will take for next semester:

• CHALLENGE PROBLEMS: I will provide a stack of more challenging, more open-ended problems for students such as yourself to select from in each class and spend the time working on them if you are bored with the baseline problems.
• PEER MENTORING: I will make it clear at the outset of the course that students who feel they understand the material have an obligation, in the structure of the course, to help their peers. I will encourage students who feel the way you felt (after the first exam) to identify themselves to me and design a strategy for them to rotate around the room and go where they are needed. I hope this will result in a “Mentoring Corps” wherein the best and brightest in my classes learn to serve others with their knowledge.

[There was a] huge amount of homework and useless lecture. (and we have to purchase the stupid website to do the homework)

I am sorry you felt the lecture was useless; I assume here you are either referring to the videos, which you did not enjoy, or the use of class time for engaging in problem solving. It’s a little unclear from your comment what you mean. Let me address both.

First, class time was not viewed as useless by most of your peers. The physics education research, as I mentioned in my response to the last comment, is clear on the subject and consistent with the quantitative observations of my own course. Students who are encouraged to struggle, but have access to the professor and peer mentors while doing so, master the material far better than students who are told to simply copy down what I say during class time.

Second, if you meant instead that the lecture videos are useless, then I have additional comments. Of course, I can’t expect everyone to like the videos; I am sorry they didn’t work for you. That said, I challenge you on the claim that they are useless. I wish my instructors in introductory physics, who only did “sage on the stage” lectures, told me about the connections to other sciences, daily life, and especially some of the historical context of the material in my intro physics classes. But, as I said, I can’t be all things to all people. Caveat emptor.

I also challenge you on the homework point. I had far more homework to do each week in my own intro classes in the 1990s, and physics has not grown simpler since then. At the time. I recall feeling that the volume of that work was unnecessary. As a teacher now, when I select homework problems, I do so under the following strategy:

• Try to keep it to no more than about 12 problems a week (on average)
• Of those problems:
  • about 10-15% should be conceptual to assess baseline absorption of concepts
  • 20% should be very basic problems to make sure you have absorbed the most foundational definitions and can apply them
  • 20-25% should be very hard problems intended to really challenge you to think, sometimes even about what the question is asking (that is more reflective of problem solving in real life)
  • the rest should be at a level where I expect a student to wrestle, but no too much (a kind of “intermediate” level).

The goal of the homework is to make sure you are building a mastery of how to understand the question and then answer the question; to combine thinking and mathematics to solve problems and answer questions; to get a sense of accomplishment, the sense that you are beginning to understand (and perhaps even master) the subject. The exams test the latter assumption, especially because you can work together on homework but the exams are each a solo flight. Of course, I don’t expect all students to be able to answer all questions perfectly; that is just not realistic. That is why homework is only worth 15% of your final grade (equivalent to just 1 of the in-class exams). You are expected to try and fail on homework.

Finally, while we can have a separate conversation about academic publishing as an industry, you were not just purchasing a website. Wiley provides you:

1. An electronic copy of the textbook, searchable and interactive – I don’t even have a physical copy of the textbook; I use this to do all my reading for the course.
2. A homework distribution, collection, and grading system for numerical correctness.
3. A whole bank of extra problems and test questions, so you can find problems to help you study.
4. Interactive learning content independent of what I provide, including interactive problem solving demonstrations and other such material.

The price tag is something you can complain about, but it’s not inconsistent with the cost of books for college courses. If there is something the university needs to do to help you afford these things, you should reach out. Don’t suffer silently. You should at least ask me or your advisor if you have concerns about the costs of things.

The homeworks are online and significantly harder than anything we do in class or on the test, which is not necessarily a problem. The problem is how he also has us turn in our scratch paper work for the homework and he grades that separately from the homework itself with the single shittiest grading scale I have ever seen. He picks a “random” (the hardest) question on the hw and grades the work for only that question, and most of the points are for neatness. as someone who is not neat, I am consistently getting 3 out of 10 points for homework methods that I did correctly. Also, the grader often gives little to no feedback on returned work.

I am sorry you are unhappy with the fact that your written work is assessed. However, this part of the course was also made clear on the first day of class and I hope you spoke to me about this during the semester.

Let me begin by saying simply this: the quality of your written work will be forever judged; accept this truth and make peace with it. I still have all my papers assessed by third parties and I always make at least some mistakes; I am grateful for review of my written work. Admitting your fallibility is the first step toward embracing the scientific method. It’s this method that helps assure that humanity makes progress.

The grader was instructed to pick a problem of their choosing to grade. Sometimes it was the hardest one, but often it was not. I didn’t always agree with which problem the grader chose, but I trust my teaching assistants so long as they keep me in the loop. I encouraged students, like you, to speak to me if they had questions about the grading. Many did. Here, for the record, was the rubric for assessing the quality of your written work:

1. [4 Points] Applied the physical and/or mathematical concepts required to solve the problem in an accurate way, with no mistakes in the writing of formulas, use of formulas, use of approximations or assumptions, and application of general concepts. (Take off 1 point for each unique deficiency in this category, and leave positive feedback on the student’s homework indicating how to improve)
2. [5 Points] Clearly wrote out the steps required to solve the problem, indicating how to go from step to step in a way that was legible and methodical. (Take off 1 point for each unique deficiency in this category, and leave positive feedback on the student’s homework indicating how to improve)
3. [1 Points] Followed the guidelines of the written assignment policy on boxing (or circling) final answers, writing out the answers with their appropriate units, putting the assignment name (e.g. “Homework 1”) at the top of the page (or at least the first page of the assignment), putting their name, and putting the date on the assignment. (Take off 0.2 points for each unique deficiency in this category, and leave positive feedback on the student’s homework indicating how to improve)

I will leave it to you, dear reader, to decide if this is the “single shittiest grading scale” you have ever seen. There is no perfect scale, and based on my teaching assistant’s feedback I’ll be adjusting it for the coming year. We can always haggle about what things are worth what points, or about how this could be more finely designed to capture specific elements. There is no perfect assessment system that is all things to all people.

However, I think you’ll find that other science and engineering instructors are generally (and sadly) more vague about what they expect, and I would have hoped that you would react to the expectations of the rubric and improved your performance accordingly. The rubric is intended to show you what is important in a written solution: it must be clear, with sufficient steps to demonstrate the thought process (after all, we cannot read your mind… only your words and equations), and have all the elements of good-quality communication. To wish for less than this in a science class would be to hold science to a lower standard than the humanities, which doesn’t make sense; after all, science is supposed to be the most effective way to establish reliable knowledge. How can we know if your knowledge is reliable unless you communicate your process effectively?

Put yourself in my shoes. Would you, as an instructor in a class where students are supposed to master the basic elements of a field, accept anything less than the above? Would you, perhaps, even demand more? And, taking a breath and evaluating your own work for a moment: would you have accepted your own work quality as showing mastery of the subject?

I provided at least one example of a good written solution on the course website. Did you compare your work to it? How did you match up? Would you recommend your work as a “gold standard” of solution writing? If so, I’d love to use it.

Getting a correct answer is not enough in science. The method matters (a broken clock is right twice daily), and most importantly: you have to communicate it. Yes, I expect you to communicate clearly. Why did you not type your solutions, if your handwriting is not clear? That, and other approaches, are always an option. I make it clear in the guidelines I hand out at the beginning of the semester. There are many strategies to adapt to situations where much is asked of you. Next time, speak to your instructor about finding a strategy that works best for you.

The grader could always give more feedback, and that is something I work with my teaching assistants to improve over the course of the semester. There is a reason I ask them to pick one problem to assess on process each week; with 10-15 problems per week, and 100 students, they cannot be expected to provide feedback on absolutely everything. Even choosing only 1 problem per week, they have to decide how much they write for each student. I would hope that the quality of my own solutions to the in-class problems would provide a strong indicator of the quality I expect from my students; while I cannot expect you to meet that standard, I can expect you to shoot for it, fall short, and nonetheless succeed. What kind of teacher would I be if I didn’t set high standards?

Let me address a final comment from another student.

Physics 1 and 2 AP credit should be valid to skip the course. I made a 5 on the physics 1 AP test and made high A’s and 100’s on every test because of my previous knowledge. I learned very little besides what I already knew. If I was new to physics I would have learned a mountain of knowledge though!

I am glad you succeeded in the course. Sounds like you had a good teacher in high school. Your gripe, however, sounds to me to be more with SMU’s registrar than with me, the humble instructor of an introductory physics class. Why were you not able to have your AP course get you out of at least first-semester physics? SMU generally allows that. If they did not, then you should have taken that up with the University. You might also have reached out to the department chair and challenged the need to take the course; a test might have been offered (e.g. a previous final exam) to assess your claims.

That said, AP Physics is not the same as college physics. AP courses are taken by a large fraction (about 40%) of high school students, which hardly suggests they are advanced or distinctive at all. And, universities make a clear distinction between “AP Courses” and “College-Level Courses”; for instance, there are high schools that offer AP classes and also classes that are the equivalent of university-level introductory chemistry, physics, etc. Universities will accept the college-level course as a substitution for the equivalent intro-level university course; this is not always so for AP courses.

They are not the same course, and for a reason. AP courses put a strong emphasis on memorization and regurgitation of information; college courses are intended to force you to emphasize process and methodology, the first preparations for independent critical thinking. Some high schools have even begun to drop AP courses since they don’t confer clear advantages anymore for college-bound students.

Since it’s clear there will always be students who feel they needn’t be in my course, I have as a priority the initiatives mentioned above – challenge problems and peer mentoring. A mind is a terrible thing to waste; nothing will demonstrate more to me that you have actually mastered the material than to have to either get out of your comfort zone on it or, even better, mentor someone else to the same level of comfort as you.

One last comment. There is a fallacy implicit in your whole statement: that one masters a subject the first time one sees it. This is demonstrably false, and not just from my own experience. Certainly, there are exceptional individuals; perhaps you are one of them. But in general, seeing a subject twice makes you better at it. You see things you didn’t see the first time. You see things in a new light when its framed in a different way by a new instructor. You should embrace repetition. After all: repetition is the essence of learning. You did not learn to walk on your first step. I promise you: you have not actually mastered the physics needed to be a mechanical or electrical engineer, or even a medical doctor.

Last Thoughts

The spring 2018 introductory physics course, overall, went really well. In-person student feedback was generally positive, and where it was critical adjustments were made (e.g. in the assessment of the communication quality of homework). Based on the end-of-course evaluations, there are still some useful adjustments that can be made. But I’ll note that these evaluations are consistent with what I have seen in the past for introductory physics classes, and if anything were less critical than what I have seen in the past. There is no clear reason to abandon the flipped classroom model, and if anything the feedback strengthens my own resolve the continue working within this educational model to promote physics learning.

Most students, maybe 97-99% or so, who take intro physics are taking it for an engineering program or pre-health program requirement. Very few plan to major in physics. Some students discover a love of physics after taking an introductory physics course, and I hope to encourage them to take more. Why not? After all, physics is a fundamental science that deals with the nature of energy, matter, space and time: all that there is in the cosmos. Most students just need the foundation in physics to continue on in their program of choice; that said, I’d be remiss in my duties as a teacher to let them walk away thinking they’ve learned physics when they have not. I am pleased with how this first pass at teaching this course went, and I look forward to many more such passes.