a quantitative analysis of a teaching experiment

I decided to try an experiment in my class this semester. Since some of my students aced both midterms and since others bombed one of their midterms (not all of whom deserved their low grades) I decided to offer a choice to my students. They could either:

1. get a “do over” on their lowest midterm score and replace it with their final exam score, thus increasing the weight of the final to 50% of their grade (even if they do worse on the final),
2. not take the final exam and receive a final grade with certainty based on their other grades,
3. take the final as is (with the final accounting for 30% of their grade).

The goal was twofold. First, I wanted to give each student the grade that they deserved. Some students had a low midterm score (there were many legitimate medical emergencies this semester, partly due to swine flu). Second, I wanted the students to actually use the material we learned this semester to make a tough decision under uncertainty.

I’m not sure I achieved my goals, but here are a few observations:

• 28/44 decided not to take the final exam.
• Of course, all of the A students opted out of the final, but a few others opted out as well.  8/11 B students, 3/6 C students, and 1/4 D students also opted out of the final. I didn’t like this, but these students took a systems perspective and opted for more time to study for their other finals (these letter grades are based on the grades going into the final exam).
• 14/16 students taking the final replaced a midterm score (option #1 above). One asked about replacing both midterms with the final (with the final then counting for 70% of the final grade), which I allowed and offered to the rest of the class.  In the end, the others were deterred by the possible risk of this choice.
• Two students took the final as originally planned. These students were typically close to the a grade cutoff and did not have much to gain from the first two policies, so they chose the status quo.
• Two students asked me for “favors” to change their grade. Of course, this would not be fair to the other students. However, when I made changes to the grading policy, it somehow gave the students permission to come up with their own grading schemes that, of course, benefitted themselves to a great degree.
• All the time I saved from grading finals was more than made up for with extra discussions with students and extra bookkeeping.  But I enjoyed how open students became about talking about their final grade with me before the final.  I started a dialog where students were encouraged to talk about their grade concerns.  I often have trouble getting the students at the bottom of the curve to come in and talk to me before the semester ends.  This really did the trick.
• Did this experiment actually work?  Of course, I can never know how the students who opted out of the final would have done.  I computed the grades of those who took the final and compared them to their grades if they chose another one of the options.  Of the 16 students taking the final, five ended up with a better letter grade than if they had opted out of the final (one of whom improved by two letter grades).  Of the fourteen students that replaced a midterm score, four ended up with a better letter grade had they opted to choose the status quo (and none did worse).  One of the two students who selected the status quo, however, would have improved by a letter grade if they had opted to replace a midterm score.
• The most amazing thing is how responsible the students became. I have never seen such conscientiousness in a large group of undergraduates.  By offering a choice to the students, it put the ball in their court.  They rose to the challenge and accepted the amount of control they exerted over their own grades.  I heard a lot more accountability for grades in the language that they used (“If I score at least an 82 on the final, I can get a B…” rather than “Would you give me a B if I scored an 82…?”).

If I do something like this again, I will likely make some changes in the offer that I make to the students.  And although I wish all of my C and D students took the final, I was happy with the vast majority of the student decisions as well as my interactions with them.  The students who needed to take the final really thought about making a tough decision under uncertainty.  That was what I was trying to teach them in the first place.

6 responses to “a quantitative analysis of a teaching experiment”

• Marcelo Augusto

Hi, Professora McLay!

Interesting experiment!

I remember the way I dealt with OR during my undergrad years — I call it “The Old School Approach”: I went to the library; took Luenberger’s book; studied in advance; and was successful after the semester. That’s it!

My OR professor was one of those persons that think everything may be solvable using just one method. In her case, it was gradient-based methods. One day, we gave her a function to be minimized inside a given constraint. She said it would be as a piece of a cake to solve it through gradient technique. In the next class, she said she had solved the problem, but didn’t let us see the solution.

There was just one problem: The function we gave her was not differentiable! 🙂

See You!

Marcelo

P.S: The word “professora” is the female version of the word “professor” (in Portuguese).

• Jon

I wished I had that luxury in one of my upcoming classes. Aced two midterms and failed one horribly so I would obviously take option #1. Thanks for the interesting stats Laura

• Francisco Marco-Serrano

When I was a student a professor did something similar, but he gave the rules at the beginning of the term. He gave us the opportunity to choose between doing a presentation+paper+exam, paper+exam, or just the exam; in each group he would be marking an A+ to the best student, grading down the rest on the group. I have to admit as we had time to answer and I noticed the best students decided to go for the first and second options I acted estrategically and chose to go only for final exam. By the way, this is how I got my A+. ; )

• Paul

I’m curious, could you state what you would change in a future version of your experiment? I’d assume it would have something to do with helping to motivate C and D students.

I’m partly curious for curiosity’s sake, and partly in case I want to use your uncertainty experiment in the future 🙂

Great post!

• Laura

I haven’t decided what I will do next. I would like to build my decision into the syllabus prior to the semester starting, but I am not that ready to be bold. In the past, I counted each student’s best exam score for more than the worst exam score (and this was in the syllabus). I like that approach.

If I did it again this semester, I would have replaced the final score with the lowest midterm score. In retrospect, giving the students the choice to opt out of the final was pretty generous, so I would lessen the possible reward to encourage more to improve their final grade. I have no reservations about entirely replacing a midterm. It was a nice benefit and a Get Out of Jail Free card for any, but the final exams were highly correlated with the midterm scores.

I am thinking about how to restructure my syllabuses for my courses this coming semester. A course in stochastic processes should have a stochastic grading scheme, right?

• Philip

I think most of us students are a bit too risk averse for stochastic grades with a high variance, but there should be at least some random component, I would think!