When you grade exams in a large class you inevitably face the misunderstood question dilemma. A student has given a correct answer to a question but not the question you asked. As an answer to the question you asked it is flat out wrong. How much credit should you give?
It should not be zero. You can make this argument at two levels. First, ex post, the student’s answer reveals some understanding. To award zero points would be to equate this with writing nothing at all. That’s unfair.
You might respond by saying, tough luck, it is my policy not to reward misunderstanding the question. But even ex ante it is optimal to commit to a policy which gives at least partial credit to fortuitous misunderstanding. The only additional constraint at the ex ante stage is incentive compatibility. You don’t want to reward a student who interprets the question in a way that makes it easier and then supplies a correct answer to the easier question.
But you should reward a student who invents a harder question and answers that. And you should make it known in advance that you will do so. Indeed, taken to its limit, the optimal exam policy is to instruct the students to make up their own question and answer it, with harder questions (correctly answered) worth more than easier ones.
Incidentally I was once in a class where a certain professor asked exactly such a question.
Cheap Talk 
7 comments
Comments feed for this article
October 8, 2012 at 5:33 am
michael costello (@mcmc)
I think you are underweighting the strategic elements here. On a five question test, why would the student not prepare five stellar answers, rather than prepare for an infinitude of possible answers? Better to have an open book exam with truly inspired questions.
October 8, 2012 at 9:22 am
Dan Hirschman
Also, it’s not clear to me what you are maximizing. If every student made up and answered their own question, grading would take substantially more time and effort. Rewarding that behavior would increase the instructor’s expenditure of effort. To what end? If grading time is costless, sure, then ask students to make up a laborious question and answer it to prove their worth. But grading time is not costless, and it substitutes for time in office hours, time spent preparing lectures, etc.
October 8, 2012 at 9:38 am
DS
My experience in high school was that we were sometimes given examples of stock answers to study with, and inevitably some of my friends would essentially memorise those answers. This would sometimes results in them deliberately ‘misinterpreting’ the question – or at the very least, attempting to twist it slightly – so that they could get some impressive content down. I think there should be a significant penalty (I agree, not zero) for an extremely egregious interpretation of the question.
October 8, 2012 at 9:40 am
rjd100
The simple solution is to have the professor in the exam room and announce that if you don’t understand the question, please ask for clarification. Then the problem is eliminated.
If several students don’t understand the same question, then it should be struck from the test, as the professor was clearly not “on his game” that day. An incompetently written question deserves no answer.
October 8, 2012 at 10:37 am
David
Isn’t this basically what politicians do? — answer the question they want to answer instead of the question they are asked.
I agree with the first comment. I find students often will give an answer to demonstrate that they know something, even if (especially if) they don’t know the answer to the actual question.
October 8, 2012 at 11:26 am
Aaron
A related exam grading issue: carry-over errors. We would like to only penalize students once for arithmetic errors at the beginning of a multi-step problem if the remaining logic is consistent. However, sometimes a problem involves a fork: if your answer is A (say, some expression exceeds 0) go here and do more calculations, but if B (say < 0), you are done. If a student mistakenly gets B, they don't attempt the rest of the problem that would follow A. Rewarding points for the rest of the problem, therefore, is too generous AND creates an incentive compatibility problem. On the other hand, 0 credit for the rest of the problem seems too harsh, as their mistake may have been genuine and they may have eagerly and easily answered the Reston the question had they obtained A. What is a fair amount of partial credit? Or should these types of forks be strongly avoided?
October 8, 2012 at 11:35 am
Aaron
Side note: an interesting study on risk aversion in young students could be done by looking at their answers to matching problems. If 10 items in column 1 are to be matched with 10 in column 2, and a student gets down to 2 unmatched pairs, do they play it safe and ensure exactly 1 out of 2 points, or do they gamble between 0 and 2? However we don’t observe their prior, nor do we observe whether a student faced a decision and made the risky choice but got the matches right.