Very true and valid points, especially regarding being hooked up, I overemphasised the work involved. There is no doubt some rationalizing for convenience on top of the very real is/ought mentality (that I was caught in until I had to write a coherent reply).

The below is on how students should exam instead of how professors should exam, which are tangent on your points. But I wrote it, so i'm going to post it.

Don't forget the very students that are 'hooked up' with fraternity exam libraries and StudyBlue (that I'm not familiar with) become the productive employees that are 'networked' with their professional organizations and peers.

Those students are handicapping themselves based on their perceived ethical right thing. This is a similar argument to the 'we shouldn't be shallow so I'm not going to play the appearances game'. You are not winning by taking an ethical high ground, you are forfeiting for the sake of your ego.

Maybe there is something to exam writing in a way that captures that there is more to a successful university graduate than course mastery, but any attempt to defend it from the exam writers end looks very is/ought.

that I was caught in until I had to write a coherent reply

Upvote for catching yourself

Those students are handicapping themselves based on their perceived ethical right thing. This is a similar argument to the 'we shouldn't be shallow so I'm not going to play the appearances game'. You are not winning by taking an ethical high ground, you are forfeiting for the sake of your ego.

You could say that... or you could say that they don't wish to sacrifice to Moloch. You've got an argument that allows you to give into any and all perverse incentives here. C... (read more)

Exams and Overfitting

by robot-dreams 1 min read6th Jan 201547 comments

12


When I hear something like "What's going to be on the exam?", part of me gets indignant.  WHAT?!?!  You're defeating the whole point of the exam!  You're committing the Deadly Sin of Overfitting!

Let me step back and explain my view of exams.

When I take a class, my goal is to learn the material.  Exams are a way to answer the question, "How well did I learn the material?"[1].  But exams are only a few hours long, so it's unfeasible to have questions on all of the material.  To deal with this time constraint, an exam takes a random sample of the material and gives me a "statistical" rather than "perfect" answer to the question, "How well did I learn the material?"

If I know in advance what topics will be covered on the exam, and if I then prepare for the exam by learning only those topics, then I am screwing up this whole process.  By doing very well on the exam, I get the information, "Congratulations!  You learned the material covered on the exam very well."  But who knows how well I learned the material covered in class as a whole?  This is a textbook case of overfitting.

To be clear, I don't necessarily lose respect for someone who asks, "What's going to be on the exam?".  I understand that different people have different priorities[2], and that's fine by me.  But if you're taking a class because you truly want to learn the material, in spite of any sacrifices that you might have to make to do so[3], then I'd like to encourage you not to "study for the test".  I'd like to encourage you not to overfit.


[1] When I say "learned", I mean in the "Feynman" sense, not in the "teacher's password" sense.  I believe that a necessary (but not sufficient) condition for an exam to check for this kind of learning is to have problems that I've never seen before.

[2] Someone might care much more about getting into medical school than, say, mastering classical mechanics.  I respect that choice, and I acknowledge that someone might be in a system where getting a good grade in physics is required for getting into medical school, even though mastering classical mechanics isn't required for becoming a good doctor.

[3] There were a few terms when I felt like I did a really good job of learning the material (conveniently, I also got really good grades during these terms).  But for these terms, one (or both) of the following would happen:

  • I would take a huge hit in social status, because I was taking barely more than the minimum courseload.  At my university, there was a lot of social pressure to always take the maximum courseload (or petition to exceed the maximum courseload), and still participate in lots of extracurricular activities.
  • My girlfriend at the time would break up with me because of all the time I was spending on my coursework (and not with her).

 

12