Wednesday, August 26, 2009

"Annual" Multiple Choice Testing Post

It's been nearly two years since my "annual" post opposing multiple choice examinations for law students. The last one generated some good comments and can be found here. I still find the question intriguing. Before going on a bit, some basics. First, I am writing about machine graded exams; not multiple choice or true/false with explanation questions which are actually short essays that focus the students on specific topics. Second, I am not really writing about the mixed exam in which some "objective" (what a crazy thing to call them) questions are included with the essays. Third, I sincerely want the multiple choice machine graded (MCMG) supporters to be right. I hate grading more than anything else associated with my job. Finally, I think the whole matter presents a wonderful opportunity to examine self governance. More specifically, has anyone actually studied the effects of MCMG exams as opposed to essay exams or is the trend toward MCMG exams strictly a matter of convenience?

Here is what I like to know: If you use MCMG aren't you teaching a different course than if you use essay exams. I am not saying the teacher is doing anything differently but aren't the students "hearing" and making note of different things? Which course should be taught?

Do teachers at the fancy schools use MCMG exams? If so, does that mean the today's law schools are hiring people who are good at MCMG exams? If so, is that reflected in their teaching, testing and ultimately their evaluation of today's students?

What does it mean when someone defends MCMG by saying it produced a "great curve" or a "normal distribution." Does that mean the students were tested on the right things, whatever they are? I suppose you would get a normal distribution if you used a soft-ball throwing contest.

What does it mean when someone defends MCMG by saying the same students do well on both types of tests. What is the connection between that and what they are learning and teaching effectiveness?

Has anyone using MCMG exams actually studied how to write "good" multiple choice questions?
As a comment to my last post on this, Nancy Rappaport had some interesting views.

If you use MCMG exams, how do you perform the diagnostic element of teaching and testing? By that I mean the process of identifying individual and group weaknesses in reasoning and expression so you can adjust your teaching the next time around.

Having said all this and revealed my distrust if MCMG exams, I realize that some of the same questions could be asked about essay exams. What is the connection between good essay exam writing and a student's potential as an attorney, judge or law professor? I think I have a better chance of spotting the ones with great potential when they are forced to reveal themselves in an essay. But, that too has not been tested. In effect, our testing needs to be tested.

At one level what worries me the most is the thought that if essay exams could be graded even faster than MCMG exams, a fair number of law professors would switch back and then defend the new position as consistent with good teaching and evaluation.

3 Comments:

Anonymous Anonymous said...

From a student perspective, isn't your last point the most important one. Whatever happens it will be for the convenience of the professors.

8/26/2009 7:26 PM  
Anonymous Anonymous said...

If I were a student I would avoid courses where the professor gave multiple choice exams. As signalling devices go, multiple choice exams are pretty good ones with regard to how seriously the professor takes teaching.

I speak with some experience. When I was in law school we had a visiting professor (who went on to serve as a commissioner of the FTC and on the Competition Authority of a European Union country), and he gave us a multiple choice exam. That pretty well summed up his attitude toward students - be done with that nuisance as quickly as possible. Fortunately, I was able to learn antitrust once I was in practice.

8/27/2009 10:27 AM  
Blogger Stephen said...

Rex Lee gave complex multiple choice mixed with essay question exams. As far as I could tell, all it did was let him test more. As a student you basically got the normal essay test plus a bunch of very messy multiple choice questions.

The other teacher who did that confided that the multiple choice questions let him seperate out those who didn't know anything and deserved to be flunked from those who could not write.

Basically we ended up with a lot of extra work aimed at a student every other year ...

9/04/2009 3:52 PM  

Post a Comment

Links to this post:

Create a Link

<< Home