"Annual" Multiple Choice Testing Post

It's been nearly two years since my "annual" post opposing multiple choice examinations for law students. The last one generated some good comments and can be found here. I still find the question intriguing. Before going on a bit, some basics. First, I am writing about machine graded exams; not multiple choice or true/false with explanation questions which are actually short essays that focus the students on specific topics. Second, I am not really writing about the mixed exam in which some "objective" (what a crazy thing to call them) questions are included with the essays. Third, I sincerely want the multiple choice machine graded (MCMG) supporters to be right. I hate grading more than anything else associated with my job. Finally, I think the whole matter presents a wonderful opportunity to examine self governance. More specifically, has anyone actually studied the effects of MCMG exams as opposed to essay exams or is the trend toward MCMG exams strictly a matter of convenience?

Here is what I like to know: If you use MCMG aren't you teaching a different course than if you use essay exams. I am not saying the teacher is doing anything differently but aren't the students "hearing" and making note of different things? Which course should be taught?

Do teachers at the fancy schools use MCMG exams? If so, does that mean the today's law schools are hiring people who are good at MCMG exams? If so, is that reflected in their teaching, testing and ultimately their evaluation of today's students?

What does it mean when someone defends MCMG by saying it produced a "great curve" or a "normal distribution." Does that mean the students were tested on the right things, whatever they are? I suppose you would get a normal distribution if you used a soft-ball throwing contest.

What does it mean when someone defends MCMG by saying the same students do well on both types of tests. What is the connection between that and what they are learning and teaching effectiveness?

Has anyone using MCMG exams actually studied how to write "good" multiple choice questions?
As a comment to my last post on this, Nancy Rappaport had some interesting views.

If you use MCMG exams, how do you perform the diagnostic element of teaching and testing? By that I mean the process of identifying individual and group weaknesses in reasoning and expression so you can adjust your teaching the next time around.

Having said all this and revealed my distrust if MCMG exams, I realize that some of the same questions could be asked about essay exams. What is the connection between good essay exam writing and a student's potential as an attorney, judge or law professor? I think I have a better chance of spotting the ones with great potential when they are forced to reveal themselves in an essay. But, that too has not been tested. In effect, our testing needs to be tested.

At one level what worries me the most is the thought that if essay exams could be graded even faster than MCMG exams, a fair number of law professors would switch back and then defend the new position as consistent with good teaching and evaluation.