The Linda Hypothetical and Testing
Which alternative is more probable?
(1) Linda is a bank teller.
(2) Linda is a bank teller and is active in the feminist movement.
Most people choose (2) over one although it is illogical to do so since it includes (1) and involves a joint and, thus, lower probability.
I've never been a believer in the Linda hypothetical. First, I am not sure it tells us as much about logic as it does interpretation. Having listed feminist as a possibility in choice (2), "not a feminist" may be inferred to be included in (1). If so,both are examples of joint probability. I am also not sure if many people know what "more probable" means. Suppose one reads that as "best" answer meaning the one that captures Linda in a more precise fashion. Finally, how about telling subjects this is a test on logic? In short, is the test a valid test of reasoning?
For law professors, especially those using multiple choice machine graded exams the same questions are relevant. There are many reasons for choosing a wrong answer and sometimes the answers reveal more about the teaching and testing than the students. On the typical essay exam or multiple choice with explanation test the teacher can assess the quality of the question by examining why people missed it. Flaws in the questions are revealed. On a machine graded exam the process of "testing the test" needs to happen before the test is used. I wonder how many machine graders either copy the questions from another source and assume the questions are pretested or actually do test the questions by having a sample of students answer the questions and then debrief those giving the wrong answer. I am betting not many. Ironically, when the of issue machine graded testing comes up many of the defenses are also illogical.
posted by Jeffrey Harrison at 1:51 PM
4 Comments:
In my experience with multiple choice exams in law school, the questions are either too easy or do not provide a correct answer. In the too easy category, you have questions like “What are the elements of negligence.” Almost everyone is going to be able to answer that correctly, and if every question were like that, there wouldn’t be much of a grade distribution.
The questions that are unanswerable are the ones that are asking about “gray” areas of the law. If the question were posed on an essay exam, I would provide reasons that support different conclusions but probably not actually reach a conclusion. The multiple choice exam, however, forces the test taker to reach a conclusion when there is no objective correct conclusion to the issue. So I always ended up trying to psychoanalyze how the professor would view the issue to determine what he would think is the correct conclusion.
Only one of the reasons I do not give them. Others: I want to know how a student gets to the answer; I can better assess the fairness of the question; I do not want to test memorization.
Jeff, how do you feel about the use of multiple choice questions on the multistate bar exam?
Jim, It's a tough one. I am told those questions are written, pretested and vetted by experts. That does not describe your basic law professor. Plus, the happenstance of grading thousands of essay exams by hundreds of graders who have different standards could lead to some odd outcomes. So, I guess I would say it's necessary -- an unfortunate consequence.
Post a Comment
<< Home