The Grading Thing
I often tell my first year students that writing an exam is the hardest thing I do and grading them the most miserable.Now I have heard of a new way to handle both tasks. Substitute hours of constructing an exam with a few seconds of changing the date on last year’s exam. As for grading, multiply choice, machine graded (mcmg) tests are just the ticket.
Obviously, this saves an enormous amount of time but I am not sure what the grades mean. My premise here is that most first year teachers, in particular, devote a fair – if not most – of their time to critical thinking and analysis. Most issues the students see have a Rashomon like quality.
I am far from convinced that a mcmg exam is a useful tool for assessing the development of those skills. Worse, if you are known to be mcmg person I think it is likely that students know not to take much of what goes in class seriously unless you are just downloading information. Mcmg people may say that there is correlation between how students do on complex essay questions and mcmg exams but that may only hold for the first couple of times giving the mcmg exam.After that, the professor is teaching one thing and the students are preparing for a test on something else.Even if good multiple choice questions can be written for law courses and I believe it is possible beyond the first year, it is very difficult. That difficulty leads to the recycling temptation. If you recycle multiple choice questions and do not think many of the questions are “out,” please get in touch with me because your powers of rationalization are far beyond mine and I would sincerely like to escape my own feelings of regret about some of my own misdeeds.
Ok, so let me predict. Those of you who use mcmg exams are annoyed because I have called into question your work ethic and integrity. Of course not! What I am actually saying is that recycled mcmg testing in law schools "gives me pause," "concerns me," "puzzles me," "makes me wonder," "may require extra care," "needs careful consideration" or "is a good idea for some." And, if you have checked to determine whether your recycled mcmg exam actually tests what you are teaching, I am not even saying anything that harsh.I think one of the tenets of MoneyLaw is to do all the functions of a teacher – even the grunge of assigning grades – so that stakeholders are not shortchanged. It seems to me that recycled mcmg tests and, for that matter, recycled short answer questions may be like giving $2 back to a customer who has given you a $5 for a $1 item. But then the privileged make the rules -- even when it comes to making change, don’t they?
Maybe I am just irritated about grading 180 essay exams.
All kidding aside, this is one thing I really would like to be wrong about. If you give mcmg exams to your first year class, how about sharing one in the comments area along with your explanation of what the question tests. I'd like to be converted to your point of view. But first, I'd appreciate if you would respond to following poll.
posted by Jeffrey Harrison at 7:45 PM
9 Comments:
I'd also be interested in how many people use a mix of essays and short-answer questions, just essays, just short-answer questions, etc. For all but my first-year classes, I've actually moved to having students write papers, in part because I really want them to improve their research, analytical, and writing abilities.
Jeff,
Does it not strike you as ironic that you close this post with a multiple-choice, machine-graded question?
Jim
Jim: I chose multiple choice to cut down on my grading time.
While I have my own concerns over exams that depend too much on multiple choice grading, let me make the case for incorporating such questions.
1) First, they can contribute to a holistic style of learning assessment and legal training. While it is important that students be assessed on their ability to write effective essay answers, there are situations in which we simply need a a correct answer among alternatives. I have concerns that certain students are very effective writers and can essentially "shoot the bull" to get out of answering questions in a straightforward manner. If you are able to sort out eloquent writing and advocacy from truly insightful analysis in your grading then good for you; but I think that some instructors are swayed by beautiful writing and have inflated estimates of such well written, but wrong or not insightful answers and perhaps deflate answers by people who basically get it right, but not in a eloquent manner (or may have bad handwriting if you dont have typed exam answers).
2) Incorporating mc questions taps into different learning styles and allows people who think in different ways to demonstrate their grasp of the subject, rather than simply putting all of our grading weight on one's ability to do issue spotting analysis (i.e. only one way of demonstrating knowledge). Again, this only sees mc as one of a number of ways of grading and would go against using solely mc tests.
3) If we are interested in students doing well on the bar exam -- well one whole day of the bar (the national portion) has traditionally been devoted to entirely mc testing and some states use mc testing in their portions. A pragmatic concern, to be sure - but one that constituents (i.e. students) care about.
Again, I'm certainly not in favor of tests that use just mc questions, but I think that they can be a valid way to assess student's learning.
I do not fully understand Mouse's defense of multiple choice. Item by item.
1. It means the teacher does not have to wade through eloquent writing to assess insightful analysis. Yes, perhaps but it is at the cost of the insightful analysis.
2. It allows students with different learning skills to demonstrate their grasp of the subject. Does multiple choice actually test "grasp" of the subject?
3. To help prepare for the bar. Here I agree. In fact, I have heard of but do not personally know of professors who actually use old bar exam questions.
Doesn't it depend on the course? I've been an adjunct and have been teaching a course in venture capital for 5 years. The course is broad, rather than deep, covering many different subject areas (corporations, securities, tax, labor & employment, contracts, etc). Essays seem less valuable as a measure of knowledge for this type of class.
For the first 4 years I used a short answer format. Last year I used half short answer, half multiple choice. I did this in part because my class size significantly increased (I had the same size limit, but was soft and waived it for a number of people who wanted to take the class).
I admit my sample size was still small (about 20 students), but there was almost perfect correlation between performance on the short answer section and that on the multiple choice. That is, I could have ignored either section and the students would have ranked about the same.
I use a mix of modified multiple choice, short essay, and long essay questions in all of my classes, and I write new ones every year. The students get 4 points for the right letter in the MC, and 5 for the right explanation. This guards against rewarding lucky guessers and allows for more nuanced evaluation of the students' understanding of the material and application. I use MC questions to test careful reading, knowledge of individual rules, application of rules to facts in rather straightforward fact scenarios, and differentiating between related doctrines. Short essay I use to test the student's engagement with legal arguments and to test the ability to use policy to frame an argument, make a business decision, or inform the creation of law. Alternatively, I use them to test doctrinal areas that we covered very superficially in class, asking for the high points. Long essay I use to test careful reading, the ability to spot issues, to explain rules, and to apply those rules to the facts thoroughly. My tests are also open book and open note.
I think the different types of questions test different abilities, all of which are required of lawyers in practice, even if they're not required in precisely this form and in a 3-hour window.
It's not the easiest to grade, but it's not nearly as difficult as grading papers or all long essay issue spotters.
The way I read the survey, I think Harrison is referring to machine gradeable multiple choice, not multiple choice with explanation which I have used. I write new ones each test and find that this type of question allows me to direct the student's attention to exactly what I want him or her to discuss. It does elimiate issue spotting.
My worry about the traditional essay is that 10 students could spot 10 issues and ten may spot different ones. Each discusses the ones he or she spots and I feel like I am grading answers to different questions.
Did Mr. Mouse number 2 say that he or she went to multiple choice because the class size had gone all the way up to 20? Must be nice!
Mouse 3 is correct. My use of the term "multiple choice" meant a question that allowed for a letter answer only, no explanation. True/False would also fit.
Post a Comment
<< Home