I’ve been ruminating lately about tests and wondering if our thinking about them hasn’t gotten into something of a rut. We give exams for two reasons. First, we use exams to assess the degree to which students have mastered the content and skills of the course. But like students, we can get too focused on this grade-generating function of exams. We forget the second reason (or take it for granted): exams are learning events. Most students study for them, perhaps not as much or in the ways we might like, but before an exam most students are engaged with the content. Should we be doing more to increase the learning potential inherent in exam experiences?
We tend to see exams as isolated events, not learning experiences that can be enhanced by other activities within the course. I’m convinced that a well-structured exam review, one in which the students are doing the reviewing, not the teacher, can motivate test preparation, promote good exam study habits, and effectively integrate and add coherence to large chunks of course content. I believe we can structure exam debriefs to help students learn what they didn’t know or missed on the exam. We cannot accomplish that goal if teachers “go over” the most-missed questions. Students are the ones who made the mistakes. They need to correct them. We also can use debrief sessions to encourage examination of the strategies and approaches students used to prepare for the exam.
The types of exams we give are remarkably similar. For objective exams, we create multiple-choice questions, maybe some fill-ins, matching or short answer, occasionally some true/false questions, or problems to solve. For subjective exams, we provide essay questions. For the most part, we give exams during a designated time frame, with no access to resources or expertise, and then teachers grade them. These features have prevailed for decades. Generally, we use the same exam formats within an individual course, within the courses that make up a program, and pretty much across various disciplines. I’m not saying that there’s anything wrong with these types of exams, I’m only noting how widely and consistently we use them.
Previous posts in this blog and the Teaching Professor newsletter have described an interesting array of alternatives—crib sheets, student-written exams, student-generated test questions, group exams, and two-stage testing (where students do the exam in class, submit it, and get a new copy that they complete before the next class, with the in-class test counting more than the take home). There are other options, but use of them continues to be the exception, not the rule.
The prevailing norm endorses challenging, difficult exams. Teachers want their exams to be hard to show students and others that the course has standards and rigor. But exams can be too hard. Difficulty has a point of diminishing returns—if students decide that intense study isn’t going to get them a decent grade, they stop trying. Lots of low exam scores are not usually indicative of a good test or lazy students. The challenge is finding that sweet spot where the test functions to differentiate those who’ve mastered the material from those who haven’t.
There are aspects of the typical exam scenario that tend to be rather artificial. How often are professionals required to demonstrate their knowledge within a discreet time period without access to resources or expertise? Yes, there times when knowledge is needed immediately—the emergency room comes to mind. And there are times when there’s no access to resources or expertise—a windy day when a lake with lots of rocks and boat motor that has stopped running. So there are times, but the question is: how many?
I understand that grades need to measure how well an individual student has mastered the material, and that does justify how we administer exams. But I’m not convinced that those details a student has in his or her head at the time of a test are as important as being able to find and assess information as it’s needed.
What are your thoughts? Do our testing assumptions and practices merit a revisit? Could they be doing a better job of promoting learning?
© Magna Publications. All rights reserved.
This Post Has 9 Comments
Pingback: Is It Time to Rethink Our Exams? | blogendahl
I think it’s worth examining those “most-missed” questions. How can we frontload concepts so students don’t build the flawed models that lead to those errors?
I’ve long held that exams should not only be assessment tools, but learning tools as well. I teach a specific accounting discipline. I build custom exams. I eschew computer or other questions provided by the textbook companies. I take long hours creating them. I take long hours grading them, so that I can give understand and give input on the thought process shown by the student in their supporting work and give partial credit as appropriate. The students get a solved version, with supporting work, to learn and study from. In addition, they get to keep their exams for the same purpose. Also, where possible, I give 3 midterms and a final, allowing each exam to be worth a little less in terms of the grade, a little less stressful, a little less material, etc. It also allows more chance for students to show progress through the course, which I can recognize in their final grade.
Exams must also have questions that are answerable by different levels of students. There are some which only the A students are expected to know, others for the A-, B+ and above. Others for the B and B- students. Etc. It is critical in evaluating an exam whether the right people got the right grades. I don’t care what the average is … there is no point to and exam where the entire range is like 20 points … say between 80 and 100 .. unless this accurately reflects the difference in the mastery of the students.
Unfortunately, in my career, testing is critical. I hear students say that they need to tell me that they aren’t good test takers. To get promoted in my field, you need a CPA certification, or some other certification, and those are hard tests. You must be able to take tests, or learn how, otherwise, there is a very low ceiling in the real world.
And speaking of the real world – much of our profession is based on being able to think on your feet and complete work according to rather tight time budgets and time frames. That’s the nature of the beast. Accordingly, there must be time pressure on my exams in order to help them develop this skill. The will need it on certification exams and out in the field. Giving exams that are easily finished by the whole class does a disservice to them in the long run.
Hello Dr Weimer
I could not agree with you more….In my teacher’s noviceness, I had stopped giving exams (my work is mostly with graduate students but I believe in certain undergraduate classes this can apply). Instead I wanted to ensure a low threat classroom with high doses of experiential learning. The latter was abundant and meaningful learning was happening but I often wondered how and what students were retaining….
I developed concept inventories (lists of what I thought would be critical information to know and understand) for my courses which I administered in the first week of class…This approach allowed me to customize my courses…then I had students complete them again during the last week of the course. I was easily able to track where the learning occurred (or did not).
Over time, I began to see that students’ are motivated by “the exam” and do study a bit more in preparation for the assessments. So, I tried utilizing the same concept inventory as the framework for my final exam. It was a success since what I considered important to know was mutually agreed upon with my students and quite transparent. I also believe in teaching “processes of knowledge attainment”; not minutia which fosters memorization with eventual depreciation. Therefore I do allow my students to bring in their resources (books and class notes) but I also time my exams …so students have to have some prior working knowledge to build upon.
I also agree that the exam review should be done in small groups of peers….not the teacher simply going over the item analysis…my students learned so much more about content and expectations for learning in these exam reviews than I could ever do alone. They became much less defensive about the “right” answer….whew!
I hope to write up this process of concept inventory as a means of assessment for your upcoming conference.
Sincerely
Cheryl Williams PhD RN CNE NP-C
Salem State University
Salem MA
Pingback: Building a better ruler: Is it time to rethink exams? | Innovative Course-Building Group
Exams are very good at assessing end-point learning and understanding, but seem to encourage students to expect to be spoon-fed the information during their learning. We need to have more balanced view of assessment that provides students with the expectation to come prepared for developments of the concepts in-class. My research led me to use Team-Based Learning in which students need to have read and understood some aspects of a concept and they have front-end testing both individually and as a team. Data gathered seemed to show that TBL was a very significant and effective strategy to raise the achievement of all/most students by at least one standard deviation and very significantly raised the pass rate for the whole group of about 300 students. TBL increases the student responsibility for learning and the interaction between students to support each other, while maintaining the sense of competition. Most important it shifts the balance towards intrinisc motivation, rather than extrinsic motivation.
I think exams in particular and assessments in general should be first and foremost opportunities for students to demonstrate how well they attained the learning outcomes. Whether it is content or skills students should come out of an exam feeling they have been given this opportunity.
As mentioned, exams should also serve as instruments to identify high-achievers and those who need to repeat the unit. They should have a clear indication of where the bar is. Anyone below the bar should repeat the subject, while at the same time only the very best can get 100%.
I think exams should be designed with a purpose in mind. And that purpose should be to give students opportunities to demonstrate what they know and can do.
My issue is with the obsolete view of college faculty as Quality-Control Inspectors, with exams as their instrument to detect defects. This also explains the prevalence of cheating, and our obsession with stopping it. But if the job extends beyond “covering the material” to creating an intellectual “tent” which incentivizes all the interactions and behaviors which result in learning, then the role of exams can change from an instrument to detect defects, into an instrument of summary learning, higher on Bloom’s Taxonomy.
In my Genetics course, I used three forms of evaluation. Two or three times during each class students were given simple quizzes and entered their answers via a student response system. At the end of the semester, these results were weighted equally to the four exams. I rarely had an empty class, or a lack of questions. I also provided lists of broad “Important Concepts” for each unit. During a recitation section, students worked in groups to answer these questions, which were graded on a “good, fair, poor” basis. Exam questions were narrower and more precise, but were drawn directly from this list.
I also used 30-hour take-home exams, on which students were allowed to collaborate in groups of 2, 3 or4. Interestingly, “free loading” did not seem to be a problem, because there was strong self-selection into groups of similar abilities and ambitions (Survivor: Clarion?) – the students themselves discouraged free-loaders, at least at the upper end. The exam was posted online at 3 pm Sunday afternoon, and was due before class at 8 am Monday morning. This time limit provided ample opportunity for intense engagement with Genetics (probably more than they would have spent studying!) but was not long enough for wholesale exchanges of information. It may have been quaint, but every student was also required to sign a pledge that on their honor they had not helped any other group.
The exams themselves were all problems and essays which required anywhere from 3-6 hours to complete, depending on how good the students were. Answers were typed, and professional standards of scientific writing were part of the grading rubric. This emphasized the importance of communication skills for the students and made grading easier for me.
The complaints I heard were that the tests were too intense, and too long, but never that they were too hard. I found I could live with this. I never had any complaints about unfair questions or unfair grading. Cheating was a moot point.
This way of doing things did not necessarily make me popular with students in the lower third of the class. They preferred old school methods. On the other hand, I flatter myself that the students in the upper third learned and retained more than they would have otherwise, and that they realized it too. I remain friends with many of them still.
I am writing my dissertation on the effect of different teaching methods and learning for students with ADHD. Constructivist or student-based learning uses a collaborative “group” teaching pedagogy – enhances development of critical thinking skills, executive reasoning, impulse control, cooperative social interaction, for in-class as well as distance education models. Assessment should be a tool used to determine how well the student has engaged with the material not a source for grades – I believe you should pass out all the tests at the start of the semester or quarter, encourage student feedback, as in let them propose one or two questions on each test, or even skip a question and substitute one of their own; either open-book or take home for all tests during the semester – then a comprehensive final in-class at the end of the semester, but pass this out at the start of the semester as well. Tests should not be an exercise in “surprise, here is something you did not think to study about, or in spite of your best efforts, here is something with more detail that you thought you needed” or some stupid multiple “guess” question that is supposed to make the student evaluate each of the possible answers – all of which are technically correct, but one is “more” correct than the others. These methods are not teaching and certainly not assessment – yes they are easy to grade but that is not the point of teaching – students should be encouraged to think about the material and interact with the material, not try to remember specific answers to trick questions – and a teacher who would use a true false format is just lazy. Just my thoughts,