There’s a lot of talk these days about evidence-based instructional practices, so much that I’ve gotten worried we aren’t thinking enough about what that means. Let me see if I can explain with an example.
Recently I’ve been trying to locate the evidence that supports quizzing, wondering if it merits the evidence-based label. Tracking down this evidence in our discipline-based research is challenging because although quizzing has been studied across our disciplines, it’s not easily searchable. My collection of studies is good, but I know it’s not complete. As you might suspect, the results are mixed; they are more positive than negative, but still, a significant number of researchers don’t find that quizzes affect learning outcomes.
I’ve been looking closely at a set of seven studies, which you will find listed at the end of the article. (These studies were randomly selected—no empirical objective here.) Not all of the studies report the same positive results, but if they are viewed collectively, the use of quizzes seems to yield some impressive benefits. Students reported they spent more time reading and more time studying between tests, and that they were more motivated to come to class prepared when the course included quizzes. These quizzes also increased student participation, lowered failure rates, improved exam scores, resulted in better overall course grades, and did not lower course evaluations. That all sounds pretty good, doesn’t it?
But the devil is in the details, as in the specific combination of factors and conditions that produced the results. When I looked closely at this subset, I was amazed at the array of details that could potentially affect whether quizzes improve learning.
- Are they pop quizzes or scheduled on the syllabus?
- What types of questions are used (multiple choice, short answer, etc.)?
- What’s the relationship between quiz questions and questions on the exam (same questions, similar questions, or completely different)?
- How many quizzes are given throughout the semester?
- When are the quizzes given—before content coverage or after? How soon after?
- Do students take the quizzes in class or online?
- Are the quizzes graded or ungraded? If graded, how much do they count?
- Is the lowest score dropped?
- What kind of feedback are students provided?
In addition to these design details, there are content variables derived from what’s being taught, the level at which it’s taught, the type of course, and the instructional method used to deliver it. And then there are student variables, such as their year in college and academic performance to date. In all likelihood, the classroom climate exerts some influence on the outcomes as well.
What this evidence tells us is that given a particular set of conditions, quizzes produce positive results, in most cases a range of them. And that gives us three things to consider. First, based on studies done in our disciplines, quizzes are an evidence-based instructional strategy only in a general sense. If your course design details and teaching context aren’t the same as those in the study, you aren’t assured the same results.
Second, to be sure that your quizzes produce the desired results, you need evidence. You can conduct your own empirical analysis. One of the benefits of all these different studies is that they provide a range of different ways quiz performance can be analyzed. That will give you the best evidence, but you can also do something quasi-empirical. You can look at exam scores in sections with and without quiz scores. You can ask students how a course with quizzes affects their attendance, preparation, and study habits. Or, you can carefully, thoughtfully, and objectively observe how quizzes are affecting learning. What we need to stop doing is assuming that just because an instructional strategy has been studied and judged effective, we can use that same strategy and accrue the same benefits.
Finally, looking at a set of studies (whether on quizzing or a range of other instructional strategies) illustrates the profound importance of instructional design. So often, when we decide on an instructional approach, we just do it. Without much thought or purposeful decision-making, we come up with a way to use quizzes. And yet it’s those easy, seemingly minor decisions about the details that determine the outcome.
Remember, though, that you haven’t gotten the whole story here. You’ve gotten the sum of a sample of studies done in our disciplines. Regular repeated testing has been studied elsewhere. In the next post, we’ll continue this consideration of what it means to be an evidence-based instructional strategy.
Resources:
Azorlosa, J. W. (2011). The effect of announced quizzes on exam performance: II. Journal of Instructional Psychology, 38, 3-7.
Batsell, Jr., W. R., Perry, J. L., Hanley, E., and Hostetter, A. B., (2017). Ecological validity of the testing effect: The use of daily quizzes in introductory psychology. Teaching of Psychology, 44 (1), 18-23.
Braun, K. W., and Sellers, R. D. (2012). Using a “daily motivational quiz” to increase student preparation, attendance and participation. Issues in Accounting Education, 27 (1), 267-279.
Hardsell, L. (2009). The effect of quiz timing on exam performance. Journal of Education for Business, 84 (3), 135-141.
Hatteberg, S. J. and Steffy, K., (2013). Increasing reading compliance of undergraduates: An evaluation of compliance methods. Teaching Sociology, 41 (4), 346-352.
Johnson, B. C., and Kiviniemi, M. T. (2009). The effect of online chapter quizzes on exam performance in an undergraduate social psychology course. Teaching of Psychology, 36 (1), 33-37.
Kouyoumdjian, H. (2004). Influence of unannounced quizzes and cumulative final on attendance and study behavior. Teaching of Psychology, 31 (2), 110-111.
© Magna Publications. All rights reserved.
This Post Has 6 Comments
So, what are the conditions under which quizzes are more likely to promote student learning?
Great question. I was wondering the same thing.
This article is more of a warning to not take research studies at face value. Need to consider the various variables that were a part of the study.
Perhaps some of us here have anecdotal evidence from our own experience using quizzes to suggest what might and might not be more effective in promoting student learning.
One problem with your collection of studies is that you are focusing too much on the overlap between “teaching” and “quizzing”—you are looking for evidence in the wrong discipline. You need to look at cognitive psychology. A VERY good resource (even if a bit dated now) is the book How People Learn (revised in 2000)–here is a link: http://a.co/8Huc3VS Focus on ways to test memory, and the learning styles that improve memory and learning. For example, what is the focus of the quiz? Are you asking about important details, or are you asking about major/global aspects of the material? Also, how high is the match between how you presented the information originally and how you asked about it later? Two key theories: 1) transfer appropriate processing, and 2) encoding specificity both lay out important variables in learning and later recall. Another thing that is important is depth of processing at various stages of learning. So–you are not looking in the right place for what you want to know. Look in the social/behavioral sciences, specifically, cognitive psych.
Thanks for the comment and resource, Dani. It’s as if you’re reading our minds. Here’s a snippet from part 2 of the article, which will run next week.
“Not all the evidence that supports the use of quizzes comes from our discipline-based research. There’s more to the story and much of it can be found in the research done in cognitive psychology.”
Stay tuned, and thanks again!
Mary Bart
Managing Editor
It’s retrieval practice that supports learning and that does not necessarily have to occur in the form of a quiz. There are motivational and emotional effects that go along with quizzing that cut both ways. Anticipation of assessment can improve attention and effort. It can also foster anxiety. The nature and timing of feedback also play a role. Students can learn the distractors. Testing for recognition? cued recall? or free recall and at which point in the learning process? It can get complicated. Bottom line: practice remembering.
Pingback: More on Evidence-Based Teaching