How much time do we put into the design of the assessment plans in our online courses? Is most of that time focused upon summative graded assignments that factor into the course grade? Or, do they also include opportunity for practice and informal feedback?
I’ve taken, taught, or designed more than a hundred online courses over the years, and I can appreciate many online course designs and approaches. With that said, I confess that I have an increasingly difficult time with online courses that limit assessment plans to a few papers, projects, quizzes, and tests. In an age of educational innovation and online learning, perhaps it is time to further explore enhancements to traditional notions of grading. With that in mind, here is part one of an article on my ten suggestions for enhancing or improving the assessment design in your online courses. There is no expectation that you use or even agree with all of the suggestions. Rather, consider them ideas to help jump-start your thinking about designing or re-imagining the assessment plan in your online courses.
1. Focus on Formative Assessment – Formative feedback is the annual checkup at the doctor. Summative feedback is the autopsy. The former gives one feedback that can be used to improve the patient’s well-being or the learner’s progress toward meeting the course goals. The latter doesn’t do much for the person being assessed. With that in mind, why not put most of our energy into designing high-quality formative feedback plans in our online courses? This is the feedback that helps learners discover how they are progressing toward one or more goals. It need not be high-stakes, graded, or made to influence the overall grade in the course (which would make it part of the summative assessment plan). Formative feedback can include self-assessments, peer-assessments, informal instructor feedback, computer-generated feedback, or feedback from mentors and people outside of the course. It allows students to use the feedback to improve and refine their work, rather than simply accumulate points that count toward an overall grade or certificate.
2. Make the Assessments Authentic – Learners thrive on experiences that relate to real world needs and contexts. Powerful learning occurs when participants see themselves as taking the course to connect with others and to learn something of value. “Of value” usually means something that they can use in work, avocations, or some other part of life beyond school. With this in mind, design assessments where learners actually build, create, or design something that they can use in other aspects of their lives. If they can’t use it, make it authentic enough that they can easily transfer the tasks completed in the assessment to a similar task outside of the classroom. This usually means setting aside or minimizing the use of things like true and false or multiple choice quizzes and tests. Performance on such assessments does not transfer to post-course life nearly as well as authentic assessments.
3. Beware of Using a Grading System to Punish – If you are going to use a grading system, make it a measure of what students have or have not learned. This is lost if you start removing points for late work and penalizing for behaviors that you want to discourage. It may work to do these things, but it turns your grading system into something other than a measure of student learning. If you want a system to track or encourage certain behaviors, then build a second and separate system for that, maybe a special badging system that publicly recognizes certain contributions.
4. Consider Using an Alternative to the Traditional Letter Grade System – Those of us in a traditional school system often need to use letter grades (unless there is adequate support for alternatives). However, we might be able to use a second and parallel system as well. Why not consider a digital badging system with clear criteria, a series of rubrics related to specific concepts that are important, or even a mini standards-based report card that gives more granular and helpful feedback than a single letter grade for the class? Or, depending upon the size of the class, what about trying out a narrative assessment plan where instructors and peers provide rich feedback in paragraph form, using a checklist? In the case of peer feedback, remember that this requires guidance from the instructor. Students will likely need help learning to give good narrative feedback to one another.
5. Design for a Pick-and-Choose Mindset – Many students treat courses as more of a buffet than a pre-served meal. They do not complete every single thing that is suggested or required. They figure out what they need to do to accomplish their goals, whether it is the goal to learn certain things or the goal of earning a specific grade. What if we designed the course with more of the buffet mindset? People will pick and choose what they want to do and what they want to avoid. Of course, there will still be certain required elements, but leaving room for choice allows students to self-direct more of their learning, which can help enhance student motivation. This buffet approach can work for assessment as well. If your goal is to measure what students have learned, why not give them different options from which to choose? If they could demonstrate their learning in any number of ways, why not leave room for that?
Editor’s Note: Part two of this article explores five additional suggestions for enhancing the assessment plan for your online courses. Continue reading »
Dr. Bernard Bull is the Assistant Vice President of Academics, Associate Professor of Education, and Director of the M.S. in Educational Design & Technology at Concordia University Wisconsin.
This Post Has 0 Comments
Pingback: 10 Assessment Design Tips for Increasing Online Student Retention, Satisfaction and Learning | EduWire.com
I'd be interested in studies validating the claims made in this article. Not because I wouldn't believe them but it would help convince others, too. Especially 1,2,5 are arguments where I expect a fair amount of resistance from colleagues who believe that summative feedback ("hard but fair") is being eliminated too easily; that theoretical knowledge ("not close to reality, but scientifically tested and sound") is key and tends to be overlooked (not so much fun to learn); and that we as instructors should decide on the menu, etc. — in other words, these suggestions are far from being self-evident, they're quite political in nature, hence ought to be validated.
That is a fair critique, Birken. There is indeed ample research to support many of these practices, although others are admittedly philosophical as well. One of the common threads throughout these tips in the notion of formative and authentic feedback. Toward that end, I would draw anyone's attention to one of the largest studies ever conducted on Formative Assessment called, Inside the Black Box: Raising Standards Through Classroom Assessment. While not focused upon higher education, this is a meta-analysis of 250 studies on formative assessment over close to a decade. The evidence in support of formative assessment was overwhelming. This is now well over 20 years old, but more recent research continues to verify these earlier findings. It will take a bit of time, but I will gladly compile an annotated bibliography of key studies about the benefits of formative and authentic assessment, and post them to my blog at http://www.etale.org.
@Bernard thank you — also for pointing out your other blog, etale.org — fascinating stuff. Yes to that compilation! — Something else: our school (whose head of e-learning I am) is located in Berlin, Germany. We have many international students. As much talk as there is about the global citizen, the dependency on nationality, background, culture, etc. is noticeable when it comes to that three-headed serpent student "retention, satisfaction and learning"…which I'm sure you aware of. — I wonder, rather than press for validating studies, if you have noticed anything like that and if you'd say that your suggestions hold for students from Asia, Europe etc. — Just to anchor this with one example: on average, Asian students who come to Europe to study at our school are (on average) used to hierarchy, to grade-based punishment, and do not have a pick-and-choose mindset at all, don't give up easily at all and if they do will not easily tell you why—and so on, differently for different nations, but basically…there are evident patterns of behavior. Even 1-2 terms spent in our system, alongside German students, will not necessarily alter their behavior, because their perspective is to return home. Of course, as Blake said, who generalizes is an idiot, but…the evidence for widely diverse (online) learning behaviors is there. I believe MOOC providers are also finding this out right now — esp. with regard to the low student retention rates. — In short: any views on how/if your suggestions depend on culture?
This is a great question. I'd like to think about it a bit more, but here is an initial reply.
The cultural aspect is fascinating, and I suspect that there is no simple answer. ! If you are interested, I gave an online presentation in 2013 called Global Perspectives on Grading and Assessment. It gets at part of your question, and you can listen to it here: http://www.youtube.com/watch?v=8a4ftAqirpw . In terms of my comments in this article, I suspect that different parts will resonate with people who identify with different cultures and sub-cultures. We can find these practices being explored in different parts of the world. Scan other comments to my article and you will find people who do or do not resonate with my comments. In short, I'm sure that cultural influences are an important factor. Thinking about it with different cultural backgrounds of learners in mind, I have different levels of confidence in some of my tips. My confidence in formative feedback, for example, is high across contexts. The comments about a "pick and choose mindset" less so; except to the extent that it represents notions of self-directed learning (see Sugata Mitra's Hole in the Wall Experiments as an example and articles like this one: http://www-distance.syr.edu/sdlhdbk.html that reference self-directed learning experiments in higher education in places like, "China, Indonesia, Japan, Norway, Russia, Saudi Arabia, Sweden, and Tanzania.". So, it seems like we can find examples of people exploring these different approaches to assessment with learners across culture, but it seems wise that we be sensitive to how they align or conflict with beliefs, norms, and values of different learners.
thank you, very helpful, will check these sources out. a student of mine just started a thesis on online learning design and may contact you for comments. cheers from berlin!
Those of us that teach in actual schools, colleges and universities are currently stuck with a system of grades. Students only do things in a class to earn their grades. That is how it is set up. The only way we can get students to do something is to assign points to it and punishing students for not doing it by having them lose points. If you just tell students to do something for their own learning they won't do it. Students need to not only learn the content of a course but learn that you have to do things on time, you have to use organizational skills to complete tasks on time. I do not support the buffet idea. That assumes that a student knows what they need to know about a topic they have not previously studied better than the professor who has made their career based on becoming an expert in that topic. The professor gets to choose what the student needs to do to learn this topic to an adequate level. No picking and choosing, you need all of it. If you don't do all of it, there should be consequences. In a job you don't get to pick and choose which components of the job task you want to do and what you don't, you do your job or you get fired. You get to your job on time or you get fired. We use grades to teach students that these skills are important. Grades aren't nearly as serious as getting fired, they get to practice in this setting. If students want to go to school just to learn for learning's sake then all three humans that would actually do that and pay money for it can go have their badges and non-grades. Ridiculous. I believe in formative and summative assessment but I also need to have all my students turn in their assignments at the same time (no late work accepted, no redos) so that I can grade it and move on to the next assignment. You want me to have a completely work-at-your-own-pace course with no deadlines and a gzillion redos? I could do that for a handful of students. The cost of paying my salary for a handful of students is just stupid. This is the system we are in. It is not ideal but there are certain things that work as long as the students organize their time and get their work done on time. By doing the activities, readings, and assignments I have created the students should learn the material. I try to have multiple forms of assessment. If they are terrible test takers then they also have a group discussion and written papers in which they can demonstrate to me that they have learned the content. If they do these assignments but six months after they were due, then they would be useless to hire anyway. I think what we really need is a change in our society so that there is no such thing as a student who is also working. If you can get into college through stringent, competitive admission criteria, then it is free and paid for by the government. No part time or full time jobs, your job is to be a student. Then students can focus on learning the content. They won't see themselves as customers and think they can tell professors what to do. They won't take this buffet approach because they'll perhaps realize that they don't know what they don't know, the professor does and has crafted a course that will get them there. They'll have no excuse for why they can't get their work done on time. It would solve so many problems.
Wow Shana…..this true in some sense. “The only way we can get students to do something is to assign points” …true, but I do not punish them for not doing it, or for handing it late. I have rubrics to deal with that part of the assessment of the activity and also have different kinds of assessments for different kinds of activities, but they do earn points and at the end we have to assign a letter grade…that’s the system. I let them know they are the sole responsible of their grade. I am a facilitator in whatever they need to learn, but they are the owners of their own learning and definitely they have to work hard towards their grade.
I do not like or share the buffet idea either. I teach mathematics and there is no picking or choosing…you need to understand the concepts, learn the “rules” for the game, practice and then apply what you learn. It’s like building a pyramid or any building…you need solid foundations.
Definitely, I am looking for good assessments design in order to increasing online student retention, satisfaction and learning, but do not agree with recommendations number 3, 4 or 5 that the author presents. What I’m looking when designing activities for an online course is exactly to include as many opportunities as possible to practice and to get immediate feedback. In order to achieve this goal in an online course, I have to say technology I a must.
I give my students incentives to do extra work, both by raising their final grade and with less tangible means. Doing the required work in my classes earns a student a final grade of C. All assignments are pass/needs work, meaning if it isn't done right, the student revises as many times as necessary until it IS right. There is no penalty for late work, but also no way to skip any of the required work. It ALL has to be done by the end of the semester to earn a passing grade. If the student chooses to do only the required work, just enough to pass, fine with me. It's designed to address all the learning objectives in the class and I don't waste my time giving feedback that assumes the student wants to do better than that. There are eight or ten optional assignments that students can choose from to earn a B or A. Students who are aiming for a better grade do the extra work, which asks for more from them, as A or B level effort should. I never have grade disputes because students choose their own grade and work toward it. And then, I have levels, purely intangible, that go beyond an A. 900 points gets you an A. 1000 points gets you bragging rights as a Jedi Master, represented by Yoda. Every semester, at least a couple students keep working beyond the A to earn that Yoda badge. These are college students, taking gen ed classes.
Shana, I have an issue with the thought of grading being punitive. Students earn their grades, points, badges, what have you. This is their educational currancy. I do not tax them with the grade slicer and slash their progress. That is something they do all by themselves. I do accept latework, but there is a point deduction for late subittals. Some students are absolutely fine with that. It is their choice.
Bernard this is the bomb- great article
Pingback: Design Tips | SAGU SPIRIT
Shana,
I agree with you. While I see the value in summative and formative assessment I am not in support of the buffet. I also do not accept that students should be allowed to submit their work when they can (more a case of when they like) and be assigned the same grade as a a student who submitted according to the deadline. We are training these people to enter the world of work. I work in business programmes and the business community to which we turn for support say they want graduates who have time managment skills, who can conform to regulations and who can work in teams. As for re-done work – well there is a cost to that. The person marking for the second or third time has to be paid. The time spent re-marking submissions is time taken away from other work. I always say to my (on line )students that in the real world an organisation does not get two and three chances to submit a tender for a project; it has to be right the first time and it has to be on time. So while I liked the article, there are some points I do not agree with.
I think this article encourages an important conversation about assessment, and in that, it has much value. Like some of the others, I, too, take issue with Recommendations 3, 4, and 5 — though my issues vary. Regarding #3, I believe an appropriate balance must be struck so that the grading system is fairly implemented. Awarding the opportunity for full points to those who abide the rules, submit their assignments on time, and demonstrate proficiency in their work, while exacting some form of penalty for those that choose to ignore the rules, submit late or not at all, and put little effort into what they produce seems quite fair to me. Such an approach reinforces the reality that effort is rewarded. I don't consider points deduction for late submissions (in those situations where they are accepted) to be "punishment" as much as it is being fair to those who followed the rules. Regarding #4, I work at a traditional public university, so not assigning letter grades isn't even possible. It is an interesting idea, but since so much is associated with letter grades (e.g., GPA, rank and standing, candidacy for scholarships, internships, potential jobs and programs after graduation, etc.), considering abandoning their use is a hard sell for many. Even when confronted with those who just give away grades or use arbitrary grading practices, grade inflation, and a lack of standardization in how the letter grades represent actual learning, I still think we (in the U.S.) are a long way away from broad acceptance of any alternatives. Lastly, while I can admire the notion of #5, and do attend to a form of it in my own classes, I can see the danger of broad implementation of such an approach. As Shana mentioned earlier, students don't know what they don't know — and allowing them to only choose what they want to do and learn may prevent them from venturing outside their comfort zone and actually learning something new, or challenging themselves. I mention that I attend to a form of the "buffet" approach because I allow students to choose the form of their major research projects. In recent years, I've abandoned relying on a major research paper in my courses (along with the other assessments); I now have students create multimedia research projects. I give them the freedom to choose an option for the format of the project (i.e., slide presentation/Prezi, video, infographic/digital poster, interactive animated timelines, etc.). Surprisingly, even when given a variety of choices, students typically default to slide presentations. There are a few that venture out to try something new on their own, but I'd say it is roughly the same number of students who over the years choose to go "above and beyond" in their work, or who (like Suzanne's padawans seeking "Jedi Master" status) choose to challenge themselves to achieve at a higher level, possibly for the self-edifying benefits of doing so. While I think some choice and variety in learning and assessment allows students to become more invested in the learning process, I absolutely think it is possible to exploit such an approach to the detriment of all involved. Students may love choices, but at the end of the day, most of them want to take the most expedient path to getting (not always 'earning') grades. — That said, I truly appreciate the conversation this article inspired. Looking forward to the next part! Thanks, Bernard!
I wanted to clarify a statement in my post: I once used discussions, quizzes, reflective essays, research papers, and exams in my online courses. Now, instead of the major research papers, students turn in multimedia research projects. They still have to complete the discussions, quizzes, essays, and exams. But they have digital research projects instead of research papers. 🙂
I'm delighted with the rich and robust conversation in the comment area. My goal is not as much that everyone will adopt all of my suggestions, but that it will challenge us all to reconsider and think deeply about our assessment practices…especially with a focus upon what helps students learn most effectively. That is what I see in the many comments, people grappling with such matters. I come from an educational reform perspective. Much of my research is in alternative, innovative, and emerging models of education; so I see many places where these practices are helping to cultivate new cultures of learning, and encouraging students to embrace new attitudes and perspectives on their own learning. Seeing such classes in action that practices these ideas has been quite inspiring for me.
Thanks for the comments, Tammara. The rest of this comment is in response to your post. There is certainly ample room for diverse perspectives on these topics. With regard to the same grade for work that is submitted late, I appreciate the different viewpoints. It comes down to one's answer to the question, "What is the purpose of letter grades?" If a grade is supposed to represent what a student has or has not learned in the class, we will have one opinion. If, on the other word, we say that a grade represents one's ability to meet the expectations of the teacher (timelines, following the instructions, etc.), then it seems like it would make sense to change the grade based upon matters that do not speak directly to what a student has or has not learned. I struggle with the idea that a student may have mastery of 100% of all course content, but turn everything in late and then get a C…while another student turns in everything on time and shows 85% mastery and gets a B. So, the grades would indicate that the B student knows more than the C student, but it would not be true in such a case. It would only indicate that one is better at meeting deadlines. Some accept this fact about grades, that they really don't accurately represent content mastery alone. I offer a different perspective to at least challenge us to think about our grading practices.
Here is an interesting exercise to try. Take any course syllabus and study it carefully. Then answer two questions. Given all the rules, policies and the like; what is the highest possible grade one could get while knowing very little about the course content? Also, what is the lowest possible grade one could get while knowing a great deal? I've done this with a large number of syllabi recently, and I discovered that most courses are set up in such a way that we can't, with any confidence, say that the final grade in a course accurately represents what a student has or has not learned. I fully agree that it is important for us to learn about the importance of deadlines and the like. It is just a question of how we help students develop such skills.
Regarding the buffet idea, please note that I prefaced it by saying that there should still be core, required elements. Your representation of the buffet idea is very much the sort of thing that I intended. So, we are largely on the same page there. I've seen many classes where instructors mandated not only the what of learning (which is usually and appropriately non-negotiable) but also the how of learning and how they will demonstrate their learning. I see more flexibility with how students learn and how they show what they learned….while being expected to meet a common standard. Of course, when appropriate, there is occasionally room for student choice on what to learn also (like in choosing their own research paper or project topic as long as it relates to a given theme or demonstrates mastery of a specific stated course objective).
Pingback: 10 Assessment Design Tips for Increasing Online Student Retention, Satisfaction and Learning « Analyzing Educational Technology
Pingback: 10 Assessment Design Tips for Increasing Online Student Retention, Satisfaction and Learning | Educational Innovation
Pingback: Top E-Learning Blogs | March 2014E-Learning Perspectives | Attaining Effective E-Learning for Today's Learners
Pingback: 23 Things Online Teachers Need to Learn | Morrissey EDU
Pingback: Designing Student-centered Assessments for Learning | Joni Tornwall
Pingback: Alternative Assessment Strategies for Online Courses | UAA Technology Fellows
Pingback: MindEdge Monthly: Top E-Learning Blogs E-Learning At Work