Online instructors focus most of their teaching on curricular issues—what they will teach, how they will teach it, etc. But studies have found that differences in curriculum have little, if any, effect on student outcomes. John Hattie compared more than 100 factors related to student achievement from more than 180,000 studies and ranked the factors from most significant to least significant. Remarkably, “Programmed Instruction” came out at the bottom. While faculty toil over getting that perfect lecture, the variation in learning outcomes from different lectures is negligible.
So what did matter? Feedback. Feedback on student work has been consistently shown to be among the biggest influences on student achievement. Yet faculty usually give little thought to how they provide feedback to their students. Many go through a student’s work, making margin comments such as “vague” or “grammar” or “needs synthesis.”
Unfortunately, these brief margin comments provide almost no help to the student. The student seeing “vague” immediately asks “Why is it vague? It’s not vague to me. Why is it vague to you?” There is nothing in this statement to help the student understand what he or she did wrong or how to correct it.
Students need far more detailed feedback in order for it to have any value. In particular, students need both “feed forward” and feedback on their work. Feedback is a description of what a student did well or poorly in his or her performance. You used “they’re” here when you should have used “their” is feedback.
By contrast, feed forward is advice on how to do better in the future. For instance, the above feedback on the incorrect form of “there” could be followed by:
“They’re” is a conjunction of “they” and “are,” and so only use it when it can be replaced with “they are.” “Their” is a possessive pronoun, which means that it tells you that something belongs to something or someone else. Since you were using the term to refer to the Smiths’ house, which belongs to them, then you should have used “their.” Here is a good source for more information on the proper uses of the different forms of “there”: www.theretheyretheir.com.
This latter comment shows students how to use the correct form in the future. This is important, as some faculty just write “grammar” in the margin of a work, thinking that the student will take the time to look up the problem and solution on his or her own. But this is unlikely. You have the student’s attention now, so it is best to use the opportunity to provide information on both the problem and solution. The solution can also include a link to a resource that you deem good for learning more about the issue. That way the student can click it right there, rather than hunting around and possibly finding a poor resource on his or her own.
The Teaching Toolbox
The time involved in giving proper feedback can be an issue. While it will likely take more time, you can save yourself a huge amount of time by creating a “Teaching Toolbox.” A Teaching Toolbox is simply a repository of feedback on common problems. Throughout the courses you teach, you likely will see repeated mistakes on students’ work, and you will give similar feedback on those mistakes.
Instead of re-creating the same feedback over and over, you can save your comments to a repository that can be copied and pasted into student work where appropriate. The above feed forward is a perfect example. You can save the feedback on the differences between “there, their, they’re” to your Teaching Toolbox to be used the next time you see this problem.
Building and using the Teaching Toolbox
A Teaching Toolbox needs two elements: the content that is copied into students’ work and an organizing principle for finding it. Creating a good organizing method is often the hardest part. The Teaching Toolbox must be in a form that allows you to find desired content in less time than it would take to write it out again. Plus, it must be easy to add new content to the Toolbox.
The best way to host a Teaching Toolbox is with a text document. A text document allows content to be organized by headings, and the table of contents can be used for search. Building it requires deciding on the highest level of organization, say, content issues related to the course subjects and writing issues. You can write each heading as normal text, such as “Research Methodology,” and then designate them as “heading 1” so that the system understands them to be the highest level of organization. Next, write headings for the subcategories within these highest level categories, such as “Internal Validity” and “External Validity” and assign these “heading 2.” The process continues until reading the heading level for the actual feedback, such as “History Effect.” After designating that heading at its proper level, you can write the actual feedback to the student below it in normal text.
Once done, all you need do is tell the program to insert a Table of Contents at the front and it will build that table from the headings. Now all the headings that structure the Teaching Toolbox are right at the front, and reaching any particular content is as simple as hitting “ctrl” [left click] on a Table of Contents item in Word and you will be taken to the location in the document associated with it. You now can copy and paste the feedback into the student’s work.
Another option is to use the MS Word Quick Parts function, which allows the user to save blocks of text into the MS Word software itself. Quick Parts was designed to allow users to enter repeated content without having to write all of it out, such as signature blocks. But it can easily be used as a shortcut to providing repeated comments to students.
First, write the comment into the first student’s work, as is explained above, and then save it to the Quick Parts gallery in Word. When you want to use it again, simply type out the first three letters of the comment, followed by “F3,” and the entire comment will appear.
This latter method can be faster for providing feedback than the separate Teaching Toolbox discussed above, as you don’t have to move between documents, but it also has its drawbacks. For one, there is very little organization to the Quick Part gallery, which will make a particular comment harder to find, especially when quite a few comments are accumulated. Plus, since the comments are saved in the actual Word software, they will not be available on any other computer. Moreover, if you upgrade to a newer version of Word, you will need to figure out how to migrate the comments over.
The Teaching Toolbox not only saves time, but allows for far more feedback than is likely when the feedback is being reproduced from scratch over time. We don’t like repeating ourselves and so we often skimp on feedback on common problems. By contrast, the Teaching Toolbox will grow over time as you encounter new resources or have new thoughts to add to a particular type of feedback. Students are now getting more and better feedback as you preserve and add to your best feedback.
Reference: Hattie, J. (2009). Visible learning. New York: Routledge.
John Orlando writes, consults, and teaches faculty how to use technology to improve learning. He is the editor of Online Classroom.
Excerpted from Online Learning 2.0: The Teaching Toolbox, Online Classroom, 14.5 (2014): 4-5. © Magna Publications. All rights reserved.
This Post Has 8 Comments
This advice looks like it comes from the right place, but it strikes me as ineffective. The "Teaching toolbox" may work for mistakes involving basic grammar and rote memorization; but even if those mistakes are highly annoying and need to be corrected, they are not the most important mistakes student make. I teach philosophy (in french), and if I was to explain common mistakes like that, I would devote a lot of time giving feedback about things that don't have much to do with my class. I want to give better feedback about the quality of their arguments, the originality of their idea and the clarity of their writing, not about how to spell (the french analogue of) their/there/they're.
In my opinion, the Teaching toolbox you present is not really useful to give this type of feedback. I still need, every time, to explain why and how their particular argument is weak, why and how the definition they used is flawed, etc. So when it comes to giving substantial feedback on papers, it does not really help most teachers doing most of what they want (and should want) to do.
Moreover, feedback is not always read in a manner that promotes learning, especially when the comments are accompanied by a score. Dylan Wiliam cites a couple of different studies where students getting written feedback with a score gets no more benefit than students getting no feedback and a score, and much less than students getting feedback without a score (Dylan Wiliam, Embedded formative assessment, p. 107-111). So unless those comments are given without a score, it's a lot of time spent for mostly nothing.
Wiliam also stress the fact that effective feedback needs to be focused, and that giving a lot of feedback about a lot of different things is couterproductive (p. 130; my guess is that a lot of feedback is overwhelming and discouraging, and so the students accept their mediocrity instead of trying to get better). So not only giving a lot of information about spelling and basic grammatical mistakes takes time and is aimed at stuff that is not the main objective of the class, is actually hinders our capacity to make good, pertinent feedback sticks.
I'm going to come down between Jordan and John on this.
I primarily teach first year composition. Many first year students are too nervous to make sense of anything the first couple weeks. I keep a stack of answers to common problems, including copies of paragraphs from my syllabus in, a Shortkeys file. When I need one of them, I type the code and ShortKeys replaces it with my stored text in the destination font, so it looks written to order. When we get further along, I have other responses that I know from experience I'm going to say 87 times a week saved in ShortKeys. Most of these items are not feedback on writing, but responses to questions that students need to get on with their work. The ShortKeys keep me from getting too frazzled by repeated questions about things that have marginal relevance to writing.
When I grade, I try to leave students with two key ideas. One is advice that will make a difference on the student's next paper (read the directions, turn it in on time, etc.) The other is something the student should focus on in subsequent papers until they master that concept or skill because it is the thing that's going to keep them from becoming better writers. For example, many students have huge problems getting started because they try to make their first draft perfect before they've even decided what to write about. I would write a personal note about that and follow up orally with the student
I use the Connors and Lunsford list of errors in student papers and mark only those 20 items inserting the item number in the student's writing. I make students count and graph the errors they make in the first couple weeks. Then each student determines his/her three most common serious errors. Those become the only errors I look for in that student's writing that term. The students are responsible for learning to reduce the frequency of those errors to a certain number per X words by the end of the term. If the standard is 2 errors per 500 words, I stop marking errors when I've marked three occurrences. I cap the grade a student can earn if he/she has more than the allowable number of errors.
"The best way to host a Teaching Toolbox is with a text document. " Nah. The best way to curate frequently used comments, answers to FAQ, links to common tutorials, and any other text a faculty member might need regularly is a clipboard manager. I like ArsClip. It is free and awesome. Of course, if your school uses Turintin, their grading software has a pretty awesome comment bank feature
This is awesome and gives a great way for instructors to bank their comments. This is what I tell my instructors that I train to do. It's important to note that a well-written and detailed rubric can also help instructors handle general comments on the assignment while saving time to deliver the more in-depth personalized feedback.
I think there might be a way to be a way to be more proactive when dealing with at least some grammar/usage/syntax type errors by putting more responsibilty on the students' shoulders. It will use some class time to get underway but could save you hours of marking and frustration as the course progresses. I used it with a lot of success when I taught Communications 11 & 12.
Very early in the course, I'd assign a short written piece to get a sense of what the most repeated errors for that group were. From that & my own previous experiences, I'd make up a list of the 10 errors that either drove me nuts or recurred most often in the writing samples (or both) & called it 'Hellman's Hits'. For each Hit I'd show a model of correct usage (no explanation). Those examples had to be developed carefully to provide enough guidance for choosing correctly and at the same time to be simple enough to make comparison easy.
The next class I'd hand out a made up paragraph containing all 10 Hits (once each), hand out the list, and explain that avoiding these errors would save points deductions on all marked work. I'd also point out where this was reflected in the rubric. Then I'd break the class into groups to collaboratively edit my sample. When there was confusion over why a particular choice was right or wrong, I'd pause the group and teach a little more about it. We'd also talk about why spell checkers might miss these errors. I'd also admit that seeing these errors over & over was highly frustrating for me and many other instructors on campus and made it difficult to keep a positive frame of mind when marking students' work. The next day, I'd give back the original assignments and do an 'on my own, with a partner, with a group' proofread & edit. At the end I'd collect both originals & rewrites for marking. I'd do both for each student to show the improvement in the grade. If the entire class met the goal of resubmitting assignments which were 100% Hit free, they'd earn a prize!!!
When those were returned (& I usually gave them a second chance to win the prize if they didn't reach the goal), it was time to lay down the law: from then on I would refuse to mark any work that had even one of those Hits incorrect. If I found one in a student's paper, I would simply draw a line across the page to show how far I'd read, put the comment HH in the margin, and pass the work back to be fixed. When the new copy was resubmitted (inside 24-48 hours depending upon length), the original had to be attached to it. I would be available for help — but never to point out or correct a Hit, only to explain and provide more samples and practise in identifying and correcting. Periodically during the semester, I'd add 1 or 2 new Hits that really bothered me to list if the class was handling the original 10 pretty well.
The Hit List worked best when I collected at least one short written piece per week. It could be an answer to a sample test question, a spot quiz, or a 5 minute reflection — anything to get the students using their lists. I kept a stack of copies with me at all times so no one had to go without. I also offered bonus marks or a goofy certificate of recognition to anyone who found a hit in an academic paper, and I was very sure to get my list checked by a Writing Centre colleague so it reflected current best practice or required style on campus. What I liked best was that this strategy was pretty effective at stopping me from circling, correcting for, and commenting upon the same errors over & over and from becoming angry or disappointed when my hard work was undervalued by my students. When some non-Hit came up, I would encourage the student to see me about it and create his/her own supplemental list of Hits.
I enjoyed this article and we teach a similar strategy for grammatical and APA errors at our university. When I shared this article with faculty, they pointed out that in this sentence from the text, "They’re” is a conjunction of “they” and “are,” and so only use it when it can be replaced with “they are.”, the word conjunction is used incorrectly. They're is a contraction rather than conjunction.
I enjoyed this article and we teach a similar strategy for grammatical and APA errors at our university.
Hi DBS – Can you share this strategy?
Pingback: Instructional Design Newsletter – November 2015 | Technology at Seton Hall University