Working Toward a Fair Assessment of Students’ Reflective Writing

There is little argument that reflective writing is a good way to foster critical thinking, encourage self expression, and give students a sense of ownership of their work (Chretien et al. 2012, Kennison and Misselwitz, 2002). This generation of college students has been doing reflective writing since elementary school so they are familiar with the process, even if not all enjoy it. Almost every academic discipline includes content on which learner reflection is appropriate; so the problem, typically, is not in creating the assignment but rather in assessing the work. How do we place a fair and equitable grade on an assignment that has so many variables? What are we looking for in our students’ work that we can reward and encourage with a good grade?

A project being piloted this year at the University of Louisville School of Medicine in the Pediatric Clerkship (where third-year medical student are introduced to clinical practice) introduced students to the Four C’s of Cultural Competence in the context of an actual case study of an Amish family dealing with tetanus in their 6-year-old son. The module included diagnosis, treatment, and prevention of tetanus; and recognition of cultural issues that affected how the family viewed modern medical practice. Because the module had multiple teaching/learning objectives, we used a brief reflective writing task to assess learners’ mastery of the content. Our assessment strategy needed to balance learners’ writing skills, critical thinking, and ability to apply basic principles.

Our team of two physicians and two medical educators developed a descriptive matrix rubric that outlined five areas of writing competence: (1) organization, (2) professional presentation, (3) depth, (4) content, and (5) point of view. Each element was graded at one of three performance levels: (1) honors, (2) acceptable, or (3) needs additional training. We followed Dyrud’s (2003) line of reasoning that simplifying the grading process creates greater equity and increased student satisfaction. In addition, since we were concerned with both medical practice and cultural competence, we worked in pairs (a clinician paired with an educator) to come to consensus on the grade for each student. Each team graded 50% of the reflection papers from the six clerkships (cohorts). We shared the rubric in advance of the assignment so students had a clear description of how their work would be assessed.

In regard to the rubric, all team members agreed that it set clear expectations and put the responsibility on learners for the quality of their performance. There were very few challenges to grades, and in each case, the student was concerned that the grading was “subjective”. A face-to-face meeting between the student and the course director resolved these challenges based on the rubric notes. The downside to using a rubric, however, was that it may have stifled the creativity of some students who preferred a different style of writing and processing the material. In this situation, where respecting individual differences was a teaching point, the rubric may have worked against the content by allowing a few learners to follow the “letter of the law” to achieve high marks but miss the implications of the content.

In terms of team grading, the collaboration definitely allowed a checks and balances approach. If one member was indecisive, the other weighed in with another perspective. There was no problem with inter-rater reliability as both teams had essentially the same score distribution.

One of the unexpected outcomes of the reflective writing exercise was that faculty gained new insights into learners’ thought processes. One team member expressed it this way: “I found myself forming an opinion about each student based on my own bias about the subject matter, applauding when it was incorporated in the writing and saddened when it was lacking. I was appreciative of the rubric that was developed to allow as much objectivity in grading as possible.”

So the solution may be in having a well defined rubric but being able to apply it with discretion and sensitivity to individual learner differences. The team grading strategy was a great help in this regard because gave each of us a little more security in assessing learners who deviated from the rubrics but “made up for it” with creativity and imagination.

References:
Chretien, K.C., Chheda, S.G., Torra, D., and Papp, K.K. (2012). Reflective writing in the internal medicine clerkship: a national survey of clerkship directors in internal medicine. Teaching and Learning in Medicine, 24(1), 42-48.

Dyrud, M.A. (2003). Preserving sanity by simplifying grading. Business Communication Quarterly, 66(1), 78-85.

Kennison, M.M., and Misselwitz, S. (2002). Evaluating reflective writing for appropriateness, fairness, and consistency. Nursing Education Perspectives, 23(5) 238-242.

Slavin, S., Galanti, G.A., and Kuo, A. The 4 C’s of Culture: A Mnemonic for Health Care Professionals; available online at http://gagalanti.com/articles/The4CsofCulture.pdf

Karen Hughes Miller, PhD, V. Faye Jones, MD, PhD, Pradip Patel MD, and Michael Rowland, PhD all teach at the University of Louisville School of Medicine.