assessment

Non-Traditional (Un)Grading Systems – An Overview

It’s worth, first, marveling at the relative newness of grading. In fact, the now-standard letter-grading system only gained widespread popularity in US schools in the 1940s. This was partly a response to a decades-long concern over creating a standard of uniformity across institutions, thus standardized grading emerged as an administrative tool for interinstitutional coordination. Today, however, grading (or “evaluative feedback”) is mostly conceived as a pedagogical tool that operates as a source of communication to students and as a hotly debated source of student motivation.[1]

Over time, several different grading systems have developed, including non-grading (or un-grading) systems that see evaluative feedback as a detriment to student learning. Scholarly research on educational assessment is bewilderingly extensive, nevertheless here is a relatively short, curated list of several non-traditional methods of grading employed in higher education.

What is “Traditional Grading”?

We should note at the outset that there is no traditional “traditional grading” system, it’s an amorphous and ultimately abstract entity. Many instructors have vastly different grading policies and practices suited to their personal educational interests or tailored to their student needs. Yet, working from within a criteria-referenced paradigm, traditional grading could be characterized by its attempts to evaluate student work fairly, accurately, and uniformly across a class (and perhaps between classes or across time). Additionally, students earn points as assignments, projects, exams, and so forth, are completed to a certain level of quantified competency throughout the term. Lastly, a final grade is assigned based on these points values (often averaged) and a variety of other factors (like attendance, participation, extra credit, etc.).

Several concerns raised about this system include the perception that instructors are forced to inhabit the role of a grade “gate-keeper,” consequently engendering the distrust of their students. Some complain that the system encourages student grade performance over their learning mastery, a finding backed by research. Moreover, some question the ability for grades to be assigned fairly and uniformly across a class. Others will point to the potential waste of time for instructors who are forced to teach students who are satisfied with only the most minimal competency in a topic. Lastly, some would like to see a meaningful way to incorporate student effort in addition to student competency in any grading system.

Because of concerns like these, alternative grading systems have been proposed. While non-traditional systems are often promoted as better for learning and teaching, there is no value-neutral grading system. The mechanisms of the system will direct or encourage certain types of learning (and teaching) behaviors over others. Furthermore, certain grading systems may be more (or less) time-intensive for instructors or students, or cause efforts to be front-loaded or back-loaded in comparison to traditional systems. Because of this, it is worth making an informed decision on which grading system we employ, whether it’s a traditional or a non-traditional variety.

Wait…Aren’t these “Non-Traditional” Grading Systems Just Fads?

This is a fair question. Radical departures from norms may only seem “better” because they are new. One of the following examples, contract grading, has been widely studied since the 1970s and has been regularly found to have beneficial impacts on student learning and motivation. Additionally, both specifications grading and levels grading are built upon elements that have sound research behind them, even though, as entities, they have not yet been the focus of empirical research.

Specifically, and this applies to all three systems, a focus is placed on evaluation transparency, where the purpose of the exercise or assessment is clearly explained, the task is clearly described, and the evaluation criteria is clearly delineated (perhaps think of a grading rubric) and provided in advance so as to help students with self-evaluation. In some ways, these three alternative grading systems are designed to fully operationalize the principle of transparency, a relatively simple teaching intervention that has been shown to demonstrably enhance student success, including academic performance, student mastery of skills, student confidence, a sense of belonging leading to better retention. This does not represent the totality of these grading systems, but helps to explain their particular design. Non-grading (which could also incorporate transparency) has long been shown to be a better motivator of student effort and allows instructors to put their time and energy into areas that have a more appreciable impact on learning. Overall, these are not transitory fads, but systems built on the best available research in educational psychology and instructional design. With that being said, the individual implementation of these systems can be quite varied (any investigation into the literature on these will quickly reveal this fact) and as such rely heavily on the specific interests, purposes, and needs of the instructor.

Contract Grading

With its origins in the early 1970s, this form of grading has been championed by Peter Elbow, whose work has left a lasting mark in the field of composition studies. Due in part to Elbow, contract grading is most commonly used in composition and rhetoric courses, although it has wide application across disciplines. In an attempt to move student interest away from the commodity of the grade and towards nurturing more essential learning skills and behaviors, contract grading is based on establishing an agreement with students regarding the quantity and quality of work they need to complete, among other criteria, which is correlated to a particular grade. These agreements can be negotiated with individual students as they propose activities and projects, which, when completed, receive the agreed-upon grade. Contract grading can also be non-negotiable, or applied equally to the whole class with instructors providing the specified criteria and the related grading output.[2] Jane Danielewicz and Peter Elbow have outlined the latter method by establishing a B-grade set of criteria (attending class regularly, participating in all in-class activities, providing thoughtful peer-feedback, etc.) for students to work on for most of the semester (see resource below). Only with the submission of a final portfolio would a final grade higher than a B be considered for students who fulfilled the contract. Grades lower than a B are possible, but as the authors note, “we are frankly trying to badger and cajole every student into getting a B.” Some recent research by Dana Lindemann and Colin Harbke suggests this grading system succeeds in discouraging students from failing a course and also provides students with higher competency in the desired skills and topics. Of the non-traditional systems noted here, contract grading has received by far the most research.

Resources:

  • Danielewicz, Jane & Elbow, Peter. 2009. “A Unilateral Grading Contract to Improve Learning and Teaching,” College Composition and Communication, Vol. 61, No. 2, pp. 244-268. [here]
  • Davidson, Cathy. [Twenty-First Century Literacies syllabus]
  • Hensen, Leslie. [syllabus]
  • Inoue, Asao B. 2014. “A Grade-less Writing Course that Focuses on Labor and Assessing.” In Teague, D. & Lunsford, R. (Eds.), First-Year Composition: From Theory to Practice. West Lafayette: Parlor Press, pp. 71-110. [despite the title, the focus is on implementing contract grading][here]
  • Inoue, Asao. [syllabus]
  • Lindemann, D. F., & Harbke, C. R. 2011. “Use of Contract Grading to Improve Grades Among College Freshmen in Introductory Psychology.” SAGE Open. [here]
  • Volk, Steve 2016. Contract Improv – Three Approaches to Contract Grading (Center for Teaching Innovation and Excellence at Oberlin College)[three different methods for implementing contract grading]

Specifications Grading

This is a more recent grading system – not entirely unrelated to contract grading – proposed by Linda Nilson and most robustly discussed in her 2015 work Specifications Grading: Restoring Rigor, Motivating Students, and Saving Faculty Time. At its core, “spec grading” relies on the establishment of clear and detailed specifications for what constitutes a passing piece of work (for Nilson, typically, B-level work or better). This is not unlike creating a grading rubric, but one only needs to detail a “satisfactory” set of criteria, not a full range of grading possibilities. Assignments are bundled, and the more advanced bundles represent more complex skills and/or content. Students are graded only pass/fail for individual assignments or tests and progress as they receive passing grades. Bundles, however, are tied to overall course grades, thus spec grading allows students to determine which grade/bundle they want to compete. Also incorporated is the rather interesting idea of tokens. These are allotted at the beginning of the term to each student and can be redeemed to revise an unsatisfactory assignment, hand in work 24-hours late, take a make-up exam, and so forth. Given the limited number of tokens, students need to think about how to use tokens strategically.

Resources

Levels Grading

Modeled on video game mechanics, Dustin Locke has recently developed a grading system similar to specification grading, but with different nuances. There are a total of three levels, each of which consists of a larger, more complex project, and which are each evaluated on a four-tier system: complete, almost, good effort, and not much progress. A student progresses to the next project/level only when they have received a “complete” on the previous project/level. Like the spec system by Nilson, a student needs to gain a certain competency or mastery of a skill or content before moving on to the next project. Importantly, there are specific windows when students can attempt to complete levels, thus the project/level any given student needs to be adaptable to the content that is being covered at that time in the course. The final evaluation a student receives on a level is correlated to a final course grade. This system is currently growing among philosophy instructors.

Resources:

Portfolio Assessment

Not necessarily a fully reconceived grading system (eg. it is used as part of Peter Elbow’s contract grading system), portfolio assessment grew in popularity in the 1990s as interest gathered around alternative assessment techniques. In its simplest form, a portfolio is a collection of student work that exhibits their effort and progress in a course. It includes student-selected documents, learning products, or artifacts that they feel represent their best work, and as such, it usually represents work they have revised, sometimes significantly, and reflects their learning processes. Often students will be asked to include reflective documents, such as cover letters describing the selection process and the pieces the choose to include. Oftentimes perceived as an “authentic assessment” tool, this is very common in composition courses.

Resources

Ungrading

There have been many calls for the abolishment of grades, and there’s good research to suggest this is a wise pedagogical decision. In the broadest strokes, evaluative feedback (grades) alone, where students are essentially ranked in accordance to one another, doesn’t provide any valuable information about how to improve their understanding or competency nor has it been shown to provide any positive motivation for students to truly master a topic or take intellectual chances. Alfie Kohn, one of the most vocal proponents for diluting and removing grading, has made a career on this topic.

One point of confusion, however, is that by removing grading one removes all evaluation. This is not true. Emphasis shifts to descriptive evaluation where pertinent information related to improving student competency is shared and discussed. And while evaluative feedback and descriptive feedback are often coupled in practice in traditional grading systems, research cited by Kimberly Tanner and Jeffrey Schinske in their provocative “Teaching More by Grading Less (or Differently)” suggests that students are less likely to read comments that are paired with grades. Providing only descriptive feedback had been shown to be the most efficient for student learning and is also preferential to some students.

One of the most characteristic aspects of non-grading or ungrading is metacognition and self-assessment. Jesse Stommel, who has reflected thoughtfully on his practice of ungrading, has his students regularly engage in self-reflection through “process letters,” which open up a space of dialogue “not just about the course, but about their learning and about how learning happens.” This allows both for the instructor to provide constructive feedback and encouragement and for the student to cultivate the skills of critical self-assessment and future planning. Of course, working in a traditional institution, he needs to assign final course grades so Stommel has students grade themselves. He reserves the right to alter any grades his student submit, but he claims the most common alteration he makes is from an A- to an A, for the students who are too modest in their self-assessment.

Resources:

Final Thoughts

These alternatives are all reactions to dissatisfaction with traditional grading systems. From a bird’s eye view, these all emphasize pedagogical approaches that we should all immediately appreciate, including careful and strategic scaffolding of lesson plans and assignments, creating transparent and detailed evaluation rubrics, encouraging students to engage in metacognitive activity, and giving students a sense of purpose and ownership over their own learning. Several systems (perhaps my selection bias) foreground student competency or mastery that happens in stages, which in turn can allow for the implementation of a simpler evaluative feedback consisting of a two-tier pass/no pass (or pass/revise). In almost all cases, some power and authority is wrestled away from the instructor and placed in the hands of the student.

Other Resources on Grading

  • Docan, Tony N. 2006. “Positive and Negative Incentives in the Classroom: An Analysis of Grading Systems and Student Motivation, Journal of Scholarship of Teaching and Learning, Vol. 6, No. 2, pp. 21-40. [here]
  • Elbow, Peter. 1994. “Ranking, Evaluating, Liking: Sorting Out Three Forms of Judgment.” College English, Vol. 55, No. 2, pp. 187–206. [here]
  • Kohn, Alfie. 1999. Punished by Rewards: The Trouble with Gold Stars, Incentive Plans, A’s, Praise, and Other Bribes. Rev. ed. Boston: Houghton Mifflin.
  • Tanner, Kimberly & Schinske,, 2014. “Teaching More by Grading Less (or Differently),” CBE Life Science Education, Vol. 13, No. 2, pp. 187-206. [here]
  • Tchudi, Stephen, ed. 1997 (2011). Alternatives to Grading Student Writing. National Council of Teachers of English [here][this has been a constant source of inspiration for me]
  • Winkelmes, Mary-Ann; Bernacki, Matthew; Butler, Jeffrey; Zochowski, Michelle; Golanics, Jennifer & Weavil, Kathryn Harriss. 2016. “A Teaching Intervention that Increases Underserved College Students’ Success,” in Peer Review, Vol. 18, No. 1/2. [here]

Notes:

[1] For a brief survey on the literature regarding the positive and negative motivating effect of grades, see Docan 2006. Evaluative feedback is often distinguished from descriptive feedback which provides specific information about how a student can become more competent. Often these are used in conjunction. Grades are also used in an organizational manner, such that they are used to partition lessons, units, or terms. In this context grades are seen as a “summative assessment,” in contrast to a “formative assessment” which places more focus on informal tests of students’ understanding before administering a summative assessment.

[2] These two broad forms of contract grading are described in the 1971 work, Wad-Ja-Get? The Grading Game in American Education, by Howard Kirschenbaum, Sidney B. Simon, and Rodney W. Napier. Incidentally, when contract grading was increasingly discussed as an alternative grading system in the early 1970s, the idea of instituting a two-tier pass/fail grading system in contrast to the A-to-F system was also discussed widely. We will see that the combination of elements from both contract grading and a two-tier grading system are found in some of the most common alternative systems circulating today.

Non-Traditional Assignments Workshop

As instructors, we somtimes feel the temptation to spice up our coursework. We may feel a bolt of jealousy, for example, when we hear a colleague’s idea for a brand new approach to an old-school assignment (I’m looking at you, humdrum 10-page research paper). While reflection and refinement should be the standard to which we inspire, we should not seek novelty simply for novelty’s sake.

When seeking to craft a non-traditional assignment, we should be guided by our core educational goals, many of which cluster around critical thinking and thoughtful creativity. By foregrounding these goals it’s possible to conceive of new ways we can express our interests or student passions. In other words, the search for non-traditional assignments should be in service to expressing “traditional” education goals in new ways. Specifically, in ways that appropriately reflect our course content, student skill sets and interests, or general teaching philosophy.

I should note, it is also far easier to integrate a non-traditional assignment into a course when that course is designed “backwards.” By starting with and thinking through the learning outcomes we can more easily conceptualize the myriad ways in which they can be actualized. In contrast, if you start by thinking through all of the assigned readings written by scholar for scholars, it’s pretty natural to envision all student assignments in the same fashion, i.e. as variants of scholarly writing. Indeed, one of the first limitations you may run into with “backwards design” is that all of those scholarly monographs you assigned just do not work and need to be trashed mindfully returned to their bookshelves. (I apologize to the international cartel of scholars, please do not put cyanide in my afternoon tea. You weren’t getting any royalties from your book sales anyway…[1])

If you are not undeterred, below I offer several of the slides I presented for a non-traditional assignments workshop organized by my university’s Summer Teaching Institute for Associates program. I will also provide commentary to the slides. This workshop was far more “workshoppy” than my previous ones, thus there is much on my handout that is not covered in my slides. [NonTrad Workshop Handout]

We began the workshop by discussing the reasons why instructors might seek out non-traditional assignments. I tried to focus on the fact that traditional university research papers are a very specific genre that most students will not use later in their life. In addition, we may also recognize that this genre does not adequately reflect the goals or “spirit” of our courses. Accordingly, we might open up to the possibility that there are other modes and media more reflective of student interests, skills, and real-world needs that still ask them to engage the critical skills we value.

Slide 0.png
Slide 2.png
Slide 1

A limitation of my workshop was the presumption I held for what a “traditional” assignment embodied – this is not the same across disciplines. In planning this workshop I decided to use my disciplinary expertise (as a historian of religious studies) as the norm which often assigns a printed paper meant to inform or persuade a well-informed (scholarly) audience. This is an assignmentt type that certainly resonates across much of the humanities.[2]

By highlighting the various rhetorical elements of genre, purpose, audience, and medium, I suggested during the workshop that the alteration of any of these aspects constituted a move towards non-traditional assessment.

Slide 3

Before we moved on to discussing how we might alter each of these four variables, I suggested that a non-traditional assignment could also verge on becoming “creative” or even “experimental,” a side of the assignment spectrum that should be approached with care. I suppose I cannot dislodge the traditionalist voice ringing in my head (I am a historian of religious studies), but I suggested that truly creative or experimental assignments be paired with a more traditional writing assignment, like a cover letter offering an analysis of the assignment through the lens of course concepts or a reflection on the process of creating it (really, reflection is a good idea with any assignment). Fundamentally, this is a concern over evaluation (grading); we have to remain equitable in the assessment of our projects, an issue which arises as they turn more radically away from standard critical writing (this is discussed more on the handout). Additionally, I argued that more focus has to be placed on process and discussion as the more creative or experimental the assignment becomes. Feedback, necessary in any assignment, is simply more integral when the students are engaging in genre forms that are unfamiliar to them.

Slide 4.png

We then moved into a discussion of the variables for crafting assignments (genre, purpose, audience, medium). Considered individually for heuristic purposes, these elements are actually closely interrelated. For example, changing just one of the variables may be enough to inspire a wholesale shift in the other three. As we discussed these variables, I had the participants fill out the four-field “matrix” on the handout as new ideas came to life.

If I had to give one suggestion, I would say reconceiving the audience functions as a powerful motivation for shifting an entire assignment. Instead of assuming an ill-conceived set of imaginary scholars (or the sole instructor or TA), which new audience could students address armed with their new knowledge? What would an assignment look like if the students had to talk to the general population or to people who held conflicting views on your course materials? What if your students spoke to a local community or to different organizations in your school? Or, perhaps, you could imagine them addressing a historical person or even a living political entity.[3]. Ultimately, the object of this exercise was to let the creative juices flow and sort the pieces afterwards.

Slide 5.png
Slide 6
Slide 7
Slide 8

The last slide addressed some concerns I felt pertinent to exploring non-traditional work. It foregrounds the facts that many students may not have the required skills, money, or free time to do what you hope (or expect) from them. Student familiarity with or simple access to computer programs or the ability to go to a museum exhibit or performance may be stumbling blocks to some. Some of this could be circumvented with good planning.

Slide 9.png

The rest of the workshop was devoted to discussing potential ideas and beginning the process of putting those new sparks of insight into words.

Below is a list of potential ideas

  • Have students draft a lesson plan (LO & assessment) and lead an activity on a concept for the class
  • Have students or instructor create Twitter or Instagram accounts and post regularly on course content (blogs are an “old-school” option)
  • Visit your university library archives or special collections and have students select a document or artifact and give a mini-lecture in class
  • Analyze the different views and approaches on a course-relevant topic by both a scholarly peer-reviewed article and something written for the public (news article, magazine article, YouTube video, etc…)
  • Hold a class poster presentation session (held in the classroom or hallways of your department) and have students comment on one another’s work (written peer-review)
  • Have students create a podcast or video on a tricky concept and use as an instructional aid (remember to keep create a catalogue of past work!)
  • Have students write in a “popular” genre relevant to course materials (e.g. magazine article, pamphlet, poster, newspaper opinion article, letter to the editor)
  • Visit a local site and have students document and analyze their visit or schedule an interview with someone about the site
  • Set up a formal class debate about central themes of the course (randomly assign “for” and “against” teams)
  • Establish a scenario and have students role play figures central to your course
  • Have students craft an annotated bibliography or literature review on complementary aspects to your course
  • Have student create a material object relevant to course material (e.g. alter, home shrine, model, etc…)
  • Compile a list of available public media on a course theme and evaluate the quality of each item’s content and post the evaluations online
  • Create an infographic about a topic or theme, or create a concept map/knowledge map of the course materials
  • Have student keep a course journal where they reflect on difficult topics and plan what they will focus on for next lecture/discussion
  • Design (/and implement) a collaborative on-campus project
  • Have student write a book review of a source read in class
  • Have students interview a professor about their current research (or an influential paper)
  • Volunteer at a local nonprofit or attend a community meeting or group and write about the experience related to course themes
  • Visit a local museum exhibit and have students analyze artifacts according to course themes
  • Atten a local performance and have students analyze experience according to course themes
  • Assign “On-Going Conversations” to students where they talk about a topic and take notes on the interaction
  • Have students do 4-Sentence Papers (They say…, I say…, One might object…, I reply…)
  • Using free online applications, have students create a map and timeline of important course events and figures

Notes:

*This is part of a series of posts reflecting on my experiences offering workshops on university pedagogy. Please contact me directly if you want full versions of my slides pmr01[at]ucsb[dot]edu.

[1] Clearly, knowing the abilities of your students matters. The core of the concern is not so much the selection of works, but the training in reading and note-taking skills which allows students to truly access these works. Developing necessary reading and note-taking strategies is too often overlooked by instructors.

[2] Unsurprisingly, I had math and foreign language instructors attending the workshop, so we also talked about “traditional” assignments in their domains and folded them into our discussions as best we could. It seems some of the strategies we discussed were also effective in inspiring them.

[3] A colleague whose name unfortunately escapes me had students address the famous Chinese Empress Wuzitian and make an impassioned plea for imperial support of either the Buddhists or Daoists!

Asking Thought-Provoking Questions Workshop

Questions drive thinking.

As I’ve argued before, straight lecturing has its rightful place in the repertoir of a university instructor. When we lecture, however, we are often placed in the role of having to answer questions posed by students. An equally important skill is the ability to ask probing questions of students. When we want to stimulate class discussion, having the ability to generate, hold, and gently direct conversation requires mental dexterity and social finesse – built upon a foundation of effective questioning strategies. I’ll freely admit, discussion can be a slog at times. But having a range of questioning strategies can make us better prepared to face the wilds – and natural beauty –  of the discussion-based classroom.

Below, I offer several of the slides I presented for a worhsop organized by my university’s Instructional Development program. I will also provide some commentary and context to the slides themselves.

Using Mentimeter, I first asked the workshop participants “Why should we ask students questions? (Why not just lecure at them all the time?)” This was a not-so-sly way to introduce two different techniques I would discuss: Entrance Tickets (“Priming the Pump,” see below) and the critical rephrasing of questions (“Stirring the Silence”). Nevertheless, I was not expecting such thoughful, insightful replies created on the spot (I should have known better):

Screen Shot 2019-05-01 at 19.28.58.png

After recapping some of the comments (and asking about some of the jargon in the responses), it was clear the room was already actively engaged, which is the purpose of an exercise such as this. After noting the importance of using questions to drive and develop student thinking, we dove into several actionable strategies. Overall, I planned to adderss six different sets of techniques and protocols.

Screen Shot 2019-05-01 at 20.37.54.png

The first group of strategies was oriented around fielding potential discussion topics and  priming the students to think about the day’s material. Moreover, asking questions as soon as students arrive in class creates an expectation that they need to engage and, hopefully, contribute to our discussion (if not indivdually, in small groups).

Slide 2.png

Priming the Pump Strategies

The second grouping of strategies concerned the general phrasing of questions. Several very common forms of questions, while valid in certain contexts, can also sometimes be squealing-brake showstoppers. These are potential problems that need solutions.

Screen Shot 2019-05-01 at 19.40.36.png

Phrasing Effective Questions Strategies

We next moved to a series of strategies developed to provoke more complex conceptual processess. These form a broad category of questions sometimes called Checking Questions or Checking for Understanding Questions. When weilded artfully, these can  turn a rather blasé response into a moment of valuable class reflection. Truthfully, many of us already have these in our repertoir, with the most common simply being the follow-up question, “why?” (under “justification” in the slide below).

Screen Shot 2019-05-01 at 20.05.40.png

Checking for Understanding Strategies I

Screen Shot 2019-05-01 at 20.08.50.png

Checking for Understanding Strategies II

The next set of techniques are used to try and “decenter” the classroom from the instructor. The back-and-forth “yo-yo” pattern between student and instructor is modified to include more inter-student “cross-talk.” In many cases, the instructor still remains the moderator, but that largely depends on the purpose of the discussion exercise.

Screen Shot 2019-05-01 at 20.28.00.png

Redirection Strategies

The last pair of strategies were protocols I’ve casually developed when consulting with other TA’s and instructors about how they handle general classroom unresponsiveness. Specifically, when a question to the class appears to be “dead-on-arrival.”

Screen Shot 2019-05-01 at 20.15.16.png

Stirring the Silence Strategies

Screen Shot 2019-05-01 at 20.15.27

Revivign the Dead Strategies

In the last few minutes of our workshop, we turned to the potentialy awkward social scenario of a “wrong” or “non-optimal” student response to one of our questions (I prepared no slides for this, I preferred to hear what others thought). We discussed some ways that we’ve maneuvered through this scenario, always mindful of not chastising or “isolating” the student. Depending on the type of question, I’ve often found asking “why?” helps to determining the student’s reasoning. In some cases the rationale may be sound, but just not what I am “looking for,” and thus allows me to thank the student (“That’s a good take, I didn’t think of that”) and pivot to another response. Otherwise, if there’s a problem in the rationale, I may try to give clues to see if the student can find his or her own mistake (or problematic assumption) and offer a chance to respond again. Other times, I may ask for another student to provide a counterargument. In the end, I usually try to validate some aspect of the student’s original response, if possible. (Particualrly sensitive or controversial views may need a different approach, but that’s a discussion for another workshop.)

I closed our workshop by handing out “Exit Tickets” – providing a moment of reflection for people to think about something that was meaningful or even unclear to them. I was curious which strategies were the most helpful to my workshop attendees; I received a range of replies. Around half explicitly noted the value of Redirection. Another large chunk approved of the Reviving the Dead, and the rest were evenly split between Checking for Understanding and Priming the Pump (Mentimeter is frequently a hit).

Last Thoughts: This was my last Instructional Development workshop for the academic year and the amount I’ve learned in the process has been quite gratifying. Of the workshops I’ve given, this was probably the most time intensive and interesting for me to think about, as well as the best attended (with Managing Teaching Anxiety being a close second). Is it just coincidence that the two workshops I’ve offered that most directly deal with student interactions were the most attended? Nevertheless, I will now be transitioning into my role as a facilitator for the Summer Teaching Institute for Associates, more workshop planning abounds!

*This is part of a series of posts reflecting on my experiences offering workshops on university pedagogy. Please contact me directly if you want full versions of my slides pmr01[at]ucsb[dot]edu.

Teaching Writing When You Don’t (Want to) Teach Writing

epigraph.png

(Or…Teaching Writing in a ‘Content Course’)

I know. Student writing is horrible. The sky is falling. Doomsday has come. I’ve had several conversations with colleagues about the sordid state of student writing. I once hooped and hollered atop that bandwagon too. It was so easy to blame high school teachers and college writing instructors (or anyone else) for not preparing students adequately in the fine art of written argumentation.

Then, I got a job as a university writing instructor. I’ve come to see that the problem partly lies with me and my curmudgeon colleagues (its a blight that extends beyond religious studies). It’s our duty to teach writing too, we just don’t want to do it.[1] Student writing is also not that bad, if you give students a fighting chance.

I’m a traitor to the “but-I-don’t-teach-writing” cause. I know.

If I haven’t angered you to rage-close the browser tab, then let me explain. At one level, writing is a skill, not a “fact”; all students know how to write, but some may not be proficient at it. They need more practice – and guidance in The Writing Process.

When I say instructors need to teach writing, I really mean they need to apportion time and structure activities that allow students to critique and develop their own ideas. Once students thoughtfully struggle with their ideas and arguments, they are more likely to want to “own” those ideas. As a result, almost magically, their writing will get better. In our religious studies courses (or humanities more broadly), we do not need to formally “teach writing,” as much as we need to give students the opportunity to work out their ideas in a strategic and structured manner. Teaching “writing” is just code for teaching critical thinking skills.

One of the key aspects of my writing courses focuses on (re)shaping student writing habits. We talk at length about “The Writing Process” and students develop their own ideal Process. While many facets are individual – some do their best writing in the library, some in their rooms; some listen to background music, some prefer silence; and so forth – other facets are non-negotiable. My writing course is structured around these non-negotiable facets, namely drafting, peer-review, and revision (see below). Ideally, my goal is to have students incorporate these elements into their natural Process, but like many new habits, they can be hard to adopt.[2]

In the non-teaching-writing classroom, it can be easy to omit drafting, peer-review, and revision activities (hereafter DPR). But these are precisely what students – and us scholars – need to produce our best writing. If done skillfully, DPR can all be handled by students themselves; there is no extra work for our already-exhausted teaching souls. I will admit, however, it does take time to devise drafting prompts, structured peer-review sessions, and direction for revision. Undoubtedly, these activities will go wrong before experience teaches us how to make them right (online research will certainly help too, or, gasp, conversations with colleagues about our failures).

Of course, I do think students benefit greatly from our expert insights on their ideas. If we can provide verbal, recorded audio, or written commentary at early stages in the Process, this will greatly assist students in developing their ideas. Spending time to get students to talk about their research in class with other is also valuable.

Overall, here are some thoughts about incorporating writing good critical thinking habits into your non-teaching-writing-classroom based around DPR.

Drafting: A simple recipe for disaster is to assign a big writing project due at the end of the term which does not also formally schedule several “checking-in” assessments along the way. These formative assessments could be a simple conversation with individual students or classroom activities where students discuss the early stages of their work. Drafting is another option. This does not have to be a full draft of the final essay. I’ve seen colleagues assign small segments of the essay that are due regularly throughout the term, such as drafting the introduction with a thesis, or body paragraphs with solid evidence, or the conclusion with the potential significance of the findings. More commonly, I’ve seen colleagues assign research proposals or annotated bibliographies, which discuss proposed research questions and potential lines of argument. Personally, I prefer (when possible) to assign smaller writing assignments that allows students to work on certain facets of their larger argument for their final paper. This works best if students don’t have an open-ended final research project, but are expected to address a pre-determined topic or theme at the end of the term (but I’m sure savvy instructors could still craft prompts that function in an open-ended manner). Overall, there are plenty of drafting options to help stimulate student thinking along the way, including daily writing reflections, for example. [3]

Peer-Review: It is important to remember that drafts can also be read by other students (even as homework), if reading students’ work is too time consuming for an instructor. Peer-review can be tricky to run effectively, however. One thing to keep in mind is that students will need assistance on how to critique the work of others; offering truly constructive criticism and thoughtful insight on writing is a skill in itself (as I’ve painfully learned as a writing instructor). Consequently, review sessions need to be highly structured. Provide a commentary rubric or the grading rubric you would use for evaluating the assignment. Or, at the very least, provide a list of things for students to pay attention to. I’d suggest saving the best papers from each term and sharing them during subsequent peer-review sessions. The entire class can read and comment upon the paper(s), thus helping to align expectations about the review process. I’d also strongly suggest having student converse with each other after providing written commentary; talking can easily clarify confusions on both sides. There are numerous discussions of running peer-review online, I’d suggest finding a method that might work best with your class and tinker with it.

Revision: The purpose of doing the above activities is so students think through their ideas and potentially see how others attempt to tackle the same or similar issues. Depending on the drafting assignment (suggestions above), you would have to decide how to have students build off of their initial insights. In other words, what is the next step in the Process? Does their thesis statement need to be re-thought (and possibly re-drafted)? Does their research proposal need to be more focused and peer-reviewed again? Is their annotated bibliography missing an important work? Will students’ smaller writing assignments dovetail cleanly into their larger argument or are there potential flaws which need to be redressed? While some of these interventions can be performed by other students, instructor commentary (or even better, conversation) will prove invaluable. Importantly, only work that will be revised merits extensive feedback, there is little pedagogical value in heavy commentary on final products – the real critical thinking work has already been done.

A final note: There are numerous other suggestions to consider as well when focusing on the Writing Process. One that was not immediately apparent to me was “exposing” myself as a writer to my students. I now regularly tell my classes about all of the writing projects I need to engage professionally as a writer: conference proposal, book review, dissertation chapter, journal submission, and so forth. I share with them the struggles, setbacks, and new insights gained through developing my work. It’s important for students to see or hear about writing that isn’t published, that isn’t perfect, that needs further reflection. By seeing the care that goes into our professional work, students can come to emulate that process.

Notes:

*Some of my comments here developed out of my ruminations on abandoing page-lengths for student writing.

[1] Duty might seem like a strong word, but I’ve used it purposefully. If we consider our course goals to include cultivating critical thinking skills, this almost certainly includes (there are reasonable exceptions) some form of writing assessment. As you will see, I do not consider the teaching of writing in the non-teaching-of-writing-classroom to be a review of low-level mechanics or refinement of diction, but the critique and development of ideas that normally happens in the Writing Process. In this case, the teaching of writing is, simply, the teaching of critical thinking.

[2] I’ll say it again, but now as a footnote: So-called “good writing” is not the memorization of facts. Most students are fully aware of the elements of a good thesis or how to make a strong argument. In many circumstances, what limits them is not their misunderstanding of these facts, but the underdeveloped ideas they have. By spending time on their ideas (researching, conversing, debating, refining, reflecting, etc. ) students generally come to care for those ideas. When folks don’t care about their ideas, they hand in “poorly written” work or plagiarize it.

[3] Even regular, short responses to readings can include a few comments by students on their relevance to a proposed final project. In other words, responses to readings don’t always need to be simple summary.

Further References:

Wingate, Ursula; Andon, Nick & Cogo, Alessia. 2011. “Embedding Academic Writing Instruction into Subject Teaching: A Case Study.” Active Learning in Higher Education, Vol. 12, No. 1., pp. 69–81.

Efficient and Effective Grading of Student Writing Workshop

Once, over the coure of five weeks, I graded – and commented upon – just over four hundend student papers. It was mind-blistering work. I was asked by a colleague if this experience passed a cost-benefit analysis. Many of these papers were drafts (or important student reflections), and thus demanded more care; students would be building their ideas off my comments. In the end, their final papers – which received minimal comments from me – were mostly enjoyable to read. My work had paid off. Ultimately, efficient and effective gradings requires the frontloading of teacher effort.

After several years, I like to think I’ve become somewhat better at offering constructive criticism of student writing. It’s never become “easy,” but I think I’ve become more effective in advancing students’ writign abilities and efficient in my time spent on the task. Below, I offer some of the slides I presented on this topic for our university’s Instructional Development program for TAs and add some additional commentary.

Our workshop was fairly small and thus I wanted to start by survying our group’s attitudes towards elements of the writing process. Using Mentimeter, the first question asked the worksohp attendees to rate their opinions of the importance of drafting, peer-review, and creating rubrics:

MMQ1

The numbers above represent the averages of the individual responses (“5” being the most “necessary”). Of those, creating grading rubrics was deemed the most important among our group. Happily, this aligned with the workshopping component of my presentation. Drafting and peer review require some “experice” (ahem, failures) on the teaher’s part to get it “right.” Nevertheless, I consider all three to be closely related, I’ll return to this below.

The second question asked the participants to do a cost-benefit analysis of creating rubrics, setting up peer-review, and giving ample commentary:

MMQ2

Not surprisingly, providing feedback was the most time-intensive, but it’s value was on par with crafting a good rubric. As I noted above, there’s an inverse value to feedback as the semester progresses. It’s most valuable early in the term, when students can adjust their habits and styles (and build their ideas); there is minimal value on maximal feedback at the end of the term.

The final question was more straightforward: how long does it take to read, comment significantly, and grade a five-page paper?

MMQ3

I asked this question to get a sense of how others operate – we instructors often don’t talk about these types of things with each other. At the very least, I think its important to have an internal estimate of our grading times so we do not go overboard with commenting. Personally, I have not been able to break the 15 minute barrier for five-page papers. I average about 18 minutes. As such, I set a timer for each paper I read at 20 minutes and always try to “beat” it. (Perhaps I can call this a variant of the Pomodoro Technique.)

Before diving into my presentation formally, my favorite suggestions for managing paper load comes from Shelley Reid’s insightful thoughts posted in her “Shelley’s Quick Guides for Writing Teachers.” Many of Reid’s principles are sprinkled into my presentation here.

I start by asking why instructors should assign papers at all. I think its important to keep in mind the value of writing in that is often recruits “higher orders” of thinking, which are all but impossible to access through multiple choice exams. It is also important to think about which orders of thinking writing prompts address; some writing prompts may only ask students to list elements of a concept or theory. This remains in the lower order of “remembering” (see Bloom’s Taxonomy below).

Slide 1.png

There are several things we instructors can do to off-load more of the conceptual “heavy lifting” to students – and thus have them build more of the conceptual “muscle.” Having students draft is an important step in the writing process because it allows them to develop (and become more invested in) their ideas. If you pair this with a structured peer-reivew, there is actually minimal work for the instructor. (Of course, anyone will tell you that peer-review requires a lot of structure and guidance. Students need to practice and learn the skill of truly constructive criticism. Perhaps I will run a workshop on the practices of drafting and peer-review in the future…)

I offer the next few slides with only minimal comment.

Slide 2.png

Slide 3.png

 

Slide 4.png

“Before you Grade” considerations

 

After reviewing the basic components of grading rubrics (criteria, description, scale), we spent time lookign for relevant rubric templates online – there is no need to re-invent the wheel! There are many resources available that can inspire your rubric divisions. I provided the following handout for consultation: Creating Grading Rubrics Handout.

After discussing some strategies “before you grade,” I swith to pragmatic suggestions “while you grade.”

Slide 5Slide 6

Slide 7.png

“While you Grade” considerations

 

I have long been a covert to audio commentary and have suggested it to many of my colleagues. If you have the space (the one limiting factor is you need a generally quiet locaiton), it’s worth a try. Finally, I ended with a few thoughts on “after you grade.”

 

Slide 8.png

“After you Grade” considerations

 

Slide 9.pngSlide 10.png

*This is part of a series of posts reflecting on my experiences offering workshops on university pedagogy. Please contact me directly if you want full versions of my slides pmr01[at]ucsb[dot]edu.

Edit: This post below addresses several similar issues, but adds other interesting insights: https://movingwriters.org/2018/07/16/sy-2017-2018-top-ten-in-pursuit-of-meaningful-feedback/

 

On Readings and Reading – and Double Entry Notebooks (Pondering Pedagogy)

[Part VII of the series Pondering Pedagogy: Course Design; Read Part I, II, III, IV, V, VI, VIII, IX, X]

Is it preposterous to claim that you will teach a university student “how to read”? At one level, absolutely. Students have been reading for quite a while and are skilled at digesting and summarizing information. But, reading for information (and its close analogue, reading for comprehension) is only one way of reading, and one that serves foundational, but also minimal, purpose for classes I teach.

Yet, not all readings are equal. Some texts are seemingly impenetrable even for scholars, and reading to simply comprehend a text might be challenging enough for students. Thus, selecting appropriate readings and instructing students how to read them are important steps in the pedagogical “decision tree.”

For survey classes like the one I’m designing currently, introductory textbooks are a clear option. They are effective at delivering a broad range of ideas and terms in easily comprehensible language, and sometimes they are outfitted with convenient illustrations, maps, and charts. Yet, there are limitations to textbooks. Oftentimes, they do not include sufficient primary sources or documents (important if you are taking a historical approach) and due to their style, they tend to be pretty boring reads. Related, and often overlooked, the genre of textbooks are written as broad summaries, and as such do not function a good rhetorical models for student writing. They frequently present data as neutral facts, overlooking the (more interesting!) scholarly debates that breathe life into the field. Consequently textbook authors do not write in an argumentative style that builds to a thesis derived from evidence.

Admittedly, this may not be a concern for many, but I’d prefer to have students engage the genre of writing I’m hoping they will practice writing. I’ve come to think of textbooks as reference handbooks that function in a very similar fashion to good encyclopedia articles – informative, but uninspiring.

Other options include more formal scholarship, such as journal articles, chapters from anthologies, chapters from monographs, or even entire scholarly books. These will typically be analytically focused (and I’d argue, more interesting) and generally model good argumentative writing. Of course, these texts are not written for novice scholars like our students, and at times will assume knowledge our students simply do not have. But I think we should recognize some authors and topics are more welcoming than others, and I do not think this concern should eliminate readings such as these out of hand. In many cases, a good introduction to the text could greatly assist in comprehension and encourage students to do any other necessary background research (Wikipedia will often suffice). Providing guiding questions cna also help. I would suggenst that instead of asking questions that are merely fact based (“What does author X say about Y?”), one could make eplicit (the often implicit) research question the scholar is trying to answer.This will frame the work in the context of the discipline, and suggest where holes in knowledge exist(ed).

Regardless of selected readings, I would be remiss to not discuss on open secret – students do not read, and at best they skim for what they need to know, when they need to know it (like right before a test). Research has shown that 80% of students read the reading assignments in 1981, but that number was down to 20% in 1997. Honestly, I am not sure what to do with this hard reality. One easy way to coerce students to read is to have regular (or surprise) tests on the readings or have them write or summarize the contents. I tend to do the latter, but it is relatively easy to “game” either of these scenarios in one way or another. Another solution is to assign more assessible and interesting assignments such as documentaries, podcasts, and journalistic articles, interspersed with more difficult academic texts.

We should remember, that whatever we assign should be in alignment with our course outcomes and – most importantly – our means of assessment. This matters because students will read in a manner that best prepares them for the type of assessment they receive. If we provide multiple choice exams and “list-the-facts” essays (which, ahem, does not engage higher-order thinking), then students will only read for content. They will, understandably, takes notes on facts, figures, dates, and so forth. While this is valuable, it does not develop any critical reading skills. I would argue that in addition to reading for content, a student should also read more critically.

In fact, scholars read in several ways simultaneously, gathering facts and data, forming opinions, challenging old idea, critiquing argument, and so forth. This is something I want students to cultivate as well. One strategy to help student read more critically is to instruct them to take notes differently and keep a dialectical notebook or a double entry-journal. In terms of the latter, students take notes in one column and in the parallel column they write down their response to that information. This is often “connections” or relational knowledge – how did that fact make you think about something you already knew, about something you see in your life or in the world, about something you are learning in another class, or about something you saw in another reading (text to text, text to self, text to world, etc.). It is foremost about making a network of meanings and ideas connected to one another.

Grist for the Mill: Choosing an good reading is only part of the puzzle, students also need to know how to read that text. It is important to be clear in what purpose the reading serves for your course. Why do you want students to read it? Ask them questions along those lines, instead of questions that simply cover content. Instructing them on double entry notebooks can facilitate a move away from simply skimming for facts. I have found that assigning a large writing project due at the end of the term can function as a lens for reflection throughout the term. If you assign students daily reflective writings on the readings, a portion of that reflection can be oriented towards thinking about how that text may help build part of an argument for that final paper.

*This is part of a series where I discuss my evolving thought process on designing a new university course in Religious Studies. In practice, this process will result in a syllabus on Japanese Religions. These posts will remain informal and mostly reflective.

Expectation, Assessment, and Alignment: Drafting Learning Outcomes (Pondering Pedagogy)

[Part VI of the series Pondering Pedagogy: Course Design; Read Part I, II, III, IV, V, VII, VIII, IX, X]

This is where the rubber meets the road, when push comes to shove, and, er, where dreams come to die?! I apologize for my string of worn aphorisms, but so far, my series of posts have remained firmly in the realm of the conceptual. Now we must start turning practical. I’ve outlined some of the reasons why I prefer to focus on skills (or “critical thinking”) than on content (that will be refined soon), why it’s important to explicitly instill a sense of vitality in the humanities for undergraduates, and summarized several pedagogical approaches to teaching a religious studies course.

These general orientations will provide some of the direction for my learning objectives (see below) and will form the basis for my expectations for the course. A potential difficulty arises when we start considering what expectations students have when they walk in the first day. After reading about Gearon’s religious studies paradigms, I think it important to keep in mind that students may have different expectations; this forms one level of alignment. To refer back to Gearon’s paradigms, students may be operating from a theological, psychological, or even philosophical perspective while the instructor may assume a historical or sociological one –  it is the duty of the instructor to make it clear what is expected throughout the course.

In addition, as studies in instructional design have shown, the type of assessment will determine if the course objective are realized.[1] If the assessment tasks draw upon lower forms of cognitive activity than the course objectives (for example, assigning multiple choice tests when assessment or creation are the objectives), there will be a learning equilibrium achieved at a lower level. Proper instructional alignment needs to be conceived and deployed.[2]

I should add that alignment plays a role in several other contextual factors, including alignment with the university mission, alignment with broad departmental goals, and alignment with other courses. Ideally, a series of courses should build upon one another, deepening networks of meaning and skill cultivation.

I made a draft of my learning outcomes (LOs) below. They borrow the language and general structure of Bloom’s taxonomy, ranging from knowledge and comprehension to synthesis and evaluation. In addition, I consider classroom discussion and debate to be skills that need explicit cultivation (and thus proper teaching/learning activities or TLAs).

  • Identify and describe principle beliefs, practices, themes, or contested issues in Japanese religions, including Shinto and Buddhism (and their relationship), and other religious traditions
  • Identify and apply historical critical methods to explain the political, social, or cultural foundations for religious practices and beliefs in Japan
  • Analyze visual and material objects related to religions in Japan (based on the above skillsets)
  • Formulate a critical evaluation of the relative eminence and/or transcendence of Japanese religions
  • Communicate – and even debate – with your peers through in-class, out-of-class, and online discussion and through written assignments

I am pleased with this draft of LOs because I feel they clearly connect to potential assignments and projects (TLAs)[ideally, each outcome should only have one measurable verb, but my draft has more]. This was not accidental. I spent considerable time considering the projects I wanted my students to complete before crafting these outcomes; in other words I aligned them.

The skills of identifying and describing basic religious ideas will be developed through lecture materials, including audio-visual media where possible, and readings. I envision students identifying their own terms, figures, and concepts and constructing a bank of vocabulary for their use on low-stakes exams (a modicum of self-determination). Provisionally, I imagine students working in small “study groups” (a sublte homage to Ryuchi Abe’s analysis of Nara Buddhism) outside of class to identify central terms for that week’s readings (they can meet in person or coordiante through email or texting).[3] One group will be assigned to meet with me to discuss and analyze their choices and place them on a class-accessible website. I imagine both a temporal (Tiki Toki) and spatial (Google maps) component to this work.

Because I am primarily operating through a historical paradigm (although I hope to do class activities that are philosophically based), I will train my students in basic historical criticism so as to understand religion as a human phenomenon (and domain of the humanities). The first day of classs will introduce this framework, and likely draw from the distinctions made by Bruce Lincoln.

One particular expression of this historical practice is to analyze and assess visual and material objects related to religion. One reason I incorporate this aspect is to take religion from the domain of belief and place it firmly within human cultural practices. This is also tied to the next outcome which will focus on the eminent, practical, and mundane aspects of religion. Provisionally, I am thinking of having students select items from e-Kokuho (e国宝) website which lists various National Treasures and Important Cultural Properties of National Museums and have them write a cultural biography or object narrative of an item, possibly as a foundational element for their final project.

I am also hoping to incorporate a term-long writing project where students explore the more mundane aspects of religion and have them assess the “worldly benefits” (genze riyaku) often associated with Japanese religions (and consider if all religions are primarily concerns with the eminent or transcendent).

Finally, I hope to design in-class activities (one, two, three?) where cross-talk occurs and philosophical stances are taken, elaborated, and debated.

I did not put it explicitly in the LOs, but I also plan to teach students how to effectively read in our class (reading for facts, reading to analyze method/argumentation, reading to connect ideas; as well as realizing the different genres of textbooks and scholarly works) and effectively take notes (double entry journal). I will return to this in a later post.

Notes:

*This is part of a series where I discuss my evolving thought process on designing a new university course in Religious Studies. In practice, this process will result in a syllabus on Japanese Religions. These posts will remain informal and mostly reflective.

[1] Many of the ideas discussed throughout were derived from Biggs 1996. A great discussion can also be found at the Scholarly Teacher blog, here.

[2] Biggs 1996: 350.

[3] Because I take a constructivist perspective, I hope to encourage multiple socially constructed ways of learning, including peer-controlled activities, in addition to teacher and individual controlled activities, See Biggs 1996: 354-5.

References:

Biggs, John. 1996. “Enhancing Teaching through Constructive Alignment.” Higher Education, Vol. 32, pp. 347-364.

Should We Abolish Page-Lengths When Assigning Student Papers? (Pondering Pedagogy)

[Part IV of the series Pondering Pedagogy: Course Design; Read Part I, II, III, V, VI, VII, VIII, IX, X]

For those of us who are writing, or have written, a dissertation, it is likely that we have run across a well-meaning friend, family member, or stranger in a bar (or Buddhist temple!) who has asked us, “How many pages does a dissertation need to be?!” “Everyone knows,” I say, “it’s exactly 373, no more, no less.”

Of course, this is a silly answer to a genuine question, but I think we scholars have to admit the assumptions behind this question are, well, misdirected. I’ve seen dissertations in my field that are 200 pages and 1200 pages. I would say age-old conventions more commonly determine page length of dissertations, not whip-cracking advisors.[1] More importantly, the guiding principle behind writing a dissertation is identifying and crafting a compelling argument. The scope and depth of the argument is better at determining the length than any pre-established page length. I would say that most scholars do not have the problem of developing the length of their writing, but containing its scope of ideas! Word limit is our enemy, not word count!

In teaching academic writing, my colleagues and I foreground the importance of developing ideas through short writing assignments and drafts. Yes, we do have page-lengths, typically three 5-page papers are due throughout the term. But each of these 5-page papers were preceded by two 2-page papers, used to develop elements of the longer argument. This structure allows students to develop ideas over time (they receive critical feedback along the way) and draft verbiage that can be used for their longer paper. Even these initial 5-page papers are drafts that are revised and edited for their final portfolio.

The first time I taught the course I was caught off guard by how many portfolio papers were too long, either repetitive or unnecessarily broad or meandering. They blew by the 5-page limit because they had too many good ideas![2]  I soon established page-maximums for the portfolio papers (7 pages), and decided to start talking throughout the course about how students should think about the scope of their thesis and the depth of their evidence and analysis. Surely, some initial 5-page papers are short, but I have an equal number of students who are struggling with too many possible trajectories for their argument. They are no longer concerned with page length, but with identifying and crafting a compelling argument. To be clear, this only happens because they are allowed (err…coerced) to developed ideas through writing over a period of time with critical feedback. I commiserate with students who are vexed by having to edit out good (or just fun) ideas – I tell them that is what real scholarship often entails – save them for your next paper!

Now, I can hear (and have heard) the objection, “But, I am a scholar of Religious Studies, I teach Religious Studies, I do not teach writing!” True, but I would suggest we rephrase the statement like this: “I am a scholar of Religious Studies, I teach facts, not skills!” I would argue that writing (or argumentative writing) is a core skill of doing Religious Studies, and if one wants to develop critical thinking, the practice of writing is a main vehicle for its development. If one is truly interested in teaching only religious literacy (i.e. facts), then writing assignments do not need to be assigned – multiple choice exams will adequately test retention of ideas. (And don’t fool yourself if you think assigning an “essay question” necessarily elicits higher-order thinking – some essay prompts ask the student to list facts in one way or another. For example, a good essay question should ask the student to apply a skill in a novel context.)

Grist for the Mill: By writing this (go figure!), I think I’ve come to a better understanding of the value of page-lengths. Page-lengths really don’t matter much if you are working with a motivated student to continuously help develop his or her ideas. Depending on the depth of available research materials (both primary and secondary sources), the complexity of the research topic, and the base knowledge of the student, paper lengths may vary significantly, but still reflect an equivalent effort. The point, I concede, may be simply placing the primary focus on – and underlying motivation directed towards – the quality of the argument and not the length of the paper. One way to do this is to emphasize the need for the slow development and building of ideas with regular peer or instructor feedback.

Dare I say, page-recommendations should perhaps be determined by the time a teacher has to critically read and comment on them all – as opposed to some arbitrary 5, 10, 12, or 15 page paper?[3] I average about 20 minutes of reading and critical feedback per 5-page paper. This feedback (and sometimes this is just suggestions, not “corrections”) is then used to write a final, more polished piece of writing. This process requires more planning than simply penciling-in a “research paper” into your syllabus due at the end of the term. Is also requires a prompt that allows for the development of a student’s ideas over an extended period of time. Granted, this may not be ideal for every class, but I think it is preferable in many Religious Studies courses.

Notes:

*This is part of a series where I discuss my evolving thought process on designing a new university course in Religious Studies. In practice, this process will result in a syllabus on Japanese Religions. These posts will remain informal and mostly reflective.

[1] If I engage in a longer conversation, I will usually say that scholars often think about the number of chapters needed to develop an argument, and that by the general concern of publishing a chapter or two as a 10-20+ page article we often arrive at a determined dissertation length. In other words, we (typically) do not start by having a page length.

[2] Long papers could be plagued by what I call “stream of consciousness writing,” writing that meanders with no point. I usually tell students that this is the sign of a good rough draft, but now needs to be critically analyzed for kernels of ideas or good argumentation. Unless a student willfully wants to fail, this informal style of writing is almost always fixed after revision.

[3] Many universities have stipulations for certain classes that involve the required assignment of a certain number of pages of writing (it is 15 pages at my university). Typically this represents total pages of writing, thus, following a similar program I outline above, a final 10-page paper should easily be preceded by more than 5 pages of drafting, consequently complying with the 15-page minimum.

Zen and the Art of Multiple Choice Exams

Scantron scoring machines are fairly common, if not universal, on university campuses. Thus one can infer that multiple choice exams are equally widespread. But how do they stack up against other kinds of assessment, especially in classes where higher-level thinking (application, analysis, evaluation), and not mere memorization (recognition and recall), are emphasized?

Crafting good multiple choice (MC) stems (questions) and conceiving of plausible distractors (wrong answers) are skills in themselves. Yet, when artfully done, MC exams can be effective in testing higher-level cognitive abilities – it just takes time and effort.

It had dawned on me a few years earlier that it would benefit students to shoulder some of these question making responsibilities. Coming up with plausible (dare I say artful?) distractors is part of the practice I so enjoy about writing MC exams. I try to think of the reasons why a student may choose a wrong answer (is X conceptually close to Y?, is it easy to confuse X for Y?, is X actually the opposite of Y?, does X actually negate Y?, is X spelled similarly as Y?, and so forth), and include those as the distractors.

Indeed, through this very process of crafting good questions with good distractors I come to have a better understanding of the relationships between concepts, in other words, I built a strong network of reinforced meanings. This is precisely what I want students to do!

Thus, I decided for my Zen class to unveil a new extra-credit option: for student to craft five of their own MC questions following the criteria I set out for them. From my perspective this had the following benefits:

  1. Students would create their own web of meaning between conceptually similar or confusing terms. This was a coercive way to get them to study in a new manner.
  2. Students wold post their MC questions on an online forum, and thus would get the opportunity to take several practice tests.
  3. By selecting several of the best MC exams, students would feel they had a sense of agency in the learning process.
  4. It might save time on my end from having to craft so many MC questions.

Overall, about 50% of my class took me up on the offer. Many of the MC questions did not match my highest standards (but, honestly, not all of my questions are perfect!), but several were thoughtful and well-crafted and were incorporated into our exams.

Something I would consider doing in the future is to have students explain why they selected each distractor (laying out the relationships I noted above).

Taking multiple choice exams will never replace the value of writing well-reasoned prose, but having students write well-reasoned multiple choice questions is a step in the same direction.

Final Writing Project

My summer course fulfills the UCSB writing requirement for undergraduates, thus I knew my class would be writing intensive. I conceived of the final writing project as following a progression of “staging and scaffolding.”

My final project required students to develop their own definition of religion based solely on the Asian religious traditions we covered in class (it is presented as a challenge to the understanding of “religion” as based on monotheistic Western traditions which emphasize “belief” and “holy books”). This would require students to engage higher levels of thinking with the material (including analysis, evaluation, and creation), far beyond just memorizing foreign terms and names. It also required them to construct a written argument. Both of these goals require staged assignments, and my course was developed with these outcomes in mind.

There were three basic stages building up to the final writing project.

  1. Daily Reflections: In addition to my N-O-O assignments, students had to write and post daily informal reflections on concepts that they could use for the foundation of their definition. The concepts, which I called “threshold concepts,” were introduced through short podcasts, and each student had to locate where these concepts were found in that day’s reading assignment. They were also asked to reflect on where these concepts may have been illustrated in other materials as well, thus creating a network of comparisons. These were informal low-stakes writing tasks which focused more on critical thinking and idea development than formal writing structure. These daily ideas formed the beginnings of an argument for how a religion might be defined. Students also had to opportunity to read the posts of other students, thus increasing the cross-pollination of ideas.
  2. Idea-Generating Writing Project (Mid-term Exam Essay): For the midterm essay I had students take a definition of religion we looked at or created during our first class and either defend it or critique it according the the material we had covered thus far (we had covered “Hinduism,” Jainism, and Buddhism – or the Indian cultural sphere). They had to employ one or more threshold concepts as the basis for their argument. This was envisioned as a conceptional rough draft for their final project which required them to create their own definition. They had to bring in a copy of their paper for peer review and I provided audio commentary on what they turned in. I required this to be a formal academic essay, and gave insight into the construction of the thesis statements, the use of evidence, and organization of ideas. While some of the content would vary, these concerns would carry over into the final project.
  3. Final Writing Project (Final Exam Essay): This essay was the culmination of student efforts to understand and create useful comparisons among the rage of Asian religious traditions we covered. I made the essay due a few days after our in-class final exam so they could spend time referring to class and teacher commentary and to incorporate the East Asian material we had covered since the midterm. To make sure they remained on pace I required them to draft a definition of religion and post it to our course website a little more than a week before the paper was due.

Peter Romaskiewicz.pngFinal Paper Prompt