Assessment


Talking about assessment is important for any community of teachers, particularly when they are attempting to give a course a certain identity. But talking about assessment is difficult for a number of reasons: assessment is difficult, teachers have often developed their own implicit and explicit notions of "good" and "bad" work, and conversations such as this must be both philosophical and concrete. We want to present a framework for assessment here for the pragmatic reason that if we are asking students to think and act (write) in certain ways, then we must be sure we assess them accordingly. While we don't intended to dictate assessment methods, particularly to veteran instructors, we would like you to consider the following issues and methods when approaching assessment in Business Writing.

values

Since assessment is all about values, it probably makes sense to start here. One of the best ways I (Jeff) have found to talk about a hierarchy of value in writing-particularly in terms of this Business Writing curriculum-is to use language taken from the writing center literature. It is common in writing centers to talk about "higher order concerns" (HOCs) and "lower order concerns" (LOCs) as a way of creating, in some cases, a triage system for focusing a tutorial session. Given the limited amount of time in a tutorial, what should the tutor and student focus on? Given the assignment isn't due in ten minutes, the HOC/LOC system helps the tutor focus on more important issues and save the less important issues for a second tutorial. Of course, this assumes you are assessing a student's work at draft stage.Simply put, the system is divided into rhetorical and mechanical concerns.

Rhetorical (or HOC) concerns cover the following issues:


Focus or control of the document with respect to purpose
Development for audience and situation
Organization that makes sense given the purpose,
audience, and situation

Mechanical (or LOC) concerns cover the following issues:


Grammar/spelling
Mechanics
Usage

The thinking here is likely obvious: it matters little if a student has a mechanically perfect paper (or memo) if any number of higher order concerns are a mess--the document is a mechanically perfect disaster. In business writing, we want to focus the bulk of our energy in the classroom and during assessment on these rhetorical concerns. This is not to say that we want to abdicate all responsibility for mechanical concerns (see below). Rather, this is to say that we recognize the importance of rhetorical issues, and we therefore want to judge our students first (but not exclusively) on the basis of these issues. Along these same lines, we want to encourage a loose form of holistic response and evaluation of student writing. We use the term "loose" because holistic evaluation is actually a term that covers a range of quite detailed and sophisticated evaluation methods (e.g., checklists and likert scales). In general, we would like to encourage you (if you don't do this already) to respond to writers in terms of their development, not just individual texts in terms of their merits. As a value statement, we would like you to see your work as developing writers, and therefore, see the texts students produce as artifacts of their development. Thus, respond to texts in terms of your knowledge of your students' development as writers-in terms of how an individual text relates to a student's growing abilities as a rhetorical and mechanical writer. To be more concrete, we would encourage you to respond to a document in such a way that the bulk of your comments praise successful rhetorical skills while noting those that are less successful, with the rest of your comments similarly noting mechanical improvements and marking continued problems. See each document as part of a continuum of a writer's growth in the class. While your response may shift focus between drafts and final versions of documents, you can indicate how the writer is progressing both within the project and the course as a whole. The sense of holistic evaluation we are pushing here has to do with how you see your work (the development of writers), how you see student texts (as traces of a continuing process), and how you respond to them (focused on HOCs then LOCs within the context of a student's growth and development).

ways of responding: how, to what, & when

Many of you are experienced teachers, so forgive us (here and throughout) if we repeat what you already know and do. We still would like to cover some issues related to responding to student writing, particularly in terms of responding quickly and effectively.

Most of the research on revision took place in the early to mid 1980s, and little has been done since. While the research is old, its insights aren't, and we think they are worth repeating. Some of the best advice/reminders come from the work of Nancy Sommers ("Responding to Student Writing." College Composition and Communication 33 (1982): 148-157):

1. Teachers often respond in confusing and contradictory ways. By this, Sommers means that teachers give contradictory advice (e.g., edit and expand), and the advice is often given without reference to a "scale of concerns." The two issues are linked. Students hear their teachers say "edit" in one part of the document and "expand" elsewhere and are sometimes confused, sometimes unsure of what "edit" and "expand" mean, and sometimes unsure of which is more important. We hope you can take care of the first two problems by not giving contradictory advice and by explaining your use of terms. Our use of the HOC and LOC framework is meant to take care of the third-to give you and your students a "scale of concerns" (HOCs first, then the LOCs). Such a framework should give your students a sense of value.

2. Most teacher's comments are not text-specific and could be rubber stamped from document to document. By this, Sommers refers to the "handbook advice response" (our term). This type of response is the vague directive: be concise, avoid clichés, be specific. Instead, respond to the text in terms of its audience and purpose: What does "concise" mean in this regard? Are some clichés effective here? What "specifics" might be appropriate for this audience. If we want our students to be rhetorically sophisticated, they we must respond in rhetorically sophisticated ways as well.

3. Offer strategies and suggestions, not rules. A bit of advice from the article that we think is helpful and fairly straightforward. Instead of the handbook response, which is almost always overwhelmingly negative, provide a suggestion, which is both corrective and proactive. A suggestion lets the student know that what they have done is likely not effective, and that you can give them some advice about how to think more effectively about the rhetorical problem they face. In other words, offer positive strategies for revision and suggestions about what to do, not simply commentary about what they did wrong. Again, we think Sommers's advice is sound and provides a loose framework for how to respond. We also think the HOC and LOC system, which will change for each assignment, provides a "what" for response (to the rhetorical and mechanical concerns you specify for each assignment). In short, we suggest that you respond concisely (a few sentences) to students about their writing and development as a writer in terms of the HOCs and LOCs for a given assignment. Paper load is an obvious problem for teachers of writing classes, and we will compound this problem by asking you to look at drafts (if you don't already). Of course, you respond differently to drafts than you respond to "final" products. In general, we spend the most time responding to drafts (where the comments are more "formative" and geared toward development) and significantly less time responding to final products (where the comments are "summative" or evaluative). Therefore, we would encourage you in your course plan to give yourself more time to look at drafts, and then take the time to respond more fully to them (again, though, only a few sentences with respect to the HOCs and LOCs you have established for a given assignment). When you respond to final products, provide students with only a few sentences that explain the strengths and weaknesses of the work as a brief justification of the grade. If students want more feedback, particularly if they are interested in further revision, make them come to you in person.

teaching mechanics and usage

Mechanical fluency is important, and while we have consciously tried to place its importance within a larger framework, we insist as teachers that our students produce mechanically sound work. What follows here is one way to handle teaching mechanical concerns; we would certainly welcome others.

Students are given the primary responsibility for the mechanical fluency of their writing, and we think this is particularly important given that Business Writing students are often at the end of their student careers. Typically, this means the following:
As a teacher, establish with each student early in the
semester the problems he or she is having. This is usually
done through written responses and short conversations.
As a teacher, point out to students, both individually and
collectively, where in their handbooks or other resources
they can find guidance for correcting the problems they
have. If there are patterned problems across the class,
these issues make for good "mini-workshops" during class
time.
As a teacher, point out to students where in they text they
have a certain problem (usually done in shorthand or with
symbols in the margin of the text near the mistake).
As students, they should be responsible for finding and
correcting their problems.

We can discuss in detail how to do this, but we think our approach is fairly clear. Students typically only have a few (3-5) mechanical problems; however, they tend to make these mistakes repeatedly. Our goal is to help these students become independent writers (you are likely their last writing teacher), so the practice we suggest is to point out problems, show them how to fix them, and then make them responsible for doing so.

portfolios

Portfolio assessment is consistent with our approach to the class because it allows teachers to assess students as writers (as opposed to discrete pieces of writing), and it encourages revision. Therefore, students typically get the chance to be evaluated on the basis of their best work, and often, that work is quite good.

Portfolio assessment can be done in a variety of ways. When integrated within the framework of a writing class, all student work over the course of a semester is written and rewritten to be included in a final portfolio. We don't think Business Writing lends itself to this type of implementation (although it is certainly possible and perhaps desirable). But we want to encourage you to think about project portfolios as one form of assessment (portfolio assessment typically involves students completing one portfolio for the entire class; project portfolios cover one project only, so students may complete 2-3 of these during a course).

The project portfolios are quite simple. Students produce documents within the framework of a project, but they are not evaluated until the final project is due. To use the United Drill case (UD) as an example, a student would write memo drafts that you would respond to. A student would include these documents, your responses, and their revisions in a portfolio that culminates in the final recommendation memo. You then evaluate the portfolio in terms of a student's ability as a writer within the framework of this assignment. For the UD case in particular, it was only when one of us (Jeff) went to portfolio assessment that he got significant and effective revision. The key is to develop a rubric for assessment of the portfolio based on a specific articulation of relevant HOCs and LOCs (We have neglected the development of concrete, specific rubrics for assessment, but none of what we have discussed works unless teachers to this. We hope we can do this collectively as the semester progresses).

assigning grades

Before you begin thinking about how to assign grades, you should think about when it is appropriate to do so. Not every document that passes across your desk needs to be graded with a letter or numerical value. Our goal is to help our students develop as writers by providing them with ample feedback on their writing. Grades are necessary to that process, but should be assigned with care. Brainstorming, planning, and drafting are crucial skills to a writer and we may want to encourage experimentation with these stages of a project by responding to students with qualitative feedback rather than quantitative grades that focus on concerns appropriate to finished products (such as mechanical concerns). By the same token, we want students to know that the drafts and planning documents they produce are important and valued by us (and therefore carry some weight in their course grade as a whole). Using portfolios for grading can help distribute the weight of grades to more parts of a project than the final polished product alone. In addition, choosing to comment alone on some documents and using a mix of pass/fail and letter grades may help. One of the most difficult parts of a teacher's job is often assigning grades. Grading requires sophisticated analytical skills to be able to factor in product and process for a project, in addition to thinking about the student's progress as a writer as a whole. There is also debate over the most appropriate form of grades to assign to writing-holistic letter grades or numerical grades being the chief choices. We must recognize that grading, particularly of writing, is not strictly objective. Part of what a seasoned teacher does when assigning grades is to factor in their knowledge of the student's work with their own writing expertise. However, we can work to ensure that grades are assigned fairly within the context of our classes. Starting with clearly articulated goals will facilitate this process. You should articulate goals for your course in a variety of ways. First, broad course goals should appear on your syllabus and should be discussed during your class. We usually start by talking about these goals on the first day of class. You need to revisit these goals during the course, however, to ensure that students know they are focuses for the course. You also need to articulate clear, specific goals for each assignment. This can be done through a handout for each assignment (when the work begins and before final products are turned in) and verbally in class. It is these specific goals that should guide your grading when the time comes. If students know the goals for assignments and you have discussed ways to successfully achieve these goals in class, then students should have a clear picture of how you will assess and grade their work. Another important tool which will help you assess and grade assignments more fairly is to request student participation in grading. This may range from having students give you feedback on projects to having them assign tentative grades to their work. Personally, we feel that the final responsibility for grading is a teacher's, therefore we do not have students assign grades to themselves or each other (in group situations). But we feel strongly that a student should give us input on their work. This not only helps the student to be analytical about their own work, but it also helps us understand the project better. In many cases, it is amazing how much the student's comments reflect the course goals and our own evaluation of their work. The method you choose to use for grading is your choice. Most writing teachers feel more comfortable using letter grades instead of numerical grades since writing lends itself to holistic assessment more readily. If we are concerned with their overall writing skill (and not simply the error count for grammatical concerns), letter grades are often easier to assign.

 

Home | Teacher Resources | Course Offerings | Internships | Feedback | Technical Writing Sources On-Line | GSU Homepage | Department of English RhetoricWeb | Library Resources