PROCEDURE FOR RECORDING RATINGS

Essay ratings are recorded on a machine-readable rater sheet that is inserted in each essay. Each rater sheet has a pre-printed code that is used to match the rater sheet with another sheet that identifies the student who wrote the essay.

The rater sheets are designed so that the scanning machine can determine the ratings given to the essay, but raters cannot see the rating given by other raters. There are 16 different forms of the rater sheet with the numbers indicating the ratings in a different order on each form. After a rater records a rating on the sheet, he or she marks through the numbers indicating the ratings so that the next rater cannot tell what rating was given.

Instructions for using the rater sheets and a sample of a completed rater sheet follow.


INSTRUCTIONS

Obtain your rater number from the Honorarium List that is circulated at the beginning of the scoring session. Write this number down. If you lose your number later, ask the coordinator for the list.

Handle the essays carefully so that the rater sheets do not become separated from the essays. The pre-printed code on the sheet will be used by the computer to determine which student wrote the essay. Therefore, it is crucial that the rater sheet remain with the essay to which it belongs. (Social security numbers will not be used to identify students. However, the social security number is written on the back of the rater sheet so that raters can verify that the correct sheet is inserted into the essay.)

Follow the procedures indicated on the attached photocopy of the rater sheet and illustrated by the example of a completed rater sheet. Note that the procedures differ slightly depending on whether the essay is being rated by the first, second or third rater.

The following directions must be used for marking the rater sheets:

Use only the pencil provided by the coordinator. Pens or other pencils may not be read by the scanning machine.

When "bubbling in" circles, make each mark heavy and black enough to completely fill in the circle.

Erase completely any mark you wish to change.

Make no marks on the sheet (front or back) other than in the designated areas. Stray marks could interfere with accurate reading by the scanning machine.

When "bubbling in" the rater number or topic number, right justify the number. (If the number has fewer digits than the spaces available, leave the blanks on left side.)

HANDLING PROBLEMS

If any of the following problems occurs, do not rate the essay . Give the essay to the coordinator.

- the essay does not have a rater sheet

- the previous rater has not completely marked through the ratings (i.e., you can tell what rating was given by another rater)

- the topic number of the essay does not match the topic number recorded by the first rater

- the essay is written in pencil

If you mix up the rater sheets or have any doubt about whether a particular sheet belongs with an essay, check the social security number on the back of the sheet to see if it matches the social security number on the essay. If the numbers do not match, give the essay and the rater sheet to the coordinator.

Click here to view an image of the
Rater scan sheet with instructions on how to fill it out.

Click here to view an image of a
Rater scan sheet that has been filled out.

Last updated: November 8, 1996