Faculty Publications

Parameter Estimation For Peer Grading Under Incomplete Design

Document Type

Article

Journal/Book/Conference Title

Educational and Psychological Measurement

Volume

48

Issue

1

First Page

69

Last Page

81

Abstract

Methodology developed in the paper supports administration and analysis of peer grading exercises. The technique is especially useful for the incomplete design case, wherein each grader does not grade each paper. The procedure produces simultaneous estimates of (a) bias of each grader, (b) “true” grade for each paper, and (c) relative consistency of each grader. The approach is based on a simple linear statistical model, augmented by a zerosum constraint on grader bias parameters. Ordinary least squares (OLS) estimates are computed and then refined via generalized least squares (GLS) estimation. Case weights for the GLS are based on consistency patterns of the graders, as revealed in the OLS step. The methodology is applied to artificial data to demonstrate its effectiveness. The methodology applies to any situation wherein multiple raters rate multiple ratees on an additive scale. © 1988, Sage Publications. All rights reserved.

Department

Department of Management

Original Publication Date

1-1-1988

DOI of published version

10.1177/001316448804800109

Link to Full Text

Comments?

Share

COinS