Đang chuẩn bị nút TẢI XUỐNG, xin hãy chờ
Tải xuống
In particular, we examine whether the discourse coherence found in an essay, as de ned by a measure of relative proportion of Rough-Shift transitions, might be a signi cant contributor to the accuracy of computer-generated essay scores. Our positive nding validates the role of the Rough-Shift transition and suggests a route for exploring Centering Theory's practical applicability to writing evaluation and instruction. | The Role of Centering Theory s Rough-Shift in the Teaching and Evaluation Eleni Miltsakaki University of Pennsylvania Philadelphia PA 19104 USA elenimi@unagi. cis .upenn.edu of Writing Skills Karen Kukich Educatinal Testing Service Princeton NJ 08541 USA kkukich@ets.org Abstract Existing software systems for automated essay scoring can provide NLP researchers with opportunities to test certain theoretical hypotheses including some derived from Centering Theory. In this study we employ ETS s e-rater essay scoring system to examine whether local discourse coherence as defined by a measure of Rough-Shift transitions might be a significant contributor to the evaluation of essays. Our positive results indicate that Rough-Shifts do indeed capture a source of incoherence one that has not been closely examined in the Centering literature. These results not only justify Rough-Shifts as a valid transition type but they also support the original formulation of Centering as a measure of discourse continuity even in pronominal-free text. 1 Introduction The task of evaluating student s writing ability has traditionally been a laborintensive human endeavor. However several different software systems e.g. PEG Page and Peterson 1995 Intelligent Essay Assessor 1 and e-rater 2 are now being used to perform this task fully automatically. Furthermore by at least one measure these software systems evaluate student essays with the same degree of accuracy as human experts. That is computer-generated scores tend to match human expert scores as frequently as two human scores match each other Burstein et al. 1998 . Essay scoring systems such as these can provide NLP researchers with opportunities to test certain theoretical hypotheses and to explore a variety of practical issues in computational linguistics. In this study we employ the e-rater essay scoring system to test a hy Tttp Isa.colorado.edu. 2http www.ets.org research erater.html pothesis related to Centering Theory Joshi and .
