TAILIEUCHUNG - Báo cáo khoa học: "USING BRACKETED PARSES TO EVALUATE A GRAMMAR CHECKING APPLICATION"

We describe a method for evaluating a grammar checking application with hand-bracketed parses. A randomly-selected set of sentences was submitted to a grammar checker in both bracketed and unbracketed formats. A comparison of the resulting error reports illuminates the relationship between the underlying performance of the parsergrammar system and the error critiques presented to the user. INTRODUCTION The recent development o f broad-coverage natural language processing systems has stimulated work on the evaluation of the syntactic component of such systems, for purposes of basic evaluation and improvement o f system performance. . | USING BRACKETED PARSES TO EVALUATE A GRAMMAR CHECKING APPLICATION Richard H. Wojcik Philip Harrison John Bremer Boeing Computer Services Research and Technology Division . Box 24346 MS 7L-43 Seattle WA 98124-2964 Internet rwojcik@ pharrison@ jbremer@ Abstract We describe a method for evaluating a grammar checking application with hand-bracketed parses. A randomly-selected set of sentences was submitted to a grammar checker in both bracketed and unbracketed formats. A comparison of the resulting error reports illuminates the relationship between the underlying performance of the parsergrammar system and the eưor critiques presented to the user. INTRODUCTION The recent development of broad-coverage natural language processing systems has stimulated work on the evaluation of the syntactic component of such systems for purposes of basic evaluation and improvement of system performance. Methods utilizing hand-bracketed corpora such as the University of Pennsylvania Treebank as a basis for evaluation metrics have been discussed in Black et al. 1991 Harrison et al. 1991 and Black et al. 1992 . Three metrics discussed in those works were the Crossing Parenthesis Score a count of the number of phrases in the machine produced parse which cross with one or more phrases in the hand parse Recall the percentage of phrases in the hand parse that are also in die machine parse and Precision the percentage of phrases in the machine parse that are in the hand parse . We have developed a methodology for using hand-bracketed parses to examine both the internal and external performance of a grammar checker. The internal performance refers to the behavior of the underlying system . the toke-nizer parser lexicon and grammar. The external performance refers to the error critiques generated by the Our evaluation methodology relies on three separate eưor reports generated from a corpus of randomly selected sentences 1 a report based on unbracketed

• Báo cáo khoa học
• USING BRACKETED PARSES TO EVALUATE
• GRAMMAR CHECKING APPLICATION
• Richard H
• Wojcik
• Philip Harrison
• báo cáo khoa học
• mô hình ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học