TAILIEUCHUNG - Báo cáo khoa học: "Combining Trigram-based and Feature-based Methods for Context-Sensitive Spelling Correction"

This paper addresses the problem of correcting spelling errors that result in valid, though unintended words (such as peace and piece, or quiet and quite) and also the problem of correcting particular word usage errors (such as amount and number, or among and between). Such corrections require contextual information and are not handled by conventional spelling programs such as Unix spell. First, we introduce a method called Trigrams that uses part-of-speech trigrams to encode the context. This method uses a small number of parameters compared to previous methods based on word trigrams. However, it is effectively unable to distinguish. | Combining Trigram-based and Feature-based Methods for Context-Sensitive Spelling Correction Andrew R. Golding and Yves Schabes Mitsubishi Electric Research Laboratories 201 Broadway Cambridge MA 02139 golding schabes @ Abstract This paper addresses the problem of correcting spelling errors that result in valid though unintended words such as peace and piece or quiet and quite and also the problem of correcting particular word usage errors such as amount and number or among and between . Such corrections require contextual information and are not handled by conventional spelling programs such as Unix spell. First we introduce a method called Trigrams that uses part-of-speech trigrams to encode the context. This method uses a small number of parameters compared to previous methods based on word trigrams. However it is effectively unable to distinguish among words that have the same part of speech. For this case an alternative feature-based method called Bayes performs better but Bayes is less effective than Trigrams when the distinction among words depends on syntactic constraints. A hybrid method called Tribayes is then introduced that combines the best of the previous two methods. The improvement in performance of Tribayes over its components is verified experimentally. Tribayes is also compared with the grammar checker in Microsoft Word and is found to have substantially higher performance. 1 Introduction Spelling correction has become a very common technology and is often not perceived as a problem where progress can be made. However conventional spelling checkers such as Unix spell are concerned only with spelling errors that result in words that cannot be found in a word list of a given language. One analysis has shown that up to 15 of spelling errors that result from elementary typographical errors character insertion deletion or transposition yield another valid word in the language Peterson 1986 . These errors remain undetected by traditional .

• Báo cáo khoa học
• Combining Trigram based
• Feature based Methods
• Context Sensitive Spelling Correction
• báo cáo khoa học
• mô hình ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học