TAILIEUCHUNG - Báo cáo khoa học: "Serial Combination of Rules and Statistics: A Case Study in Czech Tagging"

A hybrid system is described which combines the strength of manual rulewriting and statistical learning, obtaining results superior to both methods if applied separately. The combination of a rule-based system and a statistical one is not parallel but serial: the rule-based system performing partial disambiguation with recall close to 100% is applied first, and a trigram HMM tagger runs on its results. An experiment in Czech tagging has been performed with encouraging results. | Serial Combination of Rules and Statistics A Case Study in Czech Tagging Jan Hajic Pavel Krbec Pavel Kveton Karel Oliva Vladimir Petkevic IFAL ICNC Computational ITCL MFFUK FF UK Linguistics FF UK Prague Prague Univ. of Saarland Prague Czechia Czechia Germany Czechia hajic krbec @ oliva@ Abstract A hybrid system is described which combines the strength of manual rulewriting and statistical learning obtaining results superior to both methods if applied separately. The combination of a rule-based system and a statistical one is not parallel but serial the rule-based system performing partial disambiguation with recall close to 100 is applied first and a trigram HMM tagger runs on its results. An experiment in Czech tagging has been performed with encouraging results. 1 Tagging of Inflective Languages Inflective languages pose a specific problem in tagging due to two phenomena highly inflective nature causing sparse data problem in any statistically-based system and free word order causing fixed-context systems such as n-gram Hidden Markov Models HMMs to be even less adequate than for English . The average tagset contains about 1 000 - 2 000 distinct tags the size of the set of possible and plausible tags can reach several thousands. Apart from agglutinative languages such as Turkish Finnish and Hungarian see . Hakkani-Tur et al. 2000 and Basque Ezeiza et al. 1998 which pose quite different and in the end less severe problems there have been attempts at solving this problem for some of the highly inflectional European languages such as Daelemans et al. 1996 Erjavec et al. 1999 Slovenian Hajic and Hladka 1997 Hajic and Hladka 1998 Czech and Hajic 2000 five Central and Eastern European languages but so far no system has reached - in the absolute terms - a performance comparable to English tagging such as Ratnaparkhi 1996 which stands around or above 97 . For example Hajic and .

• Báo cáo khoa học
• Serial Combination
• Rules and Statistics
• A Case Study in Czech Tagging
• báo cáo khoa học
• mô hình ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• BMC Cancer
• Dendritic cells
• CD40 signalling
• Preclinical cancer models
• Therapeutic combination
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học