TAILIEUCHUNG - Báo cáo khoa học: "An Unsupervised Morpheme-Based HMM for Hebrew Morphological Disambiguation"

Morphological disambiguation is the process of assigning one set of morphological features to each individual word in a text. When the word is ambiguous (there are several possible analyses for the word), a disambiguation procedure based on the word context must be applied. This paper deals with morphological disambiguation of the Hebrew language, which combines morphemes into a word in both agglutinative and fusional ways. | An Unsupervised Morpheme-Based HMM for Hebrew Morphological Disambiguation Meni Adler Department of Computer Science Ben Gurion University of the Negev 84105 Beer Sheva Israel adlerm@ Abstract Morphological disambiguation is the process of assigning one set of morphological features to each individual word in a text. When the word is ambiguous there are several possible analyses for the word a disambiguation procedure based on the word context must be applied. This paper deals with morphological disambiguation of the Hebrew language which combines morphemes into a word in both agglutinative and fusional ways. We present an unsupervised stochastic model - the only resource we use is a morphological analyzer -which deals with the data sparseness problem caused by the affixational morphology of the Hebrew language. We present a text encoding method for languages with affixational morphology in which the knowledge of word formation rules which are quite restricted in Hebrew helps in the disambiguation. We adapt HMM algorithms for learning and searching this text representation in such a way that segmentation and tagging can be learned in parallel in one step. Results on a large scale evaluation indicate that this learning improves disambiguation for complex tag sets. Our method is applicable to other languages with affix morphology. 1 Introduction Morphological disambiguation is the process of assigning one set of morphological features to each individual word in a text according to the word context. In this work we investigate morphological disambiguation in Modern Hebrew. We explore unsupervised learning method which is more challenging than the supervised case. The main motivation for this approach is that despite the development This work is supported by the Lynn and William Frankel Center for Computer Sciences and by the Knowledge Center for Hebrew Processing Israel Science Ministry. Michael Elhadad Department of Computer Science Ben Gurion University

• Báo cáo khoa học
• An Unsupervised Morpheme Based HMM
• Hebrew Morphological Disambiguation
• Meni Adler
• báo cáo khoa học
• mô hình ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học