TAILIEUCHUNG - Báo cáo khoa học: "Part-of-Speech Tagging Considering Surface Form for an Agglutinative Language"

The previous probabilistic part-of-speech tagging models for agglutinative languages have considered only lexical forms of morphemes, not surface forms of words. This causes an inaccurate calculation of the probability. The proposed model is based on the observation that when there exist words (surface forms) that share the same lexical forms, the probabilities to appear are different from each other. Also, it is designed to consider lexical form of word. By experiments, we show that the proposed model outperforms the bigram Hidden Markov model (HMM)-based tagging model. based tagging model. . | Part-of-Speech Tagging Considering Surface Form for an Agglutinative Language Do-Gil Lee and Hae-Chang Rim Dept. of Computer Science Engineering Korea University 1 5-ka Anam-dong Seongbuk-ku Seoul 136-701 Korea dglee rim @ Abstract The previous probabilistic part-of-speech tagging models for agglutinative languages have considered only lexical forms of morphemes not surface forms of words. This causes an inaccurate calculation of the probability. The proposed model is based on the observation that when there exist words surface forms that share the same lexical forms the probabilities to appear are different from each other. Also it is designed to consider lexical form of word. By experiments we show that the proposed model outperforms the bigram Hidden Markov model HMM -based tagging model. 1 Introduction Part-of-speech POS tagging is a job to assign a proper POS tag to each linguistic unit such as word for a given sentence. In English POS tagging word is used as a linguistic unit. However the number of possible words in agglutinative languages such as Korean is almost infinite because words can be freely formed by gluing morphemes together. Therefore morpheme-unit tagging is preferred and more suitable in such languages than word-unit tagging. Figure 1 shows an example of morpheme structure of a sentence where the bold lines indicate the most likely morpheme-POS sequence. A solid line represents a transition between two morphemes across a word boundary and a dotted line represents a transition between two morphemes in a word. The previous probabilistic POS models for agglutinative languages have considered only lexical forms of morphemes not surface forms of words. This causes an inaccurate calculation of the probability. The proposed model is based on the observation that when there exist words surface forms that share the same lexical forms the probabilities to appear are different from each other. Also it is designed to consider lexical form of

• Báo cáo khoa học
• Part of Speech Tagging
• Considering Surface Form
• an Agglutinative Language
• báo cáo khoa học
• mô hình ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học