TAILIEUCHUNG - Báo cáo khoa học: "Concept Unification of Terms in Different Languages for IR"

Due to the historical and cultural reasons, English phases, especially the proper nouns and new words, frequently appear in Web pages written primarily in Asian languages such as Chinese and Korean. Although these English terms and their equivalences in the Asian languages refer to the same concept, they are erroneously treated as independent index units in traditional Information Retrieval (IR). This paper describes the degree to which the problem arises in IR and suggests a novel technique to solve it | Concept Unification of Terms in Different Languages for IR Qing Li Sung-Hyon Myaeng Information Communications University Korea liqing myaeng @ Yun Jin Chungnam National University Korea wkim@ Bo-yeong Kang Seoul National University Korea comeng99@ Abstract Due to the historical and cultural reasons English phases especially the proper nouns and new words frequently appear in Web pages written primarily in Asian languages such as Chinese and Korean. Although these English terms and their equivalences in the Asian languages refer to the same concept they are erroneously treated as independent index units in traditional Information Retrieval IR . This paper describes the degree to which the problem arises in IR and suggests a novel technique to solve it. Our method firstly extracts an English phrase from Asian language Web pages and then unifies the extracted phrase and its equivalence s in the language as one index unit. Experimental results show that the high precision of our conceptual unification approach greatly improves the IR performance. 1 Introduction The mixed use of English and local languages presents a classical problem of vocabulary mismatch in monolingual information retrieval MIR . The problem is significant especially in Asian language because words in the local languages are often mixed with English words. Although English terms and their equivalences in a local language refer to the same concept they are erroneously treated as independent index units in traditional MIR. Such separation of semantically identical words in different languages may limit retrieval performance. For instance as shown in Figure 1 there are three kinds of Chinese Web pages containing information related with Viterbi Algorithm ÍỆELMEẾ . The first case contains Viterbi Algorithm but not its Chinese equivalence ÍỆELMEẾ . The second num MTSKt-romtM as the states of. HMM hidden Markov models in which the 1 attic eiVitetbi algorithm is employed for .

• Báo cáo khoa học
• Concept Unification of Terms
• Different Languages for IR
• Qing Li
• Sung Hyon Myaeng
• báo cáo khoa học
• mô hình ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học