TAILIEUCHUNG - Báo cáo khoa học: "Decompounding query keywords from compounding languages"

Splitting compound words has proved to be useful in areas such as Machine Translation, Speech Recognition or Information Retrieval (IR). Furthermore, real-time IR systems (such as search engines) need to cope with noisy data, as user queries are sometimes written quickly and submitted without review. In this paper we apply a state-of-the-art procedure for German decompounding to other compounding languages, and we show that it is possible to have a single decompounding model that is applicable across languages. . | Decompounding query keywords from compounding languages Enrique Alfonseca Google Inc. ealfonseca@ Slaven Bilac Google Inc. slaven@ Stefan Pharies Google Inc. stefanp@ Abstract Splitting compound words has proved to be useful in areas such as Machine Translation Speech Recognition or Information Retrieval IR . Furthermore real-time IR systems such as search engines need to cope with noisy data as user queries are sometimes written quickly and submitted without review. In this paper we apply a state-of-the-art procedure for German decompounding to other compounding languages and we show that it is possible to have a single decompounding model that is applicable across languages. 1 Introduction Compounding languages Krott 1999 such as German Dutch Danish Norwegian Swedish Greek or Finnish allow the generation of complex words by merging together simpler ones. So for instance the flower bouquet can be expressed in German as Blumenstraufie made up of Blumen flower and straufie bouquet and in Finnish as kukkakimppu from kukka flower and kimppu bunch collection . For many language processing tools that rely on lexicons or language models it is very useful to be able to decompose compounds to increase their coverage and reduce out-of-vocabulary terms. Decompounders have been used successfully in Information Retrieval Braschler and Ripplinger 2004 Machine Translation Brown 2002 Koehn and Knight 2003 and Speech Recognition Adda-Decker et al. 2000 . The Cross Language Evaluation Forum CLEF competitions have shown that very simple approaches can produce big gains in Cross Language Information Retrieval CLIR for German and Dutch Monz and de Rijke 2001 and for Finnish Adafre et al. 2004 . When working with web data which has not necessarily been reviewed for correctness many of the words are more difficult to analyze than when working with standard texts. There are more words with spelling mistakes and many texts mix words from different languages.

• Báo cáo khoa học
• Decompounding query keywords
• compounding languages
• Enrique Alfonseca
• báo cáo khoa học
• báo cáo ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học