TAILIEUCHUNG - Báo cáo khoa học: "Corpus representativeness for syntactic information acquisition"

The question we have addressed here is to define the size and composition of the corpus we would need in order to get necessary and sufficient information for Machine Learning techniques to induce that type of information. Representativeness of a corpus is a topic largely dealt with, especially in corpus linguistics. One of the standard references is Biber (1993) where the author offers guidelines for corpus design to characterize a language. | Corpus representativeness for syntactic information acquisition Núria BEL IULA Universitat Pompeu Fabra La Rambla 30-32 08002 Barcelona Spain Abstract This paper refers to part of our research in the area of automatic acquisition of computational lexicon information from corpus. The present paper reports the ongoing research on corpus representativeness. For the task of inducing information out of text we wanted to fix a certain degree of confidence on the size and composition of the collection of documents to be observed. The results show that it is possible to work with a relatively small corpus of texts if it is tuned to a particular domain. Even more it seems that a small tuned corpus will be more informative for real parsing than a general corpus. 1 Introduction The coverage of the computational lexicon used in deep Natural Language Processing NLP is crucial for parsing success. But rather frequently the absence of particular entries or the fact that the information encoded for these does not cover very specific syntactic contexts --as those found in technical texts make high informative grammars not suitable for real applications. Moreover this poses a real problem when porting a particular application from domain to domain as the lexicon has to be re-encoded in the light of the new domain. In fact in order to minimize ambiguities and possible over-generation application based lexicons tend to be tuned for every specific domain addressed by a particular application. Tuning of lexicons to different domains is really a delaying factor in the deployment of NLP applications as it raises its costs not only in terms of money but also and crucially in terms of time. A desirable solution would be a plug and play system that given a collection of documents supplied by the customer could induce a tuned lexicon. By tuned we mean full coverage both in terms of 1 entries detecting new items and assigning them a syntactic behavior pattern and 2 syntactic

• Báo cáo khoa học
• Corpus representativeness
• syntactic information acquisition
• Long Papers
• báo cáo khoa học
• mô hình ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học