TAILIEUCHUNG - Báo cáo khoa học: "Distributional Representations for Handling Sparsity in Supervised Sequence-Labeling"

Supervised sequence-labeling systems in natural language processing often suffer from data sparsity because they use word types as features in their prediction tasks. Consequently, they have difficulty estimating parameters for types which appear in the test set, but seldom (or never) appear in the training set. We demonstrate that distributional representations of word types, trained on unannotated text, can be used to improve performance on rare words. We incorporate aspects of these representations into the feature space of our sequence-labeling systems. . | Distributional Representations for Handling Sparsity in Supervised Sequence-Labeling Fei Huang Temple University 1805 N. Broad St. Wachman Hall 324 tub58431@ Abstract Supervised sequence-labeling systems in natural language processing often suffer from data sparsity because they use word types as features in their prediction tasks. Consequently they have difficulty estimating parameters for types which appear in the test set but seldom or never appear in the training set. We demonstrate that distributional representations of word types trained on unannotated text can be used to improve performance on rare words. We incorporate aspects of these representations into the feature space of our sequence-labeling systems. In an experiment on a standard chunking dataset our best technique improves a chunker from F1 to F1 on chunks beginning with rare words. On the same dataset it improves our part-of-speech tagger from 74 to 80 accuracy on rare words. Furthermore our system improves significantly over a baseline system when applied to text from a different domain and it reduces the sample complexity of sequence labeling. 1 Introduction Data sparsity and high dimensionality are the twin curses of statistical natural language processing NLP . In many traditional supervised NLP systems the feature space includes dimensions for each word type in the data or perhaps even combinations of word types. Since vocabularies can be extremely large this leads to an explosion in the number of parameters. To make matters worse language is Zipf-distributed so that a large fraction of any training data set will be hapax legom-ena very many word types will appear only a few times and many word types will be left out of the training set altogether. As a consequence for Alexander Yates Temple University 1805 N. Broad St. Wachman Hall 324 yates@ many word types supervised NLP systems have very few or even zero labeled examples from which to estimate parameters. The

• Báo cáo khoa học
• Distributional Representations
• Handling Sparsity
• Supervised Sequence Labeling
• báo cáo khoa học
• báo cáo ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học