TAILIEUCHUNG - Báo cáo khoa học: "You Can’t Beat Frequency (Unless You Use Linguistic Knowledge) – A Qualitative Evaluation of Association Measures for Collocation and Term Extraction"

In the past years, a number of lexical association measures have been studied to help extract new scientific terminology or general-language collocations. The implicit assumption of this research was that newly designed term measures involving more sophisticated statistical criteria would outperform simple counts of cooccurrence frequencies. We here explicitly test this assumption. By way of four qualitative criteria, we show that purely statistics-based measures reveal virtually no difference compared with frequency of occurrence counts, while linguistically more informed metrics do reveal such a marked difference. . | You Can t Beat Frequency Unless You Use Linguistic Knowledge -A Qualitative Evaluation of Association Measures for Collocation and Term Extraction Joachim Wermter Udo Hahn Jena University Language Information Engineering JULIE Lab D-07743 Jena Germany wermter hahn @ Abstract In the past years a number of lexical association measures have been studied to help extract new scientific terminology or general-language collocations. The implicit assumption of this research was that newly designed term measures involving more sophisticated statistical criteria would outperform simple counts of cooccurrence frequencies. We here explicitly test this assumption. By way of four qualitative criteria we show that purely statistics-based measures reveal virtually no difference compared with frequency of occurrence counts while linguistically more informed metrics do reveal such a marked difference. 1 Introduction Research on domain-specific automatic term recognition ATR and on general-language collocation extraction CE has gone mostly separate ways in the last decade although their underlying procedures and goals turn out to be rather similar. In both cases linguistic filters POS taggers phrase chunkers shallow parsers initially collect candidates from large text corpora and then frequency- or statistics-based evidence or association measures yield scores indicating to what degree a candidate qualifies as a term or a collocation. While term mining and collocation mining as a whole involve almost the same analytical processing steps such as orthographic and morphological normalization normalization of term or collocation variation etc. it is exactly the measure which grades termhood or collocativity of a candidate on which alternative approaches diverge. Still the output of such mining algorithms look similar. It is typically constituted by a ranked list on which ideally the true terms or collocations are placed in the top portion of the list while the non-terms

• Báo cáo khoa học
• You Can’t Beat Frequency
• A Qualitative Evaluation of Association Measures for Collocation
• Term Extraction
• báo cáo khoa học
• mô hình ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học