TAILIEUCHUNG - Báo cáo khoa học: "Alignment-Based Discriminative String Similarity"

A character-based measure of similarity is an important component of many natural language processing systems, including approaches to transliteration, coreference, word alignment, spelling correction, and the identification of cognates in related vocabularies. We propose an alignment-based discriminative framework for string similarity. We gather features from substring pairs consistent with a character-based alignment of the two strings. This approach achieves exceptional performance; on nine separate cognate identification experiments using six language pairs, we more than double the precision of traditional orthographic measures like Longest Common Subsequence Ratio and Dice’s Coefficient. . | Alignment-Based Discriminative String Similarity Shane Bergsma and Grzegorz Kondrak Department of Computing Science University of Alberta Edmonton Alberta Canada T6G 2E8 bergsma kondrak @ Abstract A character-based measure of similarity is an important component of many natural language processing systems including approaches to transliteration coreference word alignment spelling correction and the identification of cognates in related vocabularies. We propose an alignment-based discriminative framework for string similarity. We gather features from substring pairs consistent with a character-based alignment of the two strings. This approach achieves exceptional performance on nine separate cognate identification experiments using six language pairs we more than double the precision of traditional orthographic measures like Longest Common Subsequence Ratio and Dice s Coefficient. We also show strong improvements over other recent discriminative and heuristic similarity functions. 1 Introduction String similarity is often used as a means of quantifying the likelihood that two pairs of strings have the same underlying meaning based purely on the character composition of the two words. Strube et al. 2002 use Edit Distance as a feature for determining if two words are coreferent. Taskar et al. 2005 use French-English common letter sequences as a feature for discriminative word alignment in bilingual texts. Brill and Moore 2000 learn misspelled-word to correctly-spelled-word similarities for spelling correction. In each of these examples a similarity measure can make use of the recurrent substring pairings that reliably occur between 656 words having the same meaning. Across natural languages these recurrent substring correspondences are found in word pairs known as cognates words with a common form and meaning across languages. Cognates arise either from words in a common ancestor language . light Licht night Nacht in English German or from foreign .

• Báo cáo khoa học
• Alignment Based Discriminative String Similarity
• Shane Bergsma and Grzegorz Kondrak
• Long Papers
• báo cáo khoa học
• mô hình ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học