TAILIEUCHUNG - Báo cáo khoa học: "Confidence Measure for Word Alignment"

In this paper we present a confidence measure for word alignment based on the posterior probability of alignment links. We introduce sentence alignment confidence measure and alignment link confidence measure. Based on these measures, we improve the alignment quality by selecting high confidence sentence alignments and alignment links from multiple word alignments of the same sentence pair. Additionally, we remove low confidence alignment links from the word alignment of a bilingual training corpus, which increases the alignment F-score, improves Chinese-English and Arabic-English translation quality and significantly reduces the phrase translation table size. . | Confidence Measure for Word Alignment Fei Huang IBM Research Center Yorktown Heights NY 10598 USA huangfe@ Abstract In this paper we present a confidence measure for word alignment based on the posterior probability of alignment links. We introduce sentence alignment confidence measure and alignment link confidence measure. Based on these measures we improve the alignment quality by selecting high confidence sentence alignments and alignment links from multiple word alignments of the same sentence pair. Additionally we remove low confidence alignment links from the word alignment of a bilingual training corpus which increases the alignment F-score improves Chinese-English and Arabic-English translation quality and significantly reduces the phrase translation table size. 1 Introduction Data-driven approaches have been quite active in recent machine translation MT research. Many MT systems such as statistical phrase-based and syntax-based systems learn phrase translation pairs or translation rules from large amount of bilingual data with word alignment. The quality of the parallel data and the word alignment have significant impacts on the learned translation models and ultimately the quality of translation output. Due to the high cost of commissioned translation many parallel sentences are automatically extracted from comparable corpora which inevitably introduce many noises . inaccurate or non-literal translations. Given the huge amount of bilingual training data word alignments are automatically generated using various algorithms Brown et al. 1994 Vogel et al. 1996 Figure 1 An example of inaccurate translation and word alignment. and Ittycheriah and Roukos 2005 which also introduce many word alignment errors. The example in Figure 1 shows the word alignment of the given Chinese and English sentence pair where the English words following each Chinese word is its literal translation. We find untranslated Chinese and English words marked with

• Báo cáo khoa học
• Confidence Measure
• Word Alignment
• Fei Huang
• báo cáo khoa học
• báo cáo ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học