TAILIEUCHUNG - Báo cáo khoa học: "M AX S IM: A Maximum Similarity Metric for Machine Translation Evaluation"

We propose an automatic machine translation (MT) evaluation metric that calculates a similarity score (based on precision and recall) of a pair of sentences. Unlike most metrics, we compute a similarity score between items across the two sentences. We then find a maximum weight matching between the items such that each item in one sentence is mapped to at most one item in the other sentence. | Max Sim a Maximum Similarity Metric for Machine Translation Evaluation Yee Seng Chan and Hwee Tou Ng Department of Computer Science National University of Singapore Law Link Singapore 117590 chanys nght @ Abstract We propose an automatic machine translation MT evaluation metric that calculates a similarity score based on precision and recall of a pair of sentences. Unlike most metrics we compute a similarity score between items across the two sentences. We then find a maximum weight matching between the items such that each item in one sentence is mapped to at most one item in the other sentence. This general framework allows us to use arbitrary similarity functions between items and to incorporate different information in our comparison such as n-grams dependency relations etc. When evaluated on data from the ACL-07 MT workshop our proposed metric achieves higher correlation with human judgements than all 11 automatic MT evaluation metrics that were evaluated during the workshop. 1 Introduction In recent years machine translation MT research has made much progress which includes the introduction of automatic metrics for MT evaluation. Since human evaluation of MT output is time consuming and expensive having a robust and accurate automatic MT evaluation metric that correlates well with human judgement is invaluable. Among all the automatic MT evaluation metrics BLEU Papineni et al. 2002 is the most widely used. Although BLEU has played a crucial role in the progress of MT research it is becoming evident that BLEU does not correlate with human judgement well enough and suffers from several other deficiencies such as the lack of an intuitive interpretation of its scores. During the recent ACL-07 workshop on statistical MT Callison-Burch et al. 2007 a total of 11 automatic MT evaluation metrics were evaluated for correlation with human judgement. The results show that as compared to BLEU several recently proposed metrics such as Semantic-role overlap .

• Báo cáo khoa học
• A Maximum Similarity Metric
• Machine Translation Evaluation
• Yee Seng Chan and Hwee Tou Ng
• báo cáo khoa học
• báo cáo ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học