TAILIEUCHUNG - Báo cáo khoa học: "The Back-translation Score: Automatic MT Evaluation at the Sentence Level without Reference Translations"

Automatic tools for machine translation (MT) evaluation such as BLEU are well established, but have the drawbacks that they do not perform well at the sentence level and that they presuppose manually translated reference texts. Assuming that the MT system to be evaluated can deal with both directions of a language pair, in this research we suggest to conduct automatic MT evaluation by determining the orthographic similarity between a back-translation and the original source text. This way we eliminate the need for human translated reference texts. By correlating BLEU and back-translation scores with human judgments, it could be shown. | The Back-translation Score Automatic MT Evaluation at the Sentence Level without Reference Translations Reinhard Rapp Universitat Rovira i Virgili Avinguda Catalunya 35 43002 Tarragona Spain Abstract Automatic tools for machine translation MT evaluation such as BLEU are well established but have the drawbacks that they do not perform well at the sentence level and that they presuppose manually translated reference texts. Assuming that the MT system to be evaluated can deal with both directions of a language pair in this research we suggest to conduct automatic MT evaluation by determining the orthographic similarity between a back-translation and the original source text. This way we eliminate the need for human translated reference texts. By correlating BLEU and back-translation scores with human judgments it could be shown that the back-translation score gives an improved performance at the sentence level. 1 Introduction The manual evaluation of the results of machine translation systems requires considerable time and effort. For this reason fast and inexpensive automatic methods were developed. They are based on the comparison of a machine translation with a reference translation produced by humans. The comparison is done by determining the number of matching word sequences between both translations. It could be shown that such methods of which BLEU Papineni et al. 2002 is the most common can deliver evaluation results that show a high agreement with human judgments Papineni et al. 2002 Coughlin 2003 Koehn Monz 2006 . Disadvantages of BLEU and related methods are that a human reference translation is required and that the results are reliable only at corpus level . when computed over many sentence pairs see . Callison-Burch et al. 2006 . However at the sentence level due to data sparseness the results tend to be unsatisfactory Agarwal Lavie 2008 Callison-Burch et al. 2008 . Pap-ineni et al. 2002 describe this as follows BLEU s .

• Báo cáo khoa học
• The Back translation Score
• Automatic MT Evaluation at the Sentence Level
• Reference Translations
• báo cáo khoa học
• báo cáo ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học