TAILIEUCHUNG - Báo cáo khoa học: "Sub-sentential Alignment Using Substring Co-Occurrence Counts"

In this paper, we will present an efficient method to compute the co-occurrence counts of any pair of substring in a parallel corpus, and an algorithm that make use of these counts to create subsentential alignments on such a corpus. This algorithm has the advantage of being as general as possible regarding the segmentation of text. | Sub-sentential Alignment Using Substring Co-Occurrence Counts Fabien Cromieres GETA-CLIPS-IMAG BP53 38041 Grenoble Cedex 9 France Abstract In this paper we will present an efficient method to compute the co-occurrence counts of any pair of substring in a parallel corpus and an algorithm that make use of these counts to create sub-sentential alignments on such a corpus. This algorithm has the advantage of being as general as possible regarding the segmentation of text. 1 Introduction An interesting and important problem in the Statistical Machine Translation SMT domain is the creation of sub-sentential alignment in a parallel corpus a bilingual corpus already aligned at the sentence level . These alignments can later be used to for example train SMT systems or extract bilingual lexicons. Many algorithms have already been proposed for sub-sentential alignment. Some of them focus on word-to-word alignment Brown 97 or Melamed 97 . Others allow the generation of phrase-level alignments such as Och et al. 1999 Marcu and Wong 2002 or Zhang Vogel Waibel 2003 . However with the exception of Marcu and Wong these phrase-level alignment algorithms still place their analyses at the word level whether by first creating a word-to-word alignment or by computing correlation coefficients between pairs of individual words. This is in our opinion a limitation of these algorithms mainly because it makes them rely heavily on our capacity to segment a sentence in words. And defining what a word is is not as easy as it might seem. In peculiar in many Asians writings systems Japanese Chinese or Thai for example there is not a special symbol to delimit words such as the blank in most non Asian writing systems . Current systems usually work around this problem by using a segmentation tool to pre-process the data. There are however two major disadvantages - These tools usually need a lot of linguistic knowledge such as lexical dictionaries and hand-crafted .

• Báo cáo khoa học
• Sub sentential Alignment
• Substring Co Occurrence Counts
• Fabien Cromieres
• báo cáo khoa học
• mô hình ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học