TAILIEUCHUNG - Báo cáo khoa học: "Improved Smoothing for N-gram Language Models Based on Ordinary Counts"

Kneser-Ney (1995) smoothing and its variants are generally recognized as having the best perplexity of any known method for estimating N-gram language models. Kneser-Ney smoothing, however, requires nonstandard N-gram counts for the lowerorder models used to smooth the highestorder model. For some applications, this makes Kneser-Ney smoothing inappropriate or inconvenient. In this paper, we introduce a new smoothing method based on ordinary counts that outperforms all of the previous ordinary-count methods we have tested, with the new method eliminating most of the gap between Kneser-Ney and those methods. . | Improved Smoothing for N-gram Language Models Based on Ordinary Counts Robert C. Moore Chris Quirk Microsoft Research Redmond WA 98052 USA bobmoore chrisq @ Abstract Kneser-Ney 1995 smoothing and its variants are generally recognized as having the best perplexity of any known method for estimating N-gram language models. Kneser-Ney smoothing however requires nonstandard N-gram counts for the lower-order models used to smooth the highest-order model. For some applications this makes Kneser-Ney smoothing inappropriate or inconvenient. In this paper we introduce a new smoothing method based on ordinary counts that outperforms all of the previous ordinary-count methods we have tested with the new method eliminating most of the gap between Kneser-Ney and those methods. 1 Introduction Statistical language models are potentially useful for any language technology task that produces natural-language text as a final or intermediate output. In particular they are extensively used in speech recognition and machine translation. Despite the criticism that they ignore the structure of natural language simple N-gram models which estimate the probability of each word in a text string based on the N 1 preceding words remain the most widely used type of model. The simplest possible N-gram model is the maximum likelihood estimate MLE which takes the probability of a word wn given the preceding context W1. wn-1 to be the ratio of the number of occurrences in a training corpus of the Ngram W1 . .wn to the total number of occurrences of any word in the same context C w1 . . . wn p w w1---w -1 . C wi w -iw One obvious problem with this method is that it assigns a probability of zero to any N-gram that is not observed in the training corpus hence numerous smoothing methods have been invented that reduce the probabilities assigned to some or all observed N-grams to provide a non-zero probability for N-grams not observed in the training corpus. The best methods for smoothing .

• Báo cáo khoa học
• Improved Smoothing
• N gram Language Models Based
• Ordinary Counts
• báo cáo khoa học
• báo cáo ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học