TAILIEUCHUNG - Báo cáo khoa học: "DECISION LISTS FOR LEXICAL AMBIGUITY RESOLUTION"

The particular domain chosen here as a case study is the problem of restoring missing accents 1 to Spanish and French text. Because it requires the resolution of both semantic and syntactic ambiguity, and offers an objective ground truth for automatic evaluation, it is particularly well suited for demonstrating and testing the capabilities of the given algorithm. It is also a practical problem with immediate application. PROBLEM DESCRIPTION The general problem considered here is the resolution of lexical ambiguity, both syntactic and semantic, based on properties of the surrounding context. . | DECISION LISTS FOR LEXICAL AMBIGUITY RESOLUTION Application to Accent Restoration in Spanish and French David Yarowsky Department of Computer and Information Science University of Pennsylvania Philadelphia PA 19104 Abstract This paper presents a statistical decision procedure for lexical ambiguity resolution. The algorithm exploits both local syntactic patterns and more distant collocational evidence generating an efficient effective and highly perspicuous recipe for resolving a given ambiguity. By identifying and utilizing only the single best disambiguating evidence in a target context the algorithm avoids the problematic complex modeling of statistical dependencies. Although directly applicable to a wide class of ambiguities the algorithm is described and evaluated in a realistic case study the problem of restoring missing accents in Spanish and French text. Current accuracy exceeds 99 on the full task and typically is over 90 for even the most difficult ambiguities. INTRODUCTION This paper presents a general-purpose statistical decision procedure for lexical ambiguity resolution based on decision lists Rivest 1987 . The algorithm considers multiple types of evidence in the context of an ambiguous word exploiting differences in collocational distribution as measured by log-likelihoods. Unlike standard Bayesian approaches however it does not combine the log-likelihoods of all available pieces of contextual evidence but bases its classifications solely on the single most reliable piece of evidence identified in the target context. Perhaps surprisingly this strategy appears to yield the same or even slightly better precision than the combination of evidence approach when trained on the same features. It also brings with it several additional advantages the greatest of which is the ability to include multiple highly non-independent sources of evidence without complex modeling of dependencies. Some other advantages are significant .

• Báo cáo khoa học
• DECISION LISTS
• LEXICAL AMBIGUITY RESOLUTION
• David Yarowskybáo cáo khoa học
• mô hình ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Brazilian Agribusiness
• Forestry and Life
• Cycle Assessment
• Wildlife Forestry
• Fire Ignition
• Trends in Durango
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học