TAILIEUCHUNG - Báo cáo khoa học: "INTEGRATING WITH WORD BOUNDARY IDENTIFICATION SENTENCE UNDERSTANDING"

Chinese sentences are written with no special delimiters such as space to indicate word boundaries. Existing Chinese NLP systems therefore employ preprocessors to segment sentences into words. Contrary to the conventional wisdom of separating this issue from the task of sentence understanding, we propose an integrated model that performs word boundary identification in lockstep with sentence understanding. In this approach, there is no distinction between rules for word boundary identification and rules for sentence understanding. These two functions are combined. . | INTEGRATING WORD BOUNDARY IDENTIFICATION WITH SENTENCE UNDERSTANDING Kok Wee Gan Department of Information Systems Computer Science National University of Singapore Kent Ridge Crescent Singapore 0511 Internet gankw@ Abstract Chinese sentences are written with no special delimiters such as space to indicate word boundaries. Existing Chinese NLP systems therefore employ preprocessors to segment sentences into words. Contrary to the conventional wisdom of separating this issue from the task of sentence understanding we propose an integrated model that performs word boundary identification in lockstep with sentence understanding. In this approach there is no distinction between rules for word boundary identification and rules for sentence understanding. These two functions are combined. Word boundary ambiguities are detected especially the fallacious ones when they block the primary task of discovering the inter-relationships among the various constituents of a sentence which essentially is the essence of the understanding process. In this approach statistical information is also incorporated providing the system a quick and fairly reliable starting ground to carry out the primary task of relationship- building. 1 THE PROBLEM Chinese sentences are written with no special delimiters such as space to indicate word boundaries. Existing Chinese NLP systems therefore employ preprocessors to segment sentences into words. Many techniques have been developed for this task from simple pattern matching methods . maximum matching reverse maximum matching Wang et al. 1990 Kang Zheng 1991 to statistical methods . word association relaxation Sproat Shih 1990 Fan Tsai 1988 to rule-based approaches Huang 1989 Yeh Lee 1991 He et al. 1991 . However it is observed that simple pattern matching methods and stochastic methods perform poorly in sentences such as 1 2 and 3 where word boundary ambiguities exist. 1 1 ta benren sheng le She alone give birth to ASP san ge haizi .

• Báo cáo khoa học
• INTEGRATING WITH WORD
• BOUNDARY IDENTIFICATION SENTENCE UNDERSTANDING
• Kok Wee Gan
• báo cáo khoa học
• mô hình ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Integrating Multiple Knowledge Sources
• Disambiguate Word Sense
• An Exemplar Based Approach
• information technology
• SharePoint 2007
• microsoft office
• Creating Forms
• InfoPath
• SharePoint Libraries
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học