TAILIEUCHUNG - Báo cáo khoa học: "A Phonetic-Based Approach to Chinese Chat Text Normalization"

Chatting is a popular communication media on the Internet via ICQ, chat rooms, etc. Chat language is different from natural language due to its anomalous and dynamic natures, which renders conventional NLP tools inapplicable. The dynamic problem is enormously troublesome because it makes static chat language corpus outdated quickly in representing contemporary chat language. To address the dynamic problem, we propose the phonetic mapping models to present mappings between chat terms and standard words via phonetic transcription, . Chinese Pinyin in our case. . | A Phonetic-Based Approach to Chinese Chat Text Normalization Yunqing Xia Kam-Fai Wong Department of . The Chinese University of Hong Kong Shatin Hong Kong yqxia kfwong @ Wenjie Li Department of Computing The Hong Kong Polytechnic University Kowloon Hong Kong cswjli@ Abstract Chatting is a popular communication media on the Internet via ICQ chat rooms etc. Chat language is different from natural language due to its anomalous and dynamic natures which renders conventional NLP tools inapplicable. The dynamic problem is enormously troublesome because it makes static chat language corpus outdated quickly in representing contemporary chat language. To address the dynamic problem we propose the phonetic mapping models to present mappings between chat terms and standard words via phonetic transcription . Chinese Pinyin in our case. Different from character mappings the phonetic mappings can be constructed from available standard Chinese corpus. To perform the task of dynamic chat language term normalization we extend the source channel model by incorporating the phonetic mapping models. Experimental results show that this method is effective and stable in normalizing dynamic chat language terms. 1 Introduction Internet facilitates online chatting by providing ICQ chat rooms BBS email blogs etc. Chat language becomes ubiquitous due to the rapid proliferation of Internet applications. Chat language text appears frequently in chat logs of online education Heard-White 2004 customer relationship management Gianforte 2003 etc. On the other hand wed-based chat rooms and BBS systems are often abused by solicitors of terrorism pornography and crime McCullagh 2004 . Thus there is a social urgency to understand online chat language text. Chat language is anomalous and dynamic. Many words in chat text are anomalous to natural language. Chat text comprises of ill-edited terms and anomalous writing styles. We refer chat terms to the anomalous .

• Báo cáo khoa học
• A Phonetic Based Approach to Chinese
• Chat Text Normalization
• Yunqing Xia
• báo cáo khoa học
• mô hình ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học