TAILIEUCHUNG - Báo cáo khoa học: "A Comparison and Semi-Quantitative Analysis of Words and Character-Bigrams as Features in Chinese Text Categorization"

Words and character-bigrams are both used as features in Chinese text processing tasks, but no systematic comparison or analysis of their values as features for Chinese text categorization has been reported heretofore. We carry out here a full performance comparison between them by experiments on various document collections (including a manually word-segmented corpus as a golden standard), and a semi-quantitative analysis to elucidate the characteristics of their behavior; and try to provide some preliminary clue for feature term choice (in most cases, character-bigrams are better than words) and dimensionality setting in text categorization systems. . | A Comparison and Semi-Quantitative Analysis of Words and Character-Bigrams as Features in Chinese Text Categorization Jingyang Li Maosong Sun Xian Zhang National Lab. of Intelligent Technology Systems Department of Computer Sci. Tech. Tsinghua University Beijing 100084 China lijingyang@ sms@ kevinn9@ Abstract Words and character-bigrams are both used as features in Chinese text processing tasks but no systematic comparison or analysis of their values as features for Chinese text categorization has been reported heretofore. We carry out here a full performance comparison between them by experiments on various document collections including a manually word-segmented corpus as a golden standard and a semi-quantitative analysis to elucidate the characteristics of their behavior and try to provide some preliminary clue for feature term choice in most cases character-bigrams are better than words and dimensionality setting in text categorization systems. 1 Introduction1 Because of the popularity of the Vector Space Model VSM in text information processing document indexing term extraction acts as a pre-requisite step in most text information processing tasks such as Information Retrieval Baeza-Yates and Ribeiro-Neto 1999 and Text Categorization Sebastiani 2002 . It is empirically known that the indexing scheme is a nontrivial complication to system performance especially for some Asian languages in which there are no explicit word margins and even no natural semantic unit. Concretely in Chinese Text Categorization tasks the two most important index ing units feature terms are word and characterbigram so the problem is which kind of terms2 should be chosen as the feature terms words or character-bigrams To obtain an all-sided idea about feature choice beforehand we review here the possible feature variants or options . First at the word level we can do stemming do stop-word pruning include POS Part of Speech information etc. Second term .

TAILIEUCHUNG - Chia sẻ tài liệu không giới hạn
Địa chỉ : 444 Hoang Hoa Tham, Hanoi, Viet Nam
Website : tailieuchung.com
Email : tailieuchung20@gmail.com
Tailieuchung.com là thư viện tài liệu trực tuyến, nơi chia sẽ trao đổi hàng triệu tài liệu như luận văn đồ án, sách, giáo trình, đề thi.
Chúng tôi không chịu trách nhiệm liên quan đến các vấn đề bản quyền nội dung tài liệu được thành viên tự nguyện đăng tải lên, nếu phát hiện thấy tài liệu xấu hoặc tài liệu có bản quyền xin hãy email cho chúng tôi.
Đã phát hiện trình chặn quảng cáo AdBlock
Trang web này phụ thuộc vào doanh thu từ số lần hiển thị quảng cáo để tồn tại. Vui lòng tắt trình chặn quảng cáo của bạn hoặc tạm dừng tính năng chặn quảng cáo cho trang web này.