TAILIEUCHUNG - Báo cáo khoa học: "Combining Trigram and Winnow in Thai OCR Error Correction"

For languages that have no explicit word boundary such as Thai, Chinese and Japanese, correcting words in text is harder than in English because of additional ambiguities in locating error words. The traditional method handles this by hypothesizing that every substrings in the input sentence could be error words and trying to correct all of them. In this paper, we propose the idea of reducing the scope of spelling correction by focusing only on dubious areas in the input sentence. | Combining Trigram and Winnow in Thai OCR Error Correction Surapant Meknavin National Electronics and Computer Technology Center 73 1 Rama VI Road Rajthevi Bangkok Thailand surapan@ Boonserm Kijsirikul Ananlada Chotimongkol and Cholwich Nuttee Department of Computer Engineering Chulalongkorn University Thailand fengbks@chulkn. chula. ac. th Abstract For languages that have no explicit word boundary such as Thai Chinese and Japanese correcting words in text is harder than in English because of additional ambiguities in locating error words. The traditional method handles this by hypothesizing that every substrings in the input sentence could be error words and trying to correct all of them. In this paper we propose the idea of reducing the scope of spelling correction by focusing only on dubious areas in the input sentence. Boundaries of these dubious areas could be obtained approximately by applying word segmentation algorithm and finding word sequences with low probability. To generate the candidate correction words we used a modified edit distance which reflects the characteristic of Thai OCR errors. Finally a part-of-speech trigram model and Winnow algorithm are combined to determine the most probable correction. 1 Introduction Optical character recognition OCR is useful in a wide range of applications such as office automation and information retrieval system. However OCR in Thailand is still not widely used partly because existing Thai OCRs are not quite satisfactory in terms of accuracy. Recently several research projects have focused on spelling correction for many types of errors including those from OCR Kukich 1992 . Nevertheless the strategy is slightly different from language to language since the characteristic of each language is different. Two characteristics of Thai which make the task of error correction different from those of other languages are 1 there is no explicit word boundary and 2 characters are written in three levels . the .

• Báo cáo khoa học
• Combining Trigram
• Winnow in Thai
• OCR Error Correction
• báo cáo khoa học
• mô hình ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Combining Trigram based
• Feature based Methods
• Context Sensitive Spelling Correction
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học