TAILIEUCHUNG - Báo cáo khoa học: "Exploring Distributional Similarity Based Models for Query Spelling Correction"

A query speller is crucial to search engine in improving web search relevance. This paper describes novel methods for use of distributional similarity estimated from query logs in learning improved query spelling correction models. The key to our methods is the property of distributional similarity between two terms: it is high between a frequently occurring misspelling and its correction, and low between two irrelevant terms only with similar spellings. We present two models that are able to take advantage of this property. . | Exploring Distributional Similarity Based Models for Query Spelling Correction Mu Li Microsoft Research Asia 5F Sigma Center Zhichun Road Haidian District Beijing China 100080 muli@ Yang Zhang School of Computer Science and Technology Tianjin University Tianjin China 300072 yangzhang@ Muhua Zhu School of Information Science and Engineering Northeastern University Shenyang Liaoning China 110004 zhumh@ Ming Zhou Microsoft Research Asia 5F Sigma Center Zhichun Road Haidian District Beijing China 100080 mingzhou@ Abstract A query speller is crucial to search engine in improving web search relevance. This paper describes novel methods for use of distributional similarity estimated from query logs in learning improved query spelling correction models. The key to our methods is the property of distributional similarity between two terms it is high between a frequently occurring misspelling and its correction and low between two irrelevant terms only with similar spellings. We present two models that are able to take advantage of this property. Experimental results demonstrate that the distributional similarity based models can significantly outperform their baseline systems in the web query spelling correction task. 1 Introduction Investigations into query log data reveal that more than 10 of queries sent to search engines contain misspelled terms Cucerzan and Brill 2004 . Such statistics indicate that a good query speller is crucial to search engine in improving web search relevance because there is little opportunity that a search engine can retrieve many relevant contents with misspelled terms. The problem of designing a spelling correction program for web search queries however poses special technical challenges and cannot be well solved by general purpose spelling correction methods. Cucerzan and Brill 2004 discussed in detail specialties and difficulties of a query spell checker and illustrated why the existing .

• Báo cáo khoa học
• Exploring Distributional
• Similarity Based Models
• Query Spelling Correction
• báo cáo khoa học
• mô hình ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học