TAILIEUCHUNG - Báo cáo khoa học: "Collecting a Why-question corpus for development and evaluation of an automatic QA-system"

Question answering research has only recently started to spread from short factoid questions to more complex ones. One significant challenge is the evaluation: manual evaluation is a difficult, time-consuming process and not applicable within efficient development of systems. Automatic evaluation requires a corpus of questions and answers, a definition of what is a correct answer, and a way to compare the correct answers to automatic answers produced by a system. | Collecting a Why-question corpus for development and evaluation of an automatic QA-system Joanna Mrozinski Edward Whittaker Sadaoki Furui Department of Computer Science Tokyo Institute of Technology 2-12-1-W8-77 Ookayama Meguro-ku Tokyo 152-8552 Japan mrozinsk edw furui @ Abstract Question answering research has only recently started to spread from short factoid questions to more complex ones. One significant challenge is the evaluation manual evaluation is a difficult time-consuming process and not applicable within efficient development of systems. Automatic evaluation requires a corpus of questions and answers a definition of what is a correct answer and a way to compare the correct answers to automatic answers produced by a system. For this purpose we present a Wikipedia-based corpus of Why-questions and corresponding answers and articles. The corpus was built by a novel method paid participants were contacted through a Web-interface a procedure which allowed dynamic fast and inexpensive development of data collection methods. Each question in the corpus has several corresponding partly overlapping answers which is an asset when estimating the correctness of answers. In addition the corpus contains information related to the corpus collection process. We believe this additional information can be used to post-process the data and to develop an automatic approval system for further data collection projects conducted in a similar manner. 1 Introduction Automatic question answering QA is an alternative to traditional word-based search engines. Instead of returning a long list of documents more or less related to the query parameters the aim of a QA system is to isolate the exact answer as accurately as possible and to provide the user only a short text clip containing the required information. One of the major development challenges is evaluation. The conferences such as TREC1 CLEF2 and NTCIR3 have provided valuable QA evaluation methods and .

• Báo cáo khoa học
• Collecting a Why question corpus
• development and evaluation
• automatic QA system
• báo cáo khoa học
• báo cáo ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học