TAILIEUCHUNG - Báo cáo khoa học: "Sentiment Summarization: Evaluating and Learning User Preferences"

We present the results of a large-scale, end-to-end human evaluation of various sentiment summarization models. The evaluation shows that users have a strong preference for summarizers that model sentiment over non-sentiment baselines, but have no broad overall preference between any of the sentiment-based models. However, an analysis of the human judgments suggests that there are identifiable situations where one summarizer is generally preferred over the others. | Sentiment Summarization Evaluating and Learning User Preferences Kevin Lerman Columbia University New York NY klerman@ Sasha Blair-Goldensohn Google Inc. New York NY sasha@ Ryan McDonald Google Inc. New York NY ryanmcd@ Abstract We present the results of a large-scale end-to-end human evaluation of various sentiment summarization models. The evaluation shows that users have a strong preference for summarizers that model sentiment over non-sentiment baselines but have no broad overall preference between any of the sentiment-based models. However an analysis of the human judgments suggests that there are identifiable situations where one summarizer is generally preferred over the others. We exploit this fact to build a new summarizer by training a ranking SVM model over the set of human preference judgments that were collected during the evaluation which results in a 30 relative reduction in error over the previous best summarizer. 1 Introduction The growth of the Internet as a commerce medium and particularly the Web phenomenon of user-generated content have resulted in the proliferation of massive numbers of product service and merchant reviews. While this means that users have plenty of information on which to base their purchasing decisions in practice this is often too much information for a user to absorb. To alleviate this information overload research on systems that automatically aggregate and summarize opinions have been gaining interest Hu and Liu 2004a Hu and Liu 2004b Gamon et al. 2005 Popescu and Etzioni 2005 Carenini et al. 2005 Carenini et al. 2006 Zhuang et al. 2006 Blair-Goldensohn et al. 2008 . Evaluating these systems has been a challenge however due to the number of human judgments required to draw meaningful conclusions. Often systems are evaluated piecemeal selecting pieces that can be evaluated easily and automatically Blair-Goldensohn et al. 2008 . While this technique produces meaningful evaluations of

• Báo cáo khoa học
• Sentiment Summarization
• Evaluating and Learning
• User Preferences
• scientific reports
• model language
• process natural language
• A Bilingual Context Mining
• Sentiment Analysis Summarization System
• Wen Tai Hsieh
• báo cáo khoa học
• báo cáo ngôn ngữ
• ngôn ngữ tự nhiên
• Optimizing Informativeness and Readability
• Hitoshi Nishikawa
• Takaaki Hasegawa
• Yoshihiro Matsuo and Genichiro Kikui
• A Joint Model of Text
• Aspect Ratings
• xử lý ngôn ngữ tự nhiên
• A Sentimental Education
• Sentiment Analysis Using Subjectivity Summarization
• Minimum Cuts
• mô hình ngôn ngữ
• Dialog Control
• Natural Language System
• Dialog Control in a Natural Language System
• Expanding the Horizons
• Natural Language Interfaces
• Phil Hayes
• PROSPECTS OR PRACTICAL
• NATURAL LANGUAGE SYSTEMS
• Workshops
• NATURAL LANGUAGE
• INTERACTION WITH MACHINES
• Long Papers
• COMPUTER INTEBFACE DESIGN MURRAY TUROFF DEPARTMENT OF COMPUTER
• IiVFORMATION SCIENCE IIEW JERSEY INSTITUTE OF TECHNOLOGY
• EVALUATION OF NATURAL LANGUAGE
• INTERFACES TO DATABASE SYSTEMS
• A PANEL DISCUSSION
• database query system
• J
• Norwood Crout
• INTERFACES TO DATA BASE SYSTEMS
• Bozena Henisz
• INTERPRETING NATURAL LANGUAGE
• DATABASE UPDATES
• S
• Jermld Kaplan Jim David
• TRANSPORTABLE NATURAL LANGUAGE
• INTERFACES TO DATABASES
• Gary G
• Hendrlx and William H
• Lewis
• DATABASES THEORETICAL
• TECHNICAL ISSUES
• TRANSPORTABLE NATURAL LANGUAGE INTERFACES
• PROBLEMS AND TECHNIQUES
• Barbara J
• Grosz
• THEORETICAL TECHNICAL ISSUES
• NATURAL LANGUAGE ACCESS TO DATABASES
• R
• Petrick
• Domain Independent
• Database Access Systems
• ISSUES IN NATURAL LANGUAGE
• ACCESS TO DATABASES
• A LOGIC PROGRAMMING PERSPECTIVE
• Sharon C
• Salveter David Maier
• PLANNING NATURAL LANGUAGE
• EXPRESSIONS REFERRING
• Douglas E
• Appelt
• THE TEXT SYSTEM
• NATURAL LANGUAGE GENERATION
• Kathleen R
• M
• MENTING A DATABASE
• KNOWLEDGE REPRESENTATION
• THE KEY TO THE SELECTION PROBLEM
• E
• Jeffrey Conklin David D
• McDonald
• A KNOWLEDGE ENGINEERING
• NATURAL LANGUAGE UNDERSTANDING
• Stuart C
• Shapiro
• NATURAL LANGUAGE TEXTS
• NECESSARILY GRAMMATICAL
• EVEN COMPLETE
• UNGRAMHATICALITY
• EXTRA GRAMMATICALITY
• NATURAL LANGUAGE UNDERSTANDING SYSTEMS
• Taxonomy
• Descriptions
• Individuals in Natural
• Language Understanding
• The Use of Ooject Special Knowledge
• Natural Language Processing
• Mark H
• Bursteln