TAILIEUCHUNG - Báo cáo khoa học: "Using Grammatical Relations to Compare Parsers"

We use the grammatical relations (GRs) described in Carroll et al. (1998) to compare a number of parsing algorithms A first ranking of the parsers is provided by comparing the extracted GRs to a gold standard GR annotation of 500 Susanne sentences: this required an implementation of GR extraction software for Penn Treebank style parsers. In addition, we perform an experiment using the extracted GRs as input to the Lappin and Leass (1994) anaphora resolution algorithm. This produces a second ranking of the parsers, and we investigate the number of errors that are caused by the incorrect GRs. . | Using Grammatical Relations to Compare Parsers Judita Preiss Computer Laboratory University of Cambridge Abstract We use the grammatical relations GRs described in Carroll et al. 1998 to compare a number of parsing algorithms. A first ranking of the parsers is provided by comparing the extracted GRs to a gold standard GR annotation of 500 Susanne sentences this required an implementation of GR extraction software for Penn Treebank style parsers. In addition we perform an experiment using the extracted GRs as input to the Lappin and Leass 1994 anaphora resolution algorithm. This produces a second ranking of the parsers and we investigate the number of errors that are caused by the incorrect GRs. 1 Introduction We investigate the usefulness of a grammatical relation GR evaluation method by using it to compare the performance of four full parsers and a GR finder based on a shallow parser. It is usually difficult to compare performance of different style parsers as the output trees can vary in structure. In this paper we use GRs to provide a common basis for comparing full and shallow parsers and Penn Treebank and Susanne structures. To carry out this comparison we implemented a GR extraction mechanism for Penn Treebank This work was supported by UK EPSRC project GR N36462 93 Robust Accurate Statistical Parsing . parses. Evaluating parsers using GRs as opposed to crossing brackets or labelled preci-sion recall metrics can be argued to give a more robust measure of performance Carroll et al. 1998 Clark and Hockenmaier 2002 . The main novelty of this paper is the use of the Carroll et al s GR evaluation method to compare the Collins model 1 and model 2 and Charniak parsers. An initial evaluation is provided by comparing the extracted GRs to a gold standard GR annotation of 500 Susanne sentences due to Carroll et al. To gain insight into the strengths and weaknesses of the different parsers we present a breakdown of the results for each type of

• Báo cáo khoa học
• Using Grammatical Relations
• Compare Parsers
• Judita Preiss
• scientific reports
• model language
• process natural language
• Dialog Control
• Natural Language System
• Dialog Control in a Natural Language System
• Expanding the Horizons
• Natural Language Interfaces
• Phil Hayes
• PROSPECTS OR PRACTICAL
• NATURAL LANGUAGE SYSTEMS
• Workshops
• NATURAL LANGUAGE
• INTERACTION WITH MACHINES
• Long Papers
• COMPUTER INTEBFACE DESIGN MURRAY TUROFF DEPARTMENT OF COMPUTER
• IiVFORMATION SCIENCE IIEW JERSEY INSTITUTE OF TECHNOLOGY
• EVALUATION OF NATURAL LANGUAGE
• INTERFACES TO DATABASE SYSTEMS
• A PANEL DISCUSSION
• database query system
• J
• Norwood Crout
• INTERFACES TO DATA BASE SYSTEMS
• Bozena Henisz
• INTERPRETING NATURAL LANGUAGE
• DATABASE UPDATES
• S
• Jermld Kaplan Jim David
• TRANSPORTABLE NATURAL LANGUAGE
• INTERFACES TO DATABASES
• Gary G
• Hendrlx and William H
• Lewis
• DATABASES THEORETICAL
• TECHNICAL ISSUES
• TRANSPORTABLE NATURAL LANGUAGE INTERFACES
• PROBLEMS AND TECHNIQUES
• Barbara J
• Grosz
• THEORETICAL TECHNICAL ISSUES
• NATURAL LANGUAGE ACCESS TO DATABASES
• R
• Petrick
• Domain Independent
• Database Access Systems
• ISSUES IN NATURAL LANGUAGE
• ACCESS TO DATABASES
• A LOGIC PROGRAMMING PERSPECTIVE
• Sharon C
• Salveter David Maier
• PLANNING NATURAL LANGUAGE
• EXPRESSIONS REFERRING
• Douglas E
• Appelt
• THE TEXT SYSTEM
• NATURAL LANGUAGE GENERATION
• Kathleen R
• M
• MENTING A DATABASE
• KNOWLEDGE REPRESENTATION
• THE KEY TO THE SELECTION PROBLEM
• E
• Jeffrey Conklin David D
• McDonald
• A KNOWLEDGE ENGINEERING
• NATURAL LANGUAGE UNDERSTANDING
• Stuart C
• Shapiro
• NATURAL LANGUAGE TEXTS
• NECESSARILY GRAMMATICAL
• EVEN COMPLETE
• UNGRAMHATICALITY
• EXTRA GRAMMATICALITY
• NATURAL LANGUAGE UNDERSTANDING SYSTEMS
• Taxonomy
• Descriptions
• Individuals in Natural
• Language Understanding
• The Use of Ooject Special Knowledge
• Natural Language Processing
• Mark H
• Bursteln