TAILIEUCHUNG - Báo cáo khoa học: "Investigating GIS and Smoothing for Maximum Entropy Taggers"

This paper investigates two elements of Maximum Entropy tagging: the use of a correction feature in the Generalised Iterative Scaling (Gis) estimation algorithm, and techniques for model smoothing. We show analytically and empirically that the correction feature, assumed to be required for the correctness of GIS, is unnecessary. We also explore the use of a Gaussian prior and a simple cutoff for smoothing. The experiments are performed with two tagsets: the standard Penn Treebank POS tagset and the larger set of lexical types from Combinatory Categorial Grammar. . | Investigating GIS and Smoothing for Maximum Entropy Taggers James R. Curran and Stephen Clark School of Informatics University of Edinburgh 2 Buccleuch Place Edinburgh. EH8 9LW jamesc stephenc @ Abstract This paper investigates two elements of Maximum Entropy tagging the use of a coưection feature in the Generalised Iterative Scaling Gis estimation algorithm and techniques for model smoothing. We show analytically and empirically that the correction feature assumed to be required for the correctness of GIS is unnecessary. We also explore the use of a Gaussian prior and a simple cutoff for smoothing. The experiments are performed with two tagsets the standard Penn Treebank POS tagset and the larger set of lexical types from Combinatory Categorial Grammar. 1 Introduction The use of maximum entropy ME models has become popular in Statistical NLP some example applications include part-of-speech POS tagging Ratnaparkhi 1996 parsing Ratnaparkhi 1999 Johnson et al. 1999 and language modelling Rosenfeld 1996 . Many tagging problems have been successfully modelled in the ME framework including POS tagging with state of the art performance van Halteren et al. 2001 supertagging Clark 2002 and chunking Koeling 2000 . Generalised Iterative Scaling GIS is a very simple algorithm for estimating the parameters of a ME model. The original formulation of GIS Dar-roch and Ratcliff 1972 required the sum of the feature values for each event to be constant. Since this is not the case for many applications the standard method is to add a correction or slack feature to each event. Improved Iterative Scaling IIS Berger et al. 1996 Della Pietra et al. 1997 eliminated the correction feature to improve the convergence rate of the algorithm. However the extra book keeping required for IIS means that GIS is often faster in practice Malouf 2002 . This paper shows by a simple adaptation of Berger s proof for the convergence of IIS Berger 1997 that GIS does not require a correction

• Báo cáo khoa học
• Investigating GIS and Smoothing
• Maximum Entropy Taggers
• James R
• Curran
• scientific reports
• model language
• process natural language
• Dialog Control
• Natural Language System
• Dialog Control in a Natural Language System
• Expanding the Horizons
• Natural Language Interfaces
• Phil Hayes
• PROSPECTS OR PRACTICAL
• NATURAL LANGUAGE SYSTEMS
• Workshops
• NATURAL LANGUAGE
• INTERACTION WITH MACHINES
• Long Papers
• COMPUTER INTEBFACE DESIGN MURRAY TUROFF DEPARTMENT OF COMPUTER
• IiVFORMATION SCIENCE IIEW JERSEY INSTITUTE OF TECHNOLOGY
• EVALUATION OF NATURAL LANGUAGE
• INTERFACES TO DATABASE SYSTEMS
• A PANEL DISCUSSION
• database query system
• J
• Norwood Crout
• INTERFACES TO DATA BASE SYSTEMS
• Bozena Henisz
• INTERPRETING NATURAL LANGUAGE
• DATABASE UPDATES
• S
• Jermld Kaplan Jim David
• TRANSPORTABLE NATURAL LANGUAGE
• INTERFACES TO DATABASES
• Gary G
• Hendrlx and William H
• Lewis
• DATABASES THEORETICAL
• TECHNICAL ISSUES
• TRANSPORTABLE NATURAL LANGUAGE INTERFACES
• PROBLEMS AND TECHNIQUES
• Barbara J
• Grosz
• THEORETICAL TECHNICAL ISSUES
• NATURAL LANGUAGE ACCESS TO DATABASES
• R
• Petrick
• Domain Independent
• Database Access Systems
• ISSUES IN NATURAL LANGUAGE
• ACCESS TO DATABASES
• A LOGIC PROGRAMMING PERSPECTIVE
• Sharon C
• Salveter David Maier
• PLANNING NATURAL LANGUAGE
• EXPRESSIONS REFERRING
• Douglas E
• Appelt
• THE TEXT SYSTEM
• NATURAL LANGUAGE GENERATION
• Kathleen R
• M
• MENTING A DATABASE
• KNOWLEDGE REPRESENTATION
• THE KEY TO THE SELECTION PROBLEM
• E
• Jeffrey Conklin David D
• McDonald
• A KNOWLEDGE ENGINEERING
• NATURAL LANGUAGE UNDERSTANDING
• Stuart C
• Shapiro
• NATURAL LANGUAGE TEXTS
• NECESSARILY GRAMMATICAL
• EVEN COMPLETE
• UNGRAMHATICALITY
• EXTRA GRAMMATICALITY
• NATURAL LANGUAGE UNDERSTANDING SYSTEMS
• Taxonomy
• Descriptions
• Individuals in Natural
• Language Understanding
• The Use of Ooject Special Knowledge
• Natural Language Processing
• Mark H
• Bursteln