TAILIEUCHUNG - Báo cáo khoa học: "A Model of Lexical Attraction and Repulsion*"

This paper introduces new methods based on exponential families for modeling the correlations between words in text and speech. While previous work assumed the effects of word co-occurrence statistics to be constant over a window of several hundred words, we show that their influence is nonstationary on a much smaller time scale. Empirical data drawn from English and Japanese text, as well as conversational speech, reveals that the "attraction" between words decays exponentially, while stylistic and syntactic contraints create a "repulsion" between words that discourages close co-occurrence. W e show that these characteristics are well described by simple mixture. | A Model of Lexical Attraction and Repulsion Doug Beeferman Adam Berger John Lafferty School of Computer Science Carnegie Mellon University Pittsburgh PA 15213 USA dougb aberger lafferty Abstract This paper introduces new methods based on exponential families for modeling the correlations between words in text and speech. While previous work assumed the effects of word co-occurrence statistics to be constant over a window of several hundred words we show that their influence is nonstationary on a much smaller time scale. Empirical data drawn from English and Japanese text as well as conversational speech reveals that the attraction between words decays exponentially while stylistic and syntactic contraints create a repulsion between words that discourages close co-occurrence. We show that these characteristics are well described by simple mixture models based on two-stage exponential distributions which can be trained using the EM algorithm. The resulting distance distributions can then be incorporated as penalizing features in an exponential language model. 1 Introduction One of the fundamental characteristics of language viewed as a stochastic process is that it is highly nonstationary. Throughout a written document and during the course of spoken conversation the topic evolves effecting local statistics on word occurrences. The standard trigram model disregards this nonstationarity as does any stochastic grammar which assigns probabilities to sentences in a contextindependent fashion. Research supported in part by NSF grant IRI-9314969 DARPA AASERT award DAAH04-95-1-0475 and the ATR Interpreting Telecommunications Research Laboratories. Stationary models are used to describe such a dynamic source for at least two reasons. The first is convenience stationary models require a relatively small amount of computation to train and to apply. The second is ignorance we know so little about how to model effectively the nonstationary characteristics of language

• Báo cáo khoa học
• A Model of Lexical
• Attraction and Repulsion
• Doug Beeferman
• scientific reports
• model language
• process natural language
• Combining a Statistical Language Model
• Logistic Regression to Predict the Lexical
• Syntactic Difficulty of Texts for FFL
• Bootstrapping a Unified Model of Lexical
• Phonetic Acquisition
• Micha Elsner
• báo cáo khoa học
• báo cáo ngôn ngữ
• ngôn ngữ tự nhiên
• A Large Scale Distributed Syntactic
• Semantic
• Lexical Language Model
• Machine Translation
• Lexical transfer
• a vector space model
• Eiichiro SUMITA
• mô hình ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Chronometric Studies
• Lexical Ambiguity Resolution
• Mark S
• Seidenberg
• COMPUTATIONAL PLEXITY
• LEXICAL FUNCTIONAL GRAMMAR
• Robert C
• Berwick
• Controlling Lexical Substitution
• Computer Text Generation
• Robert Granville
• LEXICAL SEMANTICS
• HUMAN COMPUTER COMMUNICATION
• Jarrett Rosenberg
• LEXICAL KNOWLEDGE BASES
• Robert A
• Ameler
• Long Papers
• MACHINE READABLE
• DICTIONARIES
• THE LEXICAL SYSTEM DATA BASES
• COMMONSENSE METAPHYSICS
• On the Acquisition of Lexical Entries
• The Perceptual Origin
• Thematic Relations
• THE LOGICAL ANALYSIS
• LEXICAL AMBIGUITY
• David Stallard
• Lexical Selection
• the Process of Language Generation
• LEXICAL ACCESS
• CONNECTED SPEECH RECOGNITION
• Ted Briscoe
• THE LEXICAL EXPRESSIONS
• SEMANTICS OF COMPARATIVE
• MULTI LEVEL SEMANTIC PROCESSOR
• AUTOMATIC ACQUISITION
• THE LEXICAL SEMANTICS OF VERBS
• SENTENCE FRAMES
• COMPUTER AIDED INTERPRETATION
• LEXICAL COOCCURRENCES
• Paola Velardi
• LEXICAL AND SYNTACTIC RULES
• TREE ADJOINING GRAMMAR
• Anne Abeill
• Choosing the Word Most Typical in Context
• a Lexical Co occurrence Network
• Philip Edmonds
• Lexical surprisal
• general predictor
• reading time
• A Large Scale Unified
• Lexical Semantic
• Resource Based on LMF
• Framework of Semantic Role Assignment
• Extended Lexical Conceptual Structure Comparison with VerbNet
• FrameNet
• Contextual Phrase Level Polarity Analysis
• Lexical Affect Scoring
• Syntactic N grams