TAILIEUCHUNG - Báo cáo khoa học: "Statistical Models for Topic Segmentation"

Most documents are about more than one subject, but many NLP and IR techniques implicitly assume documents have just one topic. We describe new clues that mark shifts to new topics, novel algorithms for identifying topic boundaries and the uses of such boundaries once identified. We report topic segmentation performance on several corpora as well as improvement on an IR task that benefits from good segmentation. Introduction Dividing documents into topically-coherent sections has many uses, but the primary motivation for this work comes from information retrieval (IR). . | Statistical Models for Topic Segmentation Jeffrey c. Reynar1 Microsoft Corporation One Microsoft Way Redmond WA 98052 USA jreynar@ Abstract Most documents are about more than one subject but many NLP and IR techniques implicitly assume documents have just one topic. We describe new clues that mark shifts to new topics novel algorithms for identifying topic boundaries and the uses of such boundaries once identified. We report topic segmentation performance on several corpora as well as improvement on an IR task that benefits from good segmentation. Introduction Dividing documents into topically-coherent sections has many uses but the primary motivation for this work comes from information retrieval IR . Documents in many collections vary widely in length and while the shortest may address one topic modest length and long documents are likely to address multiple topics or be comprised of sections that address various aspects of the primary topic. Despite this fact most IR systems treat documents as indivisible units and index them in their entirety. This is problematic for two reasons. First most relevance metrics are based on word frequency which can be viewed as a function of the topic being discussed Church and Gale 1995 . For example the word header is rare in general English but it enjoys higher frequency in documents about soccer. In general word frequency is a good indicator of whether a document is relevant to a query but consider a long document containing only one section relevant to a query. If a keyword is used only in the pertinent section its overall frequency in the document will be low and as a result the document as a whole may be judged irrelevant despite the relevance of one section. The second reason it would be beneficial to index sections of documents is that once a search engine has identified a relevant document users would benefit from direct access to the relevant sections. This problem is compounded when searching multimedia .

• Báo cáo khoa học
• Statistical Models
• Topic Segmentation
• Jeffrey C
• Reynar
• báo cáo khoa học
• mô hình ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Statistical arbitrage
• Optimization procedure
• Global investments
• Portfolio optimization
• Statistical arbitrage models
• Flood forecasting including
• Physical models
• Data driven models
• Hydraulic models
• Hydrological processes
• Applied multivariate statistical analysis
• Statistical analysis
• Multivariate analysis
• Multivariate linear regression models
• Canonical correlation analysis
• Distance methods
• Mixing Multiple Translation Models
• Statistical Machine Translation
• SMT
• báo cáo ngôn ngữ
• ngôn ngữ tự nhiên
• Phrase based Statistical Language Generation
• Graphical Models
• Active Learning
• Enhancing Language Models
• Statistical Machine Translation with Backward N grams
• Mutual Information Triggers
• Automatic Story Segmentation
• Bayesian Decision Framework
• Lexical Chain Features
• Distortion Models
• Yaser Al Onaizan and Kishore Papineni
• Segmented and unsegmented
• dialogue act annotation
• statistical dialogue models∗
• Continuous Space Language Models
• Holger Schwenk and Daniel Dchelotte and Jean Luc Gauvain
• Statistical phrase based models
• interactive computer assisted translation
• Jesus Tom
• Combining Statistical
• Knowledge based Spoken Language Understanding
• Conditional Models
• Guiding Statistical
• Word Alignment Models
• Prior Knowledge
• Reading Level Assessment
• Support Vector Machines
• Statistical Language Models
• Generalized Algorithms
• Constructing Statistical Language Models
• Cyril Allauzen
• Refined Lexicon Models
• Maximum Entropy Approach
• Unsupervised Prepositional
• Phrase Attachment
• Discriminative Training
• Maximum Entropy Models
• Generative Models
• Statistical Parsing
• Combinatory Categorial Grammar
• Statistical Decision Tree Models for Parsing
• David M
• Magerman
• Long Papers
• Three Generative
• Lexicalised Models
• scientific reports
• model language
• process natural language
• Fertility Models
• Statistical Natural Language Understanding
• Workshops
• Generating statistical language models
• interpretation grammars
• dialogue systems
• BMC Plant Biology
• Nested association mapping
• Brassica napus L
• Double haploid NAM
• Backcross NAM
• Computer simulations