TAILIEUCHUNG - Báo cáo khoa học: "A Statistical Model for Domain-Independent Text Segmentation"

We propose a statistical method that finds the maximum-probability segmentation of a given text. This method does not require training data because it estimates probabilities from the given text. Therefore, it can be applied to any text in any domain. An experiment showed that the method is more accurate than or at least as accurate as a state-of-the-art text segmentation system. | A Statistical Model for Domain-Independent Text Segmentation Masao Utiyama and Hitoshi Isahara Communications Research Laboratory 2-2-2 Hikaridai Seika-cho Soraku-gun Kyoto 619-0289 Japan mutiyama@ and isahara@ Abstract We propose a statistical method that finds the maximum-probability segmentation of a given text. This method does not require training data because it estimates probabilities from the given text. Therefore it can be applied to any text in any domain. An experiment showed that the method is more accurate than or at least as accurate as a state-of-the-art text segmentation system. 1 Introduction Documents usually include various topics. Identifying and isolating topics by dividing documents which is called text segmentation is important for many natural language processing tasks including information retrieval Hearst and Plaunt 1993 Salton et al. 1996 and summarization Kan et al. 1998 Nakao 2000 . In information retrieval users are often interested in particular topics parts of retrieved documents instead of the documents themselves. To meet such needs documents should be segmented into coherent topics. Summarization is often used for a long document that includes multiple topics. A summary of such a document can be composed of summaries of the component topics. Identification of topics is the task of text segmentation. A lot of research has been done on text segmentation Kozima 1993 Hearst 1994 Oku-mura and Honda 1994 Salton et al. 1996 Yaari 1997 Kan et al. 1998 Choi 2000 Nakao 2000 . A major characteristic of the methods used in this research is that they do not require training data to segment given texts. Hearst 1994 for example used only the similarity of word distributions in a given text to segment the text. Consequently these methods can be applied to any text in any domain even if training data do not exist. This property is important when text segmentation is applied to information retrieval or summarization because both .

• Báo cáo khoa học
• A Statistical Model
• Domain Independent
• Text Segmentation
• báo cáo khoa học
• mô hình ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• A Discriminative Latent Variable Model
• Statistical Machine Translation
• A Discriminative Latent Variable Model for Statistical Machine Translation
• báo cáo ngôn ngữ
• Statistical model
• Rice yield prediction
• Cauvery delta zone
• Weather based statistical model
• Weather variables over the crop season
• Perplexity Minimization
• Translation Model Domain Adaptation
• scientific reports
• model language
• process natural language
• Improving Grammaticality
• Statistical Sentence Generation Introducing a Dependency Spanning Tree Algorithm
• Argument Satisfaction Model
• Combining a Statistical Language Model
• Logistic Regression to Predict the Lexical
• Syntactic Difficulty of Texts for FFL
• Adaptation of Statistical Machine Translation Model
• Cross Lingual Information Retrieval
• Service Context
• Nuclear engineering and technology
• Statistical model for forecasting uranium prices
• Estimate the nuclear fuel cycle cost
• Nuclear Fuel Cycle Cost
• Nuclear fuel cycle cost calculation
• The statistical autoregressive integrated moving average
• Sesame crop
• Forecast and ARIMA model
• Productivity of sesame crop
• Statistical model for forecasting area
• Forecasting area
• Fast and Scalable Decoding
• Language Model Look Ahead
• Phrase based Statistical Machine Translation
• ngôn ngữ tự nhiên
• Translation Model Size Reduction
• Hierarchical Phrase based
• Translation Model Adaptation
• Monolingual Topic Information
• A Statistical Model for Unsupervised
• Semi supervised Transliteration Mining
• Hassan Sajjad Alexander
• Lost Language Decipherment
• Benjamin Snyder and Regina Barzilay
• On line Language Model Biasing
• Sankaranarayanan Ananthakrishnan
• A non contiguous Tree Sequence Alignment based Model
• Jun Sun
• A Linguistically Annotated
• Reordering Model
• BTG based Statistical Machine Translation
• Maximum Entropy Based Phrase Reordering Model
• Deyi Xiong Qun Liu and Shouxun Lin
• A Phrase based Statistical Model
• SMS Text Normalization
• AiTi Aw
• Min Zhang
• A Clustered Global
• Phrase Reordering Model
• Masaaki Nagata Kuniko Saito
• A Statistical Machine Translation Model
• Synthetic Synchronous Grammar
• Hongfei Jiang
• Muyun Yang
• Tiejun Zhao
• Sheng Li and Bo Wang
• A Joint Statistical Model
• Simultaneous Word Spacing
• Spelling Error Correction for Korean
• A Hierarchical Phrase Based Model
• David Chiang
• A Localized Prediction Model
• Christoph Tillmann and Tong Zhang
• A Unified Statistical Model
• Identification of English BaseNP
• Endong Xun