TAILIEUCHUNG - Báo cáo khoa học: "Unsupervised Discovery of Persian Morphemes"

On the other hand, the construction of a comprehensive morphological analyzer for a language based on linguistic theory requires a considerable amount of work by experts. This is both slow and expensive and therefore not applicable to all languages. Consequently, it is important to develop methods that are able to discover and induce morphology for a language based on unsupervised analysis of large amounts of data. | Unsupervised Discovery of Persian Morphemes Mohsen Arabsorkhi Computer Science and Engineering Dept. Shiraz University Shiraz Iran marabsorkhi@ Mehrnoush Shamsfard Electrical and Computer Engineering Dept. Shahid Beheshti University Tehran Iran m-shams@ Abstract This paper reports the present results of a research on unsupervised Persian morpheme discovery. In this paper we present a method for discovering the morphemes of Persian language through automatic analysis of corpora. We utilized a Minimum Description Length MDL based algorithm with some improvements and applied it to Persian corpus. Our improvements include enhancing the cost function using some heuristics preventing the split of high frequency chunks exploiting penalty for first and last letters and distinguishing pre-parts and post-parts. Our improved approach has raised the precision recall and f-measure of discovery by respectively 32 17 and 23. 1 Introduction According to linguistic theory morphemes are considered to be the smallest meaning-bearing elements of a language. However no adequate language-independent definition of the word as a unit has been agreed upon. If effective methods can be devised for the unsupervised discovery of morphemes they could aid the formulation of a linguistic theory of morphology for a new language. The utilization of morphemes as basic representational units in a statistical language model instead of words seems a promising course Creutz 2004 . Many natural language processing tasks including parsing semantic modeling information retrieval and machine translation frequently require a morphological analysis of the language at hand. The task of a morphological analyzer is to identify the lexeme citation form or inflection class of surface word forms in a language. It seems that even approximate automated morphological analysis would be beneficial for many NL applications dealing with large vocabularies . text retrieval applications . On the

• Báo cáo khoa học
• Unsupervised Discovery
• Persian Morphemes
• Mohsen Arabsorkhi
• scientific reports
• model language
• process natural language
• Unsupervised Relation Discovery
• Sense Disambiguation
• Limin Yao
• báo cáo khoa học
• báo cáo ngôn ngữ
• ngôn ngữ tự nhiên
• Improved Unsupervised POS Induction
• Prototype Discovery
• Omri Abend
• Domain Specific Knowledge from Text
• Dirk Hovy
• Chunliang Zhang
• Rhyme Schemes
• Sravana Reddy
• Generic Relationships
• Pattern Clusters and its Evaluation by Automatically Generated SAT Analogy Questions
• xử lý ngôn ngữ tự nhiên
• Efficient Unsupervised Discovery of Word Categories
• Symmetric Patterns
• High Frequency Words
• mô hình ngôn ngữ
• Fully Unsupervised Discovery
• Concept Specific Relationships
• Web Mining
• BMC Bioinformatics
• Unsupervised feature extraction
• Principal component analysis
• Variational Bayes
• Posttraumatic stress disorder
• Heart disease
• In silico drug discovery
• Unsupervised learning
• Environmental genomics
• Sequence clustering
• Downstream analyses
• Genome Biology
• Single cell DNA sequencing
• Single cell variant calling
• Somatic variation
• Accurate variant calling