TAILIEUCHUNG - Báo cáo khoa học: "Hierarchical Bayesian Language Modelling for the Linguistically Informed"

In this work I address the challenge of augmenting n-gram language models according to prior linguistic intuitions. I argue that the family of hierarchical Pitman-Yor language models is an attractive vehicle through which to address the problem, and demonstrate the approach by proposing a model for German compounds. In an empirical evaluation, the model outperforms the Kneser-Ney model in terms of perplexity, and achieves preliminary improvements in English-German translation. | Hierarchical Bayesian Language Modelling for the Linguistically Informed Jan A. Botha Department of Computer Science University of Oxford UK Abstract In this work I address the challenge of augmenting n-gram language models according to prior linguistic intuitions. I argue that the family of hierarchical Pitman-Yor language models is an attractive vehicle through which to address the problem and demonstrate the approach by proposing a model for German compounds. In an empirical evaluation the model outperforms the Kneser-Ney model in terms of perplexity and achieves preliminary improvements in English-German translation. 1 Introduction The importance of effective language models in machine translation MT and automatic speech recognition ASR is widely recognised. n-gram models in particular ones using Kneser-Ney KN smoothing have become the standard workhorse for these tasks. These models are not ideal for languages that have relatively free word order and or complex morphology. The ability to encode additional linguistic intuitions into models that already have certain attractive properties is an important piece of the puzzle of improving machine translation quality for those languages. But despite their widespread use KN n-gram models are not easily extensible with additional model components that target particular linguistic phenomena. I argue in this paper that the family of hierarchical Pitman-Yor language models HPYLM Teh 2006 Goldwater et al. 2006 are suitable for investigations into more linguistically-informed n-gram language models. Firstly the flexibility to specify arbitrary back-off distributions makes it easy to incorporate multiple models into a larger n-gram model. Secondly the Pitman-Yor process prior Pitman and Yor 1997 generates distributions that are well-suited to a variety of powerlaw behaviours as is often observed in language. Catering for a variety of those is important since the frequency distributions of say suffixes

• Báo cáo khoa học
• Hierarchical Bayesian
• Language Modelling
• the Linguistically Informed
• scientific reports
• model language
• process natural language
• A Hierarchical Bayesian
• Language Model
• Pitman Yor Processes
• báo cáo khoa học
• mô hình ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Public health
• Integrated nested laplace approximation
• Latent gaussian model
• Bayesian hierarchical model
• Identifying risk factors
• BMC Bioinformatics
• Hierarchical latent variable models
• Misclassification errors
• Disease loci
• Human physiology
• Cox survival models
• Grouped predictors
• Hierarchical modeling
• Spike and slab prior
• Data integration
• Dirichlet multinomial
• Microbiome data
• Variable selection
• Bayesian networks
• Genetic regulatory networks
• Hierarchical bayesian modelling
• Biological interactions
• BMC Genomics
• HLA genotyping
• Next generation sequencing
• Whole genome sequencing
• Whole exome sequencing
• Cancer immunogenomics
• Markov chain Monte Carlo
• Genome Biology
• Alternative splicing
• Differential splicing
• Differential transcript usage
• Bayesian hierarchical modelling
• Non negative matrix factorization
• Bayesian hierarchical modeling
• Disease processes
• Analyze clinical data
• báo cáo hóa học
• công trình nghiên cứu về hóa học
• tài liệu về hóa học
• cách trình bày báo cáo
• RNA Seq
• Longitudinal data
• Correlated data
• Allelic imbalance
• Hierarchical generalized linear mixed model
• High throughput sequencing experiments
• Single nucleotide polymorphism
• Bayesian hierarchical models
• Flow cytometry
• Model based clustering
• Healthy individuals
• Differential expression
• Differential alternative splicing
• RNA seq experiments
• DNA methylation
• Bayesian analysis
• 5 methylcytosine oxidation
• Bisulfite sequencing
• Single cell RNA seq
• Cellular heterogeneity
• High throughput sequencing
• Powerful statistical models
• Meaningful biological signal
• Advantage illustrated
• Drosophila dataset