TAILIEUCHUNG - Báo cáo khoa học: "Discriminative Training of a Neural Network Statistical Parser"

Discriminative methods have shown significant improvements over traditional generative methods in many machine learning applications, but there has been difficulty in extending them to natural language parsing. One problem is that much of the work on discriminative methods conflates changes to the learning method with changes to the parameterization of the problem. We show how a parser can be trained with a discriminative learning method while still parameterizing the problem according to a generative probability model. We present three methods for training a neural network to estimate the probabilities for a statistical parser, one generative | Discriminative Training of a Neural Network Statistical Parser James HENDERSON School of Informatics University of Edinburgh 2 Buccleuch Place Edinburgh EH8 9LW United Kingdom Abstract Discriminative methods have shown significant improvements over traditional generative methods in many machine learning applications but there has been difficulty in extending them to natural language parsing. One problem is that much of the work on discriminative methods conflates changes to the learning method with changes to the parameterization of the problem. We show how a parser can be trained with a discriminative learning method while still parameterizing the problem according to a generative probability model. We present three methods for training a neural network to estimate the probabilities for a statistical parser one generative one discriminative and one where the probability model is generative but the training criteria is discriminative. The latter model outperforms the previous two achieving state-of-the-art levels of performance F-measure on constituents . 1 Introduction Much recent work has investigated the application of discriminative methods to NLP tasks with mixed results. Klein and Manning 2002 argue that these results show a pattern where discriminative probability models are inferior to generative probability models but that improvements can be achieved by keeping a generative probability model and training according to a discriminative optimization criteria. We show how this approach can be applied to broad coverage natural language parsing. Our estimation and training methods successfully balance the conflicting requirements that the training method be both computationally tractable for large datasets and a good approximation to the theoretically optimal method. The parser which uses this approach outperforms both a generative model and a discriminative model achieving state-of-the-art levels of performance F-measure on .

• Báo cáo khoa học
• Discriminative Training
• Neural Network Statistical Parser
• James HENDERSON
• báo cáo khoa học
• mô hình ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Joint Feature Selection
• Distributed Stochastic Learning
• Large Scale Discriminative Training in SMT
• báo cáo ngôn ngữ
• ngôn ngữ tự nhiên
• Joint Processing
• Letter to Phoneme Conversion
• A Discriminative Global
• Training Algorithm
• Statistical MT
• Soft Syntactic Constraints
• Word Alignment
• Maximum Entropy Models
• Statistical Machine Translation
• Vietnamese automatic speech recognition
• Transcription system
• Force alignment procedure
• State of the art Vietnamese ASR
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học