Đang chuẩn bị nút TẢI XUỐNG, xin hãy chờ
Tải xuống
This paper presents PEAS, the first comparative evaluation framework for parsers of French whose annotation formalism allows the annotation of both constituents and functional relations. A test corpus containing an assortment of different text types has been built and part of it has been manually annotated. Precision/Recall and crossing brackets metrics will be adapted to our formalism and applied to the parses produced by one parser from academia and another one from industry in order to validate the framework. . | PEAS the first instantiation of a comparative framework for evaluating parsers of French V. Gendner G. Illouz M. Jardino L. Monceaux p. Paroubek I. Robba A. Vilnat LIMSI - CNRS BP 133 91403 Orsay - France gendner gabrieli jardino monceaux pap isabelle anne @limsi.fr Abstract This paper presents PEAS the first comparative evaluation framework for parsers of French whose annotation formalism allows the annotation of both constituents and functional relations. A test corpus containing an assortment of different text types has been built and part of it has been manually annotated. Precision Recall and crossing brackets metrics will be adapted to our formalism and applied to the parses produced by one parser from academia and another one from industry in order to validate the framework. 1 Introduction In natural language understanding many complex applications use a syntactic parser as a basic functionality. Today in particular for the French language the developers face the great diversity of the offer in the domain. Therefore the need for a complete comparative evaluation framework - including a pivot annotation formalism a reference treebank evaluation metrics and the associated software - is increasing. It is worth noting that most of the recently developed parsers use a robust approach. Consequently they do not always produce a complete parse of the sentence but they are able to produce a result whatever the size the particularities and the grammaticality of the input. For this reason it is essential to be able to compare in a fair way the parses they produce against those produced by other parsers whatever thefr characteristics. One possible solution is to offer a common reference annotation formalism along with a fully parsed reference corpus and a set of robust metrics allowing for both complete and selective evaluation over an assortment of different text types and syntactic phenomena. The aim of our research is to build such evaluation framework which to date
