Đang chuẩn bị nút TẢI XUỐNG, xin hãy chờ
Tải xuống
The quality of the part-of-speech (PoS) annotation in a corpus is crucial for the development of PoS taggers. In this paper, we experiment with three complementary methods for automatically detecting errors in the PoS annotation for the Icelandic Frequency Dictionary corpus. The first two methods are language independent and we argue that the third method can be adapted to other morphologically complex languages. Once possible errors have been detected, we examine each error candidate and hand-correct the corresponding PoS tag if necessary. . | Correcting a PoS-tagged corpus using three complementary methods Hrafn Loftsson School of Computer Science Reykjavik University Reykjavik Iceland hrafn@ru.is Abstract The quality of the part-of-speech PoS annotation in a corpus is crucial for the development of PoS taggers. In this paper we experiment with three complementary methods for automatically detecting errors in the PoS annotation for the Icelandic Frequency Dictionary corpus. The first two methods are language independent and we argue that the third method can be adapted to other morphologically complex languages. Once possible errors have been detected we examine each error candidate and hand-correct the corresponding PoS tag if necessary. Overall based on the three methods we hand-correct the PoS tagging of 1 334 tokens 0.23 of the tokens in the corpus. Furthermore we re-evaluate existing state-of-the-art PoS taggers on Icelandic text using the corrected corpus. 1 Introduction Part-of-speech PoS tagged corpora are valuable resources for developing PoS taggers i.e. programs which automatically tag each word in running text with morphosyntactic information. Corpora in various languages such as the English Penn Treebank corpus Marcus et al. 1993 the Swedish Stockholm-Umeả corpus Ejerhed et al. 1992 and the Icelandic Frequency Dictionary IFD corpus Pind et al. 1991 have been used to train in the case of data-driven methods and develop in the case of linguistic rule-based methods different taggers and to evaluate their accuracy e.g. van Halteren et al. 2001 Megyesi 2001 Loftsson 2006 . Consequently the quality of the PoS annotation in a corpus the gold standard annotation is crucial. Many corpora are annotated semi-automatically. First a PoS tagger is run on the corpus text and then the text is hand-corrected by humans. Despite human post-editing large tagged corpora are almost certain to contain errors because humans make mistakes. Thus it is important to apply known methods and or develop new methods for .
