TAILIEUCHUNG - Báo cáo khoa học: "Cohesion and Collocation: Using Context Vectors in Text Segmentation"

Collocational word similarity is considered a source of text cohesion that is hard to measure and quantify. The work presented here explores the use of information from a training corpus in measuring word similarity and evaluates the method in the text segmentation task. An implementation, the V e c T i l e system, produces similarity curves over texts using pre-compiled vector representations of the contextual behavior of words. The performance of this system is shown to improve over that of the purely string-based TextTiling algorithm (Hearst, 1997). 1 Background . | Cohesion and Collocation Using Context Vectors in Text Segmentation Stefan Kaufmann CSLI Stanford University Linguistics Dept. Bldg. 460 Stanford CA 94305-2150 Abstract Collocational word similarity is considered a source of text cohesion that is hard to measure and quantify. The work presented here explores the use of information from a training corpus in measuring word similarity and evaluates the method in the text segmentation task. An implementation the VecTile system produces similarity curves over texts using pre-compiled vector representations of the contextual behavior of words. The performance of this system is shown to improve over that of the purely string-based TextTiling algorithm Hearst 1997 . 1 Background The notion of text cohesion rests on the intuition that a text is held together by a variety of internal forces. Much of the relevant linguistic literature is indebted to Halliday and Hasan 1976 where cohesion is defined as a network of relationships between locations in the text arising from i grammatical factors co-reference use of pro-forms ellipsis and sentential connectives and ii lexical factors reiteration and collocation . Subsequent work has further developed this taxonomy Hoey 1991 and explored its implications in such areas as paragraphing Longacre 1979 Bond and Hayes 1984 Stark 1988 relevance Sperber and Wilson 1995 and discourse structure Grosz and Sidner 1986 . The lexical variety of cohesion is semantically defined invoking a measure of word similarity. But this is hard to measure objectively especially in the case of collocational relationships which hold between words primarily because they regularly cooccur. Halliday and Hasan refrained from a deeper analysis but hinted at a notion of degrees of proximity in the lexical system a function of the probability with which one tends to co-occur with another. p. 290 The VecTile system presented here is designed to utilize precisely this kind of lexical .

• Báo cáo khoa học
• Cohesion and Collocation
• Using Context Vectors
• Text Segmentation
• báo cáo khoa học
• mô hình ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học