TAILIEUCHUNG - Báo cáo khoa học: "Unification-based Multimodal Parsing"

In order to realize their full potential, multimodal systems need to support not just input from multiple modes, but also synchronized integration of modes. Johnston et al (1997) model this integration using a unification operation over typed feature structures. This is an effective solution for a broad class of systems, but limits multimodal utterances to combinations of a single spoken phrase with a single gesture. We show how the unification-based approach can be scaled up to provide a full multimodal grammar formalism. In conjunction with a multidimensional chart parser, this approach supports integration of multiple elements distributed across the. | Unification-based Multimodal Parsing Michael Johnston Center for Human Computer Communication Department of Computer Science and Engineering Oregon Graduate Institute . Box 91000 Portland OR 97291-1000 johnston @ Abstract In order to realize their full potential multimodal systems need to support not just input from multiple modes but also synchronized integration of modes. Johnston et al 1997 model this integration using a unification operation over typed feature structures. This is an effective solution for a broad class of systems but limits multimodal utterances to combinations of a single spoken phrase with a single gesture. We show how the unification-based approach can be scaled up to provide a full multimodal grammar formalism. In conjunction with a multidimensional chart parser this approach supports integration of multiple elements distributed across the spatial temporal and acoustic dimensions of multimodal interaction. Integration strategies are stated in a high level unificationbased rule formalism supporting rapid prototyping and iterative development of multimodal systems. 1 Introduction Multimodal interfaces enable more natural and efficient interaction between humans and machines by providing multiple channels through which input or output may pass. Our concern here is with multimodal input such as interfaces which support simultaneous input from speech and pen. Such interfaces have clear task performance and user preference advantages over speech only interfaces in particular for spatial tasks such as those involving maps Ovi-att 1996 . Our focus here is on the integration of input from multiple modes and the role this plays in the segmentation and parsing of natural human input. In the examples given here the modes are speech and pen but the architecture described is more general in that it can support more than two input modes and modes of other types such as 3D gestural input. Our multimodal interface technology is implemented in

• Báo cáo khoa học
• Unification based
• Multimodal Parsing
• Michael Johnston
• báo cáo khoa học
• mô hình ngôn ngữ
• xử lý ngôn ngữ tự nhiên
• Bài giảng Xử lý ngôn ngữ tự nhiên
• Ngôn ngữ tự nhiên
• Kỹ thuật lập trình
• Xử lý nhập nhằng ngữ nghĩa
• Xử lý ngữ nghĩa
• Xử lý ngôn ngữ
• Natural Language Processing
• Ngôn ngữ lập trình
• Nhập nhằng cấu trúc
• Tri thức về ngôn ngữ
• Mô hình n gram
• Dịch máy
• Phương pháp dịch máy
• Hiểu ngôn ngữ
• Phân tích ngữ nghĩa
• Phân loại tin tự động
• Biểu diễn vị từ
• Thuộc tính về sự kiện
• Thuộc tính về sự kiện
• Mô hình ngôn ngữ Google Book N grams
• Mô hình ngôn ngữ KenLM
• Phương pháp làm mịn
• Tách từ tiếng Việt
• Gán nhãn từ loại
• Phân tích cú pháp
• Nghĩa từ vựng
• Phân giải nhập nhằng từ
• Phân loại văn bản
• Lexical chain
• Phản hồi thông tin
• Bài toán PTCP
• Cấu trúc ngữ pháp
• Phân tích cú pháp xác suất
• CKY kết hợp xác suất
• Văn phạm phi ngữ cảnh xác suất
• Hình thái học