Interactive Natural Language Processing

KeyboardThe current state of the art in different areas of natural language processing (NLP) is very far from allowing fully automatic high quality results (HQRs), therefore human intervention is required to correct the output of the NLP engines. This applies specifically to NLP fields such as: machine translation and cross-language processing, text recognition, parsing, speech recognition, information retrieval, etc.

Its goal is to produce HQRs through a tight collaboration between a human operator and a NLP system, following an interactive-predictive paradigm. On the other hand, interactivity offers a unique context in which the feedback provided by the human can be used as new training data for adapting the NLP systems to new environments.

Automatic Speech Recognition and Understanding

Dion speechHuge amounts of audiovisual media are generated on a daily basis: parliamentary session, private meetings, TV and radio shows, public speeches, medical recordings, and many more. The magnitude of such quantity of information makes it impossible to be managed efficiently solely by human intervention. Automatic Speech Recognition and Understanding (ASRU) comes in handy when managing and indexing automatically such large amounts of audiovisual content.

Multimodal Interaction and Adaptive Learning

Traditional Pattern Recognition (PR) and Machine Learning (ML) have generally focused on full automation; that is, in developing technologies ultimately aiming at fully replacing human beings in tasks that require complex perceptive and/or cognitive skills. However, full automation often proves elusive or unnatural in many applications where technology is expected to assist rather than replace the human agents. This asks for a paradigm shift which should place PR/ML within the framework of human interaction.

Multimodal Interaction and Adaptive Learning deals with the fundamental work needed to address the research challenges and opportunities entailed by this paradigm shift. These include: interaction analysis and modelling, multimodal processing and fusion, interactive performance estimation and measurement, and several emerging forms of machine learning that look especially promising in the interactive framework (online, adaptive, active, semi-supervised, limited feedback, reinforcement, etc.).