New Concept for Designing Multimodal User Interfaces

; (). New Concept for Designing Multimodal User Interfaces. In: ISCA (Hg.). ASIDE 2005: Applied Spoken Language Interaction in Distributed Environments. ISCA TR /COST278 Final Workshop.

This paper proposes a new concept for the design of multimodal user interfaces (MMUIs) based on so-called Multimodal Objects (MMOs). MMOs are a new kind of multimodal components we propose in order to simplify the construction of consistent MMUIs. An MMO is in the simplest case a basic interaction element like a single multimodal button or textfield. These MMOs are built according to the model-view-controller pattern. The model of the MMO is responsible for the logic of the MMO. The view of the MMO has different aspects, one for each modality. The controller of the MMO is responsible for the event fusion, i.e. to coordinate the different events coming from the different aspects of the view and compiling them into a semantic event of the MMO. This semantic event is then transmitted to the model of the MMO which normally results in a state change of the model.