Task Integration in Multimodal Speech Recognition Environments

A model of complementary behavior has been proposed based on arguments that direct manipulation and speech recognition interfaces have reciprocal strengths and weaknesses. This suggests that user interface performance and acceptance may increase by adopting a multimodal approach that combines speech and direct manipulation. More theoretical work is needed in order to understand how to leverage this advantage. In this paper, a framework is presented to empirically evaluate the types of tasks that might benefit from such a multimodal interface.