displaying: publications.artzi2014programming

Programming by Demonstration with Situated Semantic Parsing


*The first three authors contributed equally to this paper.


Programming by Demonstration (PbD) is an approach to programming robots by demonstrating the desired behavior (Billard et al. 2008). Speech is a natural, hands-free way to augment demonstrations with control commands that guide the PbD process. However, existing speech interfaces for PbD systems rely on ad-hoc, predefined command sets that are rigid and require user training (Weiss et al. 2009; Akgun et al. 2012; Cakmak and Takayama 2014). Instead, we aim to develop flexible speech interfaces to accommodate user variations and ambiguous utterances. To that end, we propose to use a situated semantic parser that jointly reasons about the user’s speech and the robot’s state to resolve ambiguities. In this paper, we describe this approach and compare its utility to a rigid speech command interface.

Additional information

I presented this paper at the AAAI 2014 Fall Symposium for Human-Robot Interaction.

full text