A Multimodal Human-Robot Interaction Dataset

Pablo Azagra (1) , Yoan Mollard (2) , Florian Golemo (2) , Ana Cristina Murillo (1) , Manuel Lopes (2) , Javier Civera (1)

1 DIIS - Departamento de Informática e Ingeniería de Sistemas
2 Flowers - Flowing Epigenetic Robots and Systems

This works presents a multimodal dataset for Human-Robot Interactive Learning. 1 The dataset contains synchronized recordings of several human users, from a stereo 2 microphone and three cameras mounted on the robot. The focus of the dataset is 3 incremental object learning, oriented to human-robot assistance and interaction. To 4 learn new object models from interactions with a human user, the robot needs to 5 be able to perform multiple tasks: (a) recognize the type of interaction (pointing, 6 showing or speaking), (b) segment regions of interest from acquired data (hands and 7 objects), and (c) learn and recognize object models. We illustrate the advantages 8 of multimodal data over camera-only datasets by presenting an approach that 9 recognizes the user interaction by combining simple image and language features.

Domaines: Intelligence artificielle [cs.AI]

Pablo Azagra, Yoan Mollard, Florian Golemo, Ana Cristina Murillo, Manuel Lopes, et al.. A Multimodal Human-Robot Interaction Dataset. NIPS 2016, workshop Future of Interactive Learning Machines, Dec 2016, Barcelona, Spain. ⟨hal-01402479⟩