Dottorato in INGEGNERIA INFORMATICA

Titolo della tesi: Human-Robot Interaction in Outdoor Collaborative Environments

Human-Robot Interaction (HRI) is a multifaceted field, where the choice of interaction modalities and communication protocols depend on various factors, including the application domain, task specifics, robot types, and sensor configurations. Virtual Reality (VR) has emerged as a valuable tool for assessing HRI solutions, with a strong emphasis on safety and correctness in joint tasks. This approach enables researchers to validate their methods in simulated environments, enhancing effectiveness, while avoiding potential real-world risks. The main goal of this thesis is to provide an effective solution for HRI in collaborative environments by relying upon VR. The method has been evaluated in the context of precision agriculture, specifically in the table-grape vineyard scenario of the CANOPIES project. In collaborative robotics, a domain where Human-Robot Collaboration (HRC) is essential, understanding the information exchanged during the interaction between humans and robots is crucial. To address this challenge, a novel categorization of the informative content in the context of HRI in shared environments, along with a method for speech act classification, is introduced and evaluated through user feedback in both immersive and non-immersive VR experiences. Such an approach, tested in the demanding context of table-grape vineyards, sheds light on production settings involving cobots. Speech, as a natural interaction modality for humans, plays a pivotal role in collaborative robotics. To enhance robust and effective communication in precision agriculture, a dataset constituted of textual transcriptions of spoken utterances, with associated content information, is created and used to train the developed speech-based pipeline. The system leverages informative content and frame semantics, providing a modality-independent approach to information representation and enabling efficient dialog management between humans and robots. Despite the advantages of speech-based communication, in order to overcome some limitations associated with the adoption of a unique transmission channel, gestural interaction for Human-to-Robot communication in outdoor collaborative scenarios has been explored. To this aim, a novel standardization of gestures for ground robots is introduced, and the generation of a large quantity of gestural data through human avatars is enabled by employing VR simulations. Therefore, an innovative gesture recognition pipeline, capable of assessing various combinations of real and virtual data, is proposed and evaluated. Given the potential of combining vocal, gestural, acoustical, and light-based information in a multimodal system to substantially improve the robot’s overall performance, by reducing uncertainty and minimizing communication misunderstandings, this thesis ultimately presents a multimodal approach. This approach is designed to recognize and process both spoken and gestural information from human teammates, thereby contributing to more effective HRI in challenging outdoor collaborative environments.