EDOARDO MARIA POLO

Dottore di ricerca

ciclo: XXXV


supervisore: Riccardo Barbieri

Titolo della tesi: Multimodal assessment of emotional responses by physiological monitoring: novel auditory and visual elicitation strategies in traditional and virtual reality environments

This doctoral thesis explores novel strategies to quantify emotions and listening effort through mon- itoring of physiological signals. Emotions are a complex aspect of the human experience, playing a crucial role in our survival and adaptation to the environment. The study of emotions fosters important applications, such as Human-Computer and Human-Robot interaction or clinical assess- ment and treatment of mental health conditions such as depression, anxiety, stress, chronic anger, and mood disorders. Listening effort is also an important area of study, as it provides insight into the listeners’ challenges that are usually not identified by traditional audiometric measures. The research is divided into three lines of work, each with a unique emphasis on the methods of emotion elicitation and the stimuli that are most effective in producing emotional responses, with a specific focus on auditory stimuli. The research fostered the creation of three experimental protocols, as well as the use of an available online protocol for studying emotional responses including monitoring of both peripheral and central physiological signals, such as skin conductance, respiration, pupil dilation, electrocardiogram, blood volume pulse, and electroencephalography. An emotional protocol was created for the study of listening effort using a speech-in-noise test designed to be short and not induce fatigue. The results revealed that the listening effort is a complex problem that cannot be studied with a univariate approach, thus necessitating the use of multiple physiological markers to study different physiological dimensions. Specifically, the findings demonstrate a strong association between the level of auditory exertion, the amount of attention and involvement directed towards stimuli that are readily comprehensible compared to those that demand greater exertion. Continuing with the auditory domain, peripheral physiological signals were studied in order to discriminate four emo- tions elicited in a subject who listened to music for 21 days, using a previously designed and publicly available protocol. Surprisingly, the processed physiological signals were able to clearly separate the four emotions at the physiological level, demonstrating that music, which is not typically studied extensively in the literature, can be an effective stimulus for eliciting emotions. Following these results, a flat-screen protocol was created to compare physiological responses to purely visual, purely auditory, and combined audiovisual emotional stimuli. The results show that auditory stimuli are more effective in separating emotions at the physiological level. The subjects were found to be much more attentive during the audio-only phase. In order to overcome the limitations of emotional protocols carried out in a laboratory environment, which may elicit fewer emotions due to being an unnatural setting for the subjects under study, a final emotional elicitation protocol was created using virtual reality. Scenes similar to reality were created to elicit four distinct emotions. At the physiological level, it was noted that this environment is more effective in eliciting emotions. To our knowledge, this is the first protocol specifically designed for virtual reality that elicits diverse emotions. Furthermore, even in terms of classification, the use of virtual reality has been shown to be superior to traditional flat-screen protocols, opening the doors to virtual reality for the study of conditions related to emotional control. Keywords: emotions, listening effort, physiological signals, virtual reality.

Produzione scientifica

11573/1670564 - 2022 - Comparison of different emotion stimulation modalities: an EEG signal analysis
Farabbi, Andrea; Polo, Edoardo M; Barbieri, Riccardo; Mainardi, Luca - 04b Atto di convegno in volume
congresso: 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (Glasgow)
libro: 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) - (978-1-7281-2782-8)

11573/1670558 - 2022 - Evaluation of Machine Learning Algorithms and Explainability Techniques to Detect Hearing Loss From a Speech-in-Noise Screening Test
Lenatti, Marta; Moreno-Sánchez, Pedro A; Polo, Edoardo M; Mollura, Maximiliano; Barbieri, Riccardo; Paglialonga, Alessia - 01a Articolo in rivista
rivista: AMERICAN JOURNAL OF AUDIOLOGY (American Speech Language and Hearing Association:10801 Rockville Pike:Rockville, MD 20852:(800)498-2071, (301)897-5700, EMAIL: subscribe@asha.org, subscriptions@asha.org, INTERNET: http://www.professional.asha.org, Fax: (301)897-7358) pp. 961-979 - issn: 1059-0889 - wos: WOS:000861859300011 (12) - scopus: 2-s2.0-85137992021 (15)

11573/1670565 - 2022 - Analysis of the skin conductance and pupil signals for evaluation of emotional elicitation by images and sounds
Polo, Edoardo Maria; Farabbi, Andrea; Mollura, Maximiliano; Barbieri, Riccardo; Paglialonga, Alessia; Mainardi, Luca - 04b Atto di convegno in volume
congresso: 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (Glasgow)
libro: 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) - (978-1-7281-2782-8)

11573/1615954 - 2021 - Emotion recognition from multimodal physiological measurements based on an interpretable feature selection method
Polo, E. M.; Mollura, M.; Lenatti, M.; Zanet, M.; Paglialonga, A.; Barbieri, R. - 04b Atto di convegno in volume
congresso: 43rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2021 (mex)
libro: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS - (978-1-7281-1179-7)

11573/1616521 - 2021 - Analysis of the Effect of Emotion Elicitation on the Cardiovascular System
Polo, E. M.; Mollura, M.; Zanet, M.; Lenatti, M.; Paglialonga, A.; Barbieri, R. - 04c Atto di convegno in rivista
rivista: COMPUTING IN CARDIOLOGY ([Place of publication not identified] : Computing in Cardiology) pp. 1-4 - issn: 2325-8861 - wos: WOS:000821955000160 (0) - scopus: 2-s2.0-85124765276 (1)
congresso: 2021 Computing in Cardiology, CinC 2021 (Brno, Czech Republic)

11573/1615944 - 2021 - Development and Evaluation of a Novel Method for Adult Hearing Screening: Towards a Dedicated Smartphone App
Polo, E. M.; Zanet, M.; Lenatti, M.; Van Waterschoot, T.; Barbieri, R.; Paglialonga, A. - 04b Atto di convegno in volume
congresso: 7th International Conference on Internet of Things (IoT) Technologies for HealthCare, HealthyIoT 2020 (N/A, Cyberspace)
libro: Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST - (978-3-030-69962-8; 978-3-030-69963-5)

11573/1488766 - 2021 - Preliminary Evaluation of a Novel Language Independent Speech-in-Noise Test for Adult Hearing Screening
Polo, Edoardo Maria; Zanet, Marco; Paglialonga, Alessia; Barbieri, Riccardo - 04b Atto di convegno in volume
congresso: 8th European Medical and Biological Engineering Conference, EMBEC 2020 (Portorož; Slovenia)
libro: 8th European Medical and Biological Engineering Conference - (978-3-030-64609-7; 978-3-030-64610-3)

11573/1616516 - 2021 - Evaluation of a Novel Speech-in-Noise Test for Hearing Screening: Classification Performance and Transducers' Characteristics
Zanet, M.; Polo, E. M.; Lenatti, M.; Van Waterschoot, T.; Mongelli, M.; Barbieri, R.; Paglialonga, A. - 01a Articolo in rivista
rivista: IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS (New York, NY : Institute of Electrical and Electronics Engineers, 2013-) pp. 4300-4307 - issn: 2168-2194 - wos: WOS:000728140900011 (4) - scopus: 2-s2.0-85112658406 (7)

11573/1488768 - 2020 - Assessment of Heart Rate Variability derived from Blood Pressure Pulse Recordings in Intensive Care Unit Patients
Mollura, Maximiliano; Maria Polo, Edoardo; Lehman, Li-Wei; Barbieri, Riccardo - 04b Atto di convegno in volume
congresso: Computing in Cardiology 2020 (Rimini; Italia)
libro: Computing in Cardiology 2020 - (978-1-7281-7382-5)

11573/1488762 - 2020 - An automated speech-in-noise test for remote testing: Development and preliminary evaluation
Paglialonga, A.; Polo, E. M.; Zanet, M.; Rocco, G.; Van Waterschoot, T.; Barbieri, R. - 01a Articolo in rivista
rivista: AMERICAN JOURNAL OF AUDIOLOGY (American Speech Language and Hearing Association:10801 Rockville Pike:Rockville, MD 20852:(800)498-2071, (301)897-5700, EMAIL: subscribe@asha.org, subscriptions@asha.org, INTERNET: http://www.professional.asha.org, Fax: (301)897-7358) pp. 564-576 - issn: 1059-0889 - wos: WOS:000573267600004 (13) - scopus: 2-s2.0-85091324656 (16)

© Università degli Studi di Roma "La Sapienza" - Piazzale Aldo Moro 5, 00185 Roma