LUCA FRANCO

Dottore di ricerca

ciclo: XXXVI


supervisore: Fabio Galasso

Titolo della tesi: Human Motion Modeling with Hyperbolic Uncertainty

This Ph.D. thesis delves into the domain of human motion modeling and uncertainty estimation. The first includes various aspects such as trajectory and pose forecasting, while the latter focuses on self-supervised representation for human action recognition and active domain adaptation for semantic segmentation. This thesis explores research work on human trajectory forecasting using recent transformer architectures, demonstrating how the motion patterns of people can be forecasted without bells and whistles. The versatility of these methods allows the extension to different domains. We encompass earthquake forecasting as well by leveraging the proposed model in a parallel domain. From macro-scale forecasting (i.e., trajectories), we present research on micro-scale forecasting (i.e., human poses). We introduce the Space-Time-Separable Graph Convolutional Network (STS-GCN), a novel approach for human pose forecasting. STS-GCN incorporates both temporal evolution and spatial joint interaction in a single-graph framework. It introduces the concept of space-time separability, bottlenecking motion-spatial cross-talk and capturing complete joint-joint and time-time correlations. The approach deviates from the conventional kinematic tree and linear-time series, enhancing complex structural joint spatio-temporal dynamics modeling in human pose forecasting. Following the human pose forecasting, the thesis delves into the topic of uncertainty estimation. We introduce HYSP, a novel HYperbolic Self-Paced model for learning action representations from skeleton-based data. HYSP leverages self-supervision by generating two views of the same sample through data augmentations and learns by matching the online view to the target view. To determine the learning pace, we propose the use of hyperbolic uncertainty, where certain samples are given higher weights and paces during training. Hyperbolic uncertainty arises naturally from the hyperbolic neural networks employed in HYSP, matures during training, and incurs no additional cost compared to traditional Euclidean SSL frameworks. To extend the scope of hyperbolic uncertainty research, we introduce the application of active learning for semantic segmentation using the Poincaré hyperbolic ball model. We propose a novel label acquisition strategy by exploiting the variations in the hyperbolic radii of pixel embeddings within regions. This strategy is based on a unique geometric property of the hyperbolic space, where classes are mapped to compact hyperbolic areas with comparable intra-class radii variance, as the model places classes of increasing explainable difficulty at denser hyperbolic areas. The variation of pixel embedding radii effectively captures class contours and reveals intra-class peculiar details, leading to enhanced performance in Semantic Segmentation tasks. Overall, uncertainty estimation with hyperbolic neural networks emerges as a promising yet open research problem within human motion modeling. This thesis offers novel insights and methodologies that contribute to the advancement of the field, laying the foundation for future research endeavors in this exciting area of study.

Produzione scientifica

11573/1711954 - 2024 - Hyperbolic Active Learning for Semantic Segmentation under Domain Shift
Franco, Luca; Mandica, Paolo; Kallidromitis, Konstantinos; Guillory, Devin; Li, Yu-Teng; Darrell, Trevor; Galasso, Fabio - 04b Atto di convegno in volume
congresso: International Conference on Machine Learning (Vienna, Austria)
libro: Hyperbolic Active Learning for Semantic Segmentation under Domain Shift - ()

11573/1675023 - 2023 - HYperbolic Self-Paced Learning for Self-Supervised Skeleton-based Action Representations
Franco, Luca; Mandica, Paolo; Munjal, Bharti; Galasso, Fabio - 04b Atto di convegno in volume
congresso: International Conference on Learning Representations (Kigali, Rwanda)
libro: The Eleventh International Conference on Learning Representations - ()

11573/1673543 - 2023 - Under the hood of transformer networks for trajectory forecasting
Franco, Luca; Placidi, Leonardo; Giuliari, Francesco; Hasan, Irtiza; Cristani, Marco; Galasso, Fabio - 01a Articolo in rivista
rivista: PATTERN RECOGNITION (Elsevier Science Limited:Oxford Fulfillment Center, PO Box 800, Kidlington Oxford OX5 1DX United Kingdom:011 44 1865 843000, 011 44 1865 843699, EMAIL: asianfo@elsevier.com, tcb@elsevier.co.UK, INTERNET: http://www.elsevier.com, http://www.elsevier.com/locate/shpsa/, Fax: 011 44 1865 843010) pp. 109372- - issn: 0031-3203 - wos: WOS:000944651200001 (9) - scopus: 2-s2.0-85148333409 (11)

11573/1675144 - 2023 - Using Deep Learning to understand variations in fault zone properties: distinguishing foreshocks from aftershocks
Laurenti, Laura; Paoletti, Gabriele; Tinti, Elisa; Galasso, Fabio; Franco, Luca; Collettini, Cristiano; Marone, Chris James - 04d Abstract in atti di convegno
congresso: European Geoscience Union General Assembly (Vienna)
libro: EGU European Geoscience Union General Assembly 2023 - (9781510812253)

11573/1656424 - 2022 - Deep learning for laboratory earthquake prediction and autoregressive forecasting of fault zone stress
Laurenti, Laura; Tinti, Elisa; Galasso, Fabio; Franco, Luca; Marone, Chris James - 01a Articolo in rivista
rivista: EARTH AND PLANETARY SCIENCE LETTERS (Elsevier BV:PO Box 211, 1000 AE Amsterdam Netherlands:011 31 20 4853757, 011 31 20 4853642, 011 31 20 4853641, EMAIL: nlinfo-f@elsevier.nl, INTERNET: http://www.elsevier.nl, Fax: 011 31 20 4853598) pp. - - issn: 0012-821X - wos: WOS:000878180900005 (22) - scopus: 2-s2.0-85139253799 (26)

11573/1617237 - 2021 - Space-time-separable graph convolutional network for pose forecasting
Sofianos, Theodoros; Sampieri, Alessio; Franco, Luca; Galasso, Fabio - 04b Atto di convegno in volume
congresso: IEEE International Conference on Computer Vision (Montreal; Canada)
libro: Proceedings of the IEEE/CVF international conference on computer vision (ICCV) - (978-1-6654-2812-5; 978-1-6654-2813-2)

© Università degli Studi di Roma "La Sapienza" - Piazzale Aldo Moro 5, 00185 Roma