LORENZO GIUSTI

Dottore di ricerca

ciclo: XXXVI


supervisore: Stefano Leonardi

Titolo della tesi: Topological Neural Networks: Mitigating the Bottlenecks of Graph Neural Networks via Higher-Order Interactions

The irreducible complexity of natural phenomena has led Graph Neural Networks to be employed as a standard model to perform representation learning tasks on graph-structured data. While their capacity to capture local and global patterns is remarkable, the implications associated with long-range and higher-order dependencies pose considerable challenges to such models. This work addresses these challenges by starting with the identification of the aspects that negatively impact the performance of graph neural networks in learning representations of events that strongly depend on long-range interactions. In particular, when graph neural networks require to aggregate messages among distant nodes, the message passing scheme performs an over-squashing of an exponentially growing amount of information into static vectors. It is important to notice that for some classes of graphs (i.e., path, tree, grid, ring, and ladder) the underlying connectivity allows messages to travel along edges without encountering significant interference from other paths, thus reducing the growth of information to be linear in the number of messages exchanged. When the underlying graph does not fall into the aforementioned categories, oversquashing arises because the propagation of information happens between nodes that are connected through edges, which induces a computational graph mirroring nodes’ connectivity. This phenomenon causes nodes to become insensitive to information sent from remote parts of the graph. To offer a new perspective for designing architectures that mitigate such bottlenecks, a unified theoretical framework reveals the impact of network’s width, depth, and graph topology on the over-squashing phenomena in message-passing neural networks. The thesis then drifts towards the exploitation of higher-order interactions via Topological Neural Networks. With a multi-relational inductive bias, topological neural networks propagate messages through higher-dimensional structures, effectively providing shortcuts or additional routes for information flow. With this construction, the underlying computational graph is no longer coupled with the input graph structure, thus mitigating the aforementioned bottlenecks while accounting also for higher-order interactions. Inspired by the masked self-attention mechanism developed in Graph Attention Networks alongside the rich connectivity provided by simplicial and cell complexes, two distinct attentional architectures are proposed: Simplicial Attention Networks and Cell Attention Networks. The rationale behind these architecture is to leverage the extended notion of neighbourhoods provided by the particular arrangement of groups of nodes within a simplicial or cell complex. In particular, these topological attention networks exploit the upper and lower adjacencies of the underlying complex to design anisotropic aggregations able to measure the importance of the information coming from different regions of the domain. By doing so, they capture dependencies that conventional Graph Neural Networks might miss. Finally, a communication scheme between higher-order structures is introduced with Enhanced Cellular Isomorphism Networks, which augment topological message passing schemes by letting all the cells of a cell complex receive messages from their lower neighbourhood. This upgrade enables direct interactions among node groups within a cell complex, specifically arranged in ring-like structures. This augmented scheme offers more comprehensive representation of higher-order and long-range interactions, demonstrating very high performance on large-scale and long-range benchmarks.

Produzione scientifica

11573/1687999 - 2023 - Cell Attention Networks
Giusti, Lorenzo; Battiloro, Claudio; Testa, Lucia; Di Lorenzo, Paolo; Sardellitti, Stefania; Barbarossa, Sergio - 04b Atto di convegno in volume
congresso: International Joint Conference on Neural Networks (IJCNN) 2023 (Gold Coast, Australia)
libro: 2023 International Joint Conference on Neural Networks (IJCNN) - (978-1-6654-8867-9)

11573/1670573 - 2023 - MaRF: Representing Mars as Neural Radiance Fields
Giusti, Lorenzo; Garcia, Josue; Cozine, Steven; Suen, Darrick; Nguyen, Christina; Alimo, Ryan - 04b Atto di convegno in volume
congresso: 17th European Conference on Computer Vision, ECCV 2022 (Tel Aviv, Israel)
libro: Computer Vision – ECCV 2022 Workshops - (9783031250552)

11573/1687902 - 2022 - Graph Convolutional Networks with Autoencoder-Based Compression and Multi-Layer Graph Learning
Giusti, L; Battiloro, C; Di Lorenzo, P; Barbarossa, S - 04b Atto di convegno in volume
congresso: IEEE ICASSP 2022 (Singapore)
libro: IEEE ICASSP 2022 - (978-1-6654-0540-9)

11573/1519056 - 2021 - Accuracy of self-assessment of real-life functioning in schizophrenia
Rocca, P.; Brasso, C.; Montemagni, C.; Bellino, S.; Rossi, A.; Bertolino, A.; Gibertoni, D.; Aguglia, E.; Amore, M.; Andriola, I.; Bellomo, A.; Bucci, P.; Buzzanca, A.; Carpiniello, B.; Cuomo, A.; Dell'osso, L.; Favaro, A.; Giordano, G. M.; Marchesi, C.; Monteleone, P.; Oldani, L.; Pompili, M.; Roncone, R.; Rossi, R.; Siracusano, A.; Vita, A.; Zeppegno, P.; Galderisi, S.; Maj, M.; Bozzatello, P.; Badino, C.; Giordano, B.; Di Palo, P.; Calia, V.; Papalino, M.; Barlati, S.; Deste, G.; Ceraso, A.; Pinna, F.; Olivieri, B.; Manca, D.; Piegari, G.; Brando, F.; Giuliani, L.; Aiello, C.; Poli, L. F.; Concerto, C.; Surace, T.; Altamura, M.; Malerba, S.; Padalino, F.; Calcagno, P.; Murri, M. B.; Amerio, A.; Pacitti, F.; Socci, V.; Lucaselli, A.; Giusti, L.; Salza, A.; Ussorio, D.; Iasevoli, F.; Gramaglia, C.; Gambaro, E.; Gattoni, E.; Tenconi, E.; Collantoni, E.; Meneguzzo, P.; Ossola, P.; Tonna, M.; Gerra, M. L.; Carmassi, C.; Carpita, B.; Cremone, I. M.; Corrivetti, G.; Cascino, G.; Marciello, F.; Brugnoli, R.; Comparelli, A.; Corigliano, V.; Girardi, N.; Accinni, T.; Carlone, L.; Fagiolini, A.; Goracci, A.; Bolognesi, S.; Di Lorenzo, G.; Niolu, C.; Ribolsi, M. - 01a Articolo in rivista
rivista: NPJ SCHIZOPHRENIA (London [u.a.] : Nature Publ. Group) pp. 11- - issn: 2334-265X - wos: WOS:000618349400001 (17) - scopus: 2-s2.0-85101482645 (18)

© Università degli Studi di Roma "La Sapienza" - Piazzale Aldo Moro 5, 00185 Roma