Titolo della tesi: Deep Learning applications over Heterogeneous Networks: from Multimedia to Genes
Networks are ubiquitous in nature and technology. Many research fields in industry and academia model the environments they study as networks, in that composite realities like the Internet, virtual realities, medical services, and particle physics phenomena, among others, can be seen as a group of simpler entities that interact between them. Moreover, networks are often heterogeneous because the interacting entities can be further differentiated into classes or groups, and the interactions themselves can be classified into different types. Some of the most prominent examples of heterogeneous graph models are those used in bioinformatics and bio-medicine. Many studies in these fields investigate multiple entities like molecules, proteins, and DNA segments that interact in numerous ways to drive biological processes.
Apart from bioinformatics, social networks and computer networks are also heterogeneous networks in which lots of research efforts have concentrated in the last years. The proliferation of data collection techniques and the democratization of social media production and consuming services have augmented the volume and heterogeneity of publicly available data on the Internet. One proof of this phenomenon is the non-structured database paradigm that has evolved in the last years. Non-structured databases are a building block of the recent big-data pipelines built to orchestrate multiple heterogeneous data sources and extract added value from them.
However, not only the modes of collecting and storing heterogeneous information networks have witnessed a continuous evolution. Graph analysis and network science are two disciplines that have evolved in the last years to cope with the need to extract valuable knowledge from heterogeneous relational data. Using heuristic-based and meta-heuristic-based solutions for solving computationally expensive problems is one of the most common practices adopted by network scientists that deal with big-data formatted as heterogeneous graphs (het-graph).
The rocket-fast speed at which the processor has evolved has also democratized computing resources in the last decades. Consequentially, the neural network paradigm, which was mainly forgotten by researchers in the last decade of the last century, has risen with unprecedented popularity: multiple gigabytes of training data are downloaded for free in minutes, and home computer processors are able to optimize millions of parameters through gradient descent algorithms to produce constant-time task-specific neural modules that show unprecedented accuracy in many particular inductive tasks.
This democratization of computing resources and extensive labeled data implied the democratization of deep learning. Consequentially, a question was born in the mind of researchers: how to foster synergies between deep learning and graph analysis tools? But the answer was not trivial: the main problem associated with the data collected from (heterogeneous) network environments was the non-euclidean format of the relationships between nodes of a graph. Fortunately, research efforts based on signal theory, spectral analysis, and spatial message passing frameworks solved the challenge of accommodating graphs to neural networks.
Traditional graph analysis instruments gained enormous scalability as a consequence. Moreover, this recent deep-learning based gain had immediate positive implications in the research agenda of the last years. Modern navigation, social recommender, and weather forecasting systems are just some ubiquitous proofs of this. The first contextual factor in which our research places itself is this growth in the applicability of deep learning-based heterogeneous graph analysis to academical and industrial research fields.
The synergy between deep-learning and heterogeneous graphs is, however, a two-way relationship: deep learning brings scalability and generalization power to graph analysis algorithms, but also het-graph models help deep learning practitioners design more efficient learning pipelines. More specifically, solution spaces are often prohibitively large when optimization tasks are designed in heterogeneous graph modeled scenarios. Developing algorithms that efficiently explore candidate solutions to find the global optima becomes non-trivial in these environments, even for deep-learning-based algorithms. In these cases, the injection of graph-model-specific inductive biases during the architectural and procedural design of deep-learning pipelines has proven crucial to tackle the curse of dimensionality. Thus, in the present research, we were not only interested in how deep learning has empowered modern graph analysis, but we also pretended to place our sight in the opposite direction of this synergy: how het-graph models facilitate the design of inductive and procedural biases in deep learning architectures.
This dissertation studies the state-of-the-art techniques that combine deep learning with heterogeneous graph modeled scenarios. Two main paradigms of collaboration have been identified. The first one consists of enhancing the scalability and generalization power of graph algorithms through deep learning. The second is the augmented efficiency of solution-space exploration that heterogeneous graph modeled scenarios induce in the design of deep learning pipelines. Moreover, this research identified two open research opportunities where the studied synergisms could be helpful to solve. The first one is the online optimization of service function chain deployment in virtualized content delivery networks for live-streaming. The second is the inference of developmental regulatory mechanisms between genes and cis-regulatory elements. The candidate demonstrated his proficiency in the research field by applying the synergisms identified in the first phase of the research to solve such open problems.