Phd in MATHEMATICS

Thesis title: Applications of Spin Glass Theory in Neural Networks and Evolutionary Models

This thesis presents the findings of a three-year investigation into the application of spin glass theory to the development of a rigorous mathematical framework for two distinct and seemingly unrelated fields: information processing in Artificial Neural Networks (ANNs) and the Theory of Evolutionary Models. Spin models are the starting point of all our investigations. In brief, they are mathematical models primarily used in statistical mechanics and condensed matter physics to describe collective magnetic phenomena. In 1925 G. E. Uhlenbeck and S. Goudsmit hypothesized that the electron has a ``spin'', and so behaves like a small bar magnet, and that in an external magnetic field the direction of the electron's magnetic field is either parallel or antiparallel to that of the external field. In classical spin models, the state of each electron is described by a scalar variable (or set of scalar variables) that can take values from the set {+1,-1}. The entire system is described by an Hamiltonian, which assigns an energy level to each spin configuration, and which is the sum of the interaction energies between them. The sign of these interactions determines the collective behaviour of the system. In ferromagnets, the interactions are all positive and tend to align the spins along a common direction (these are referred to as simple systems). Spin glasses are systems of interacting spins where the interactions are randomly distributed in sign. Due to the randomness in the sign of interactions, the spins experience conflicting constraints, a situation known as frustration. This gives rise to a proliferation of local minima in the Hamiltonian, leading to an intricate and complicated free energy landscape; hence, we refer to such systems as complex systems. As we will discuss, the applicability of spin glass theory to neural networks and the theory of natural evolution is rooted in this multi-state structure. Neural networks also involve cooperation between many simple units (neurons) through interactions that can conflict. These conflicts arise because each neuron receives inputs from other neurons via synapses, that can be excitatory (increasing the activity of the receiving neuron) or inhibitory (decreasing the activity). The presence of frustration within the system results in many different behaviors of the whole network, depending on the final state reached through neural dynamics. Furthermore, as we will demonstrate, the existence of a mathematical mapping between statistical mechanics of the spin systems and the dynamics of neural networks allows us to predict possible outcomes of the training process. Similarly, natural biological evolution can be understood as a dynamic process in which a group of individuals cooperates to ensure the most successful reproduction of future species. As we will discuss in detail, in the most widely recognized models of natural evolution, the reproductive process involves random mutations in the genome, which is represented as a string of binary bits. Under specific assumptions regarding the reproduction process, the distances between the genomes of individuals -once equilibrium is reached- form an ultrametric tree, similar to the equilibrium states in spin glasses. Based on this conceptual analogy, certain relationships that hold for spin glasses can also be tested in evolutionary models, enabling a comparison between the two theories.