Thesis title: Analyzing Large Language Models: Bridging Linguistics and Applied Mathematics
This thesis delves into the world of language processing using advanced deep learning models to uncover complex linguistic patterns. The exploration begins with an analysis of vector semantics, focusing on the strengths and limitations of static and contextualized embeddings in capturing semantic relationships, particularly in the Korean language. A thorough examination of the attention mechanism within Transformer architectures is conducted, highlighting its role in linguistic representation and the challenges in deriving a complete understanding of linguistic phenomena through it.
The research extends into probing techniques aiming to uncover the types of linguistic features captured by language models and their ability to generalize across different linguistic tasks. A significant part of the research investigates the relationship between token likelihood and attention values in language models, unveiling a dynamic interaction that provides insights into how these models process language.
Further, spectral analysis and signal processing are introduced as new methods to examine the inner workings of language models. These techniques provide a new perspective to understand how language models work, extract linguistic features, and generalize across different language tasks.