NATIONAL PHD IN ARTIFICIAL INTELLIGENCE

Thesis title: Enabling Robust and Reliable Commonsense Reasoning in Large Language Models

In recent years, Large Language Models (LLMs) have revolutionized the field of Natural Language Processing (NLP), demonstrating remarkable performance across a wide range of tasks. Despite these advances, fundamental questions remain about their reasoning capabilities, particularly in commonsense reasoning, which refers to the ability to draw inferences from implicit, everyday knowledge that humans typically take for granted. Current approaches often fail to provide models with knowledge in forms they can effectively utilize, while evaluation methodologies focus exclusively on final answers, ignoring the reasoning processes that lead to them. Moreover, the dominant multiple-choice evaluation paradigm can systematically misrepresent model capabilities, especially as models generate increasingly complex reasoning traces. These limitations hinder progress toward language models with robust, human-like commonsense reasoning and understanding. This thesis aims to advance both the performance and reliable evaluation of commonsense reasoning in language models, addressing fundamental challenges in knowledge augmentation and assessment methodologies. First, we introduce a retrieval augmentation framework that provides models with complete reasoning examples rather than isolated facts, demonstrating consistent improvements across multiple benchmarks without requiring model retraining. Second, we expose systematic inconsistencies in multiple-choice evaluation through comprehensive human annotation studies, revealing substantial disagreement between automated extraction strategies and human judgment, and introducing a dataset that exposes critical vulnerabilities in state-of-the-art LLM-based answer extractors. Third, we present the first comprehensive benchmark for evaluating reasoning traces in commonsense domains, demonstrating that a significant proportion of correct answers contain flawed reasoning and that reasoning-aware evaluation reveals substantial performance drops compared to answer-only assessment. Together, these findings show that progress in commonsense reasoning depends on improving how language models incorporate contextual knowledge and how their reasoning is evaluated. By enhancing reasoning through example-based retrieval and exposing limitations of answer-only evaluation, this work provides the methods needed to develop language models with more reliable commonsense reasoning capabilities.