NATIONAL PHD IN ARTIFICIAL INTELLIGENCE

Thesis title: Language Models for Information Quality: Methods and Applications

This thesis addresses the challenges and advancements in applying language models to enhance information quality across data management tasks. As organizations increasingly rely on diverse data sources, maintaining high information quality has become crucial for decision-making and operational processes. This work investigates four key areas contributing to information quality: handling of duplicate data, uncertainty quantification, data-on-demand, and ethical considerations in data management. The core contributions are as follows. First, the thesis presents methodologies for detecting, managing, and removing duplicate data across different contexts. It introduces an efficient pipeline for entity count estimation to address duplicate records in large datasets, grounded approaches for integrating entity resolution with knowledge graphs, and a method that uses transformer-based language models to manage conceptual duplicates in multiple-choice question repositories. Secondly, it explores methods for quantifying and addressing uncertainty in outputs generated by large language models in question-answering applications, improving the reliability of responses for decision-critical tasks. Thirdly, the research presents a system enabling on-demand structured data generation and integration from multiple industrial sources using language models. Lastly, the thesis examines fairness in ranking systems that rely on subjective data, proposing a framework to assess group bias in collaborative rating platforms. In summary, this thesis explores the use of language models for managing information quality by addressing critical challenges in duplicate data handling, uncertainty quantification, real-time data access, and ethical considerations, fostering trust in data-driven systems across a variety of applications.