VALERIA RUSCIO

PhD Student

PhD program:: XXXVI
email: valeria.ruscio@uniroma1.it
room: A101 (DIAG)




supervisor: Antonetta L. Bruno (Sapienza University of Rome, ISO)
co-supervisor: Fabrizio Silvestri (Sapienza University of Rome, DIAG)

Research: Korean linguistics features through Transformer-based Language Model Explainability

Research interests: Korean linguistics, Computational Linguistics, Natural Language Processing, Deep Learning.

My research is centered around transformers and their self-attention mechanism, with the goal of understanding their inner workings and the importance of self-attention in achieving high performance. Additionally, I have a strong interest in Korean linguistics, which has helped me gain a deeper understanding of the relationship between transformer architecture and textual features. I'm also investigating how large language models represent out-of-context words and the extent to which they rely on context to capture their semantics. Through our experiments, we found that unexpected tokens can cause the model to attend less to the information coming from themselves to compute their representations, particularly at higher layers, which has important implications for assessing the robustness of LLMs in real-world scenarios.


11/2020— present: PhD candidate at Istituto Italiano di Studi Orientali (ISO), Sapienza University of Rome
Research project's title: "Korean linguistics features through Transformer-based Language Model Explainability"
Advisor: Professor Antonetta L. Bruno (Sapienza University of Rome, ISO)
Co-advisor: Professor Fabrizio Silvestri (Sapienza University of Rome, DIAG)

2016 - 2018, Master’s degree in East Asian Languages and civilizations
at Sapienza University of Rome (Italy) and Hanyang University of Soul (South Korea)
Korean language curriculum
Final grade: 110 cum Laude
Dissertation's title: "Emotional expressions in contemporary Korea: The case of Webtoon"

2015 - 2016, Exchange program focusing on Korean language
at Hankuk University of Foreign Studies (Seoul, South Korea)

2013 - 2016, bachelor’s degree in East Asian Languages and civilizations
at Sapienza University of Rome (Italy)
Korean language curriculum
Final grade: 102
Dissertation's title: "Tourism phenomenon in South Korea: economic relevance and government's policies"

2019-2022 I have worked as linguist at Babelscape srl


Papers:

Valeria Ruscio, Valentino Maiorca, Fabrizio Silvestri; "Attention-likelihood relationship in transformers", 2023

Research products

11573/1669797 - 2023 - Synonymy in Korean Lexicon through the lens of vector semantics
Ruscio, Valeria - 02a Capitolo o Articolo
book: Percorsi in Civiltà dell’Asia e dell’Africa II: Quaderni di studi dottorali alla Sapienza - (978-88-9377-260-0)

11573/1696195 - 2023 - Attention-likelihood relationship in transformers
Ruscio, Valeria; Maiorca, Valentino; Silvestri, Fabrizio - 04h Atto di convegno in rivista scientifica o di classe A
paper: International Conference on Learning Representations (International Conference on Learning Representations) pp. - - issn: - wos: (0) - scopus: (0)
conference: The Eleventh International Conference on Learning Representations (Kigali)

© Università degli Studi di Roma "La Sapienza" - Piazzale Aldo Moro 5, 00185 Roma