NINA DELIU

Dottoressa di ricerca

ciclo: XXXIII



Titolo della tesi: Reinforcement Learning in Modern Biostatistics: Benefits, Challenges and New Proposals

Applications of reinforcement learning (RL) for supporting, managing and improving decision-making are becoming increasingly popular in a variety of medicine and healthcare domains where the problem has a sequential nature. By continuously interacting with the underlying environment, RL techniques are able to learn by trial-and-error on how to take better actions in order to maximize an outcome of interest over time. However, if on one hand RL offers a new powerful framework, on the other hand it poses some unique challenges for data analysis and interpretability, which call for new statistical techniques in both predictive and descriptive learning. Notably, several methodological challenges, for which the contribution of the biostatistical community may play a crucial role, limit the use of RL in real life. In an aim to bridge the statistics and RL communities, we start by assimilating the different existing RL terminologies, notations and approaches into a coherent body of work, and by translating them from a machine learning (ML) to a statistical perspective. Then, through a comprehensive methodological review, we report and discuss the state-of-the-art RL-based research in healthcare. Two main applied domains emerged: 1) adaptive interventions (AIs), encompassing both dynamic treatment regimes and just-in-time adaptive interventions in mobile health (mHealth); and 2) adaptive designs of clinical trials, specifically dose-finding designs and adaptive randomization. We illustrate existing RL-based methods in these areas, discussing their benefits and existing open problems that may impact their application in real life. A major barrier to adopting RL in real-world experiments is the lack of clarity on how statistical analyses and inference are impacted. In clinical trials for example, if on one side, to achieve the practical (and more ethical) goal of improving patients’ benefits, RL may have better abilities in terms of maximising clinical outcomes by adaptively randomizing participants to the best evidence-based treatment; on the other side, to achieve the scientific goal of e.g., discovering whether one treatment is more effective compared to a control treatment, less is known about their inferential properties. Through a simulation study, we investigate the challenges of conducting hypothesis testing from data collected through a class of RL, i.e., multi-armed bandits (MABs), outlining the harms MAB algorithms can cause to traditional statistical tests’ type-I error and power. This empirical evaluation provides guidance to two alternative ways of pursuing improved statistical hypothesis testing: 1) to explore ways of modifying the test statistic using knowledge of the adaptive data collection nature; 2) to modify the algorithm or framework for a more sensitive problem to both statistical inference as well as reward maximization. Focusing on the Thompson Sampling (a randomized MAB strategy), we show how a modified version of it results in an optimal intermediate between these two objectives. These findings can provide insights into how challenges can be surmounted by bridging machine learning, statistics, and applied sciences, to conduct adaptive experiments in the real-world, aiming to simultaneously help individuals and advance scientific research. We finally combine our methodological knowledge with a motivating mHealth study for improving physical activity, to illustrate the tremendous collaboration opportunities between statistics and RL researchers in the space of developing adaptive interventions into the increasingly growing area of mHealth.

Produzione scientifica

11573/1701842 - 2024 - Incorporating participants’ welfare into sequential multiple assignment randomized trials
Wang, Xinru; Deliu, Nina; Narita, Yusuke; Chakraborty, Bibhas - 01a Articolo in rivista
rivista: BIOMETRICS ([Washington]: [International Biometric Society etc.]) pp. - - issn: 0006-341X - wos: (0) - scopus: 2-s2.0-85185395189 (0)

11573/1693295 - 2023 - Informing Users about Data Imputation: Exploring the Design Space for Dealing With Non-Responses
Bhattacharjee, Ananya; Song, Haochen; Wu, Xuening; Tomlinson, Justice; Reza, Mohi; Chowdhury, Akmar Ehsan; Deliu, Nina; Price, Thomas W.; Williams, Joseph Jay - 04b Atto di convegno in volume
congresso: The 11th AAAI Conference on Human Computation and Crowdsourcing (HCOMP 2023) (Delft, Netherlands)
libro: Proceedings of the AAAI Conference on Human Computation and Crowdsourcing - (978-1-57735-884-8)

11573/1679437 - 2023 - Adaptive Experiments for Enhancing Digital Education: Benefits and Statistical Challenges
Deliu, Nina - 04d Abstract in atti di convegno
congresso: INTERNATIONAL CONFERENCE ON NEW ACHIEVEMENTS IN SCIENCE, TECHNOLOGY AND ARTS – ICNA-STA (Prishtina; Kosova)
libro: INTERNATIONAL CONFERENCE ON NEW ACHIEVEMENTS IN SCIENCE, TECHNOLOGY AND ARTS – ICNA-STA - (978-9951-9090-1-3)

11573/1680238 - 2023 - Multiculturality and Interculturality: A Qualitative Analysis of the Perspective of Focus Group Participants
Deliu, Nina - 02a Capitolo o Articolo
libro: Education in Multiculturality Education to Interculturality in Ecclesiastical Institutions of Higher Education and in Formation Communities for Catholic Consecrated Life in Italy - (978-88-401-9061-7)

11573/1693297 - 2023 - Reinforcement learning for sequential decision making in population research
Deliu, Nina - 01a Articolo in rivista
rivista: QUALITY AND QUANTITY (Dordrecht : Kluwer) pp. - - issn: 1573-7845 - wos: (0) - scopus: 2-s2.0-85175578608 (0)

11573/1693336 - 2023 - Multinomial Thompson sampling for rating scales and prior considerations for calibrating uncertainty
Deliu, Nina - 01a Articolo in rivista
rivista: STATISTICAL METHODS & APPLICATIONS (Physica-Verlag, berlin) pp. - - issn: 1618-2510 - wos: WOS:001114600400001 (0) - scopus: (0)

11573/1687942 - 2023 - Computing Highest Density Regions with Copulae
Deliu, Nina; Liseo, Brunero - 04b Atto di convegno in volume
congresso: SEAS IN SIS 2023 (Ancona)
libro: SEAS IN Book of Short Papers 2023 - (9788891935618)

11573/1664710 - 2023 - Microrandomized Trials: Developing Just-in-Time Adaptive Interventions for Better Public Health
Liu, Xueqing; Deliu, Nina; Chakraborty, Bibhas - 01a Articolo in rivista
rivista: AMERICAN JOURNAL OF PUBLIC HEALTH (New York, N.Y. : American Public Health Association) pp. 60-69 - issn: 0090-0036 - wos: WOS:000903755500013 (7) - scopus: 2-s2.0-85144585357 (5)

11573/1653956 - 2022 - Multi-disciplinary fairness considerations in machine learning for clinical trials
Chien, Isabel; Deliu, Nina; Turner, Richard; Weller, Adrian; Villar, Sofia; Kilbertus, Niki - 04b Atto di convegno in volume
congresso: FAccT '22: 2022 ACM Conference on Fairness, Accountability, and Transparency (Seoul, South Korea)
libro: FAccT '22: 2022 ACM Conference on Fairness, Accountability, and Transparency - (9781450393522)

11573/1656259 - 2022 - Multinomial Thompson Sampling for adaptive experiments with rating scales
Deliu, Nina - 04b Atto di convegno in volume
congresso: SIS 2022, 51st Scientific Meeting of the Italian Statistical Society (Caserta)
libro: Book of Short Papers Book of the 51st Scientific Meeting of the Italian Statistical Society - (9788891932310)

11573/1656256 - 2022 - Dynamic Treatment Regimes for Optimizing Healthcare
Deliu, Nina; Chakraborty, Bibhas - 02a Capitolo o Articolo
libro: The Elements of Joint Learning and Optimization in Operations Management - (978-3-031-01925-8; 978-3-031-01926-5)

11573/1581570 - 2022 - Daily Motivational Text Messages to Promote Physical Activity in University Students: Results From a Microrandomized Trial
Figueroa, Caroline A; Deliu, Nina; Chakraborty, Bibhas; Modiri, Arghavan; Xu, Jing; Aggarwal, Jai; Jay Williams, Joseph; Lyles, Courtney; Aguilera, Adrian - 01a Articolo in rivista
rivista: ANNALS OF BEHAVIORAL MEDICINE (Society of Behavioral Medicine:7600 Terrace Avenue, Suite 203:Middleton, WI 53562:(608)827-7267, EMAIL: sbm@tmahq.com, INTERNET: http://www.sbmweb.org) pp. 212-218 - issn: 0883-6612 - wos: WOS:000754035000009 (16) - scopus: 2-s2.0-85111115424 (18)

11573/1581568 - 2021 - Adaptive learning algorithms to optimize mobile applications for behavioral health: guidelines for design decisions
Figueroa, Ca; Aguilera, A; Chakraborty, B; Modiri, A; Aggarwal, J; Deliu, N; Sarkar, U; Jay Williams, J; Lyles, Cr - 01a Articolo in rivista
rivista: JOURNAL OF THE AMERICAN MEDICAL INFORMATICS ASSOCIATION (Hanley & Belfus Incorporated:210 South 13th Street:Philadelphia, PA 19107:(800)962-1892, (215)546-4995, INTERNET: http://www.hanleyandbelfus.com, Fax: (215)790-9330) pp. 1225-1234 - issn: 1067-5027 - wos: WOS:000671031900019 (4) - scopus: 2-s2.0-85108303548 (10)

11573/1288165 - 2019 - The prognostic value of patient-reported outcomes (PROs) for survival outcomes in cancer patients: A systematic review
Cottone, Francesco; Collins, Gary S.; Giesinger, Johannes M.; Sommer, Kathrin; Deliu, Nina; Vignetti, Marco; Anota, Amelie; Efficace, Fabio - 01h Abstract in rivista
rivista: JOURNAL OF CLINICAL ONCOLOGY (editori attuale: -American Society of Clinical Oncology. , 330 JOHN CARLYLE ST, STE 300, ALEXANDRIA, USA, VA, 22314 precedente: -W B Saunders Company:Fulfillment Department, The Curtis Center, Independence Square West:Philadelphia, PA 19106:(800)654-2452, (215)238-7800, EMAIL: wbspcs@harcourt.com, INTERNET: http://elsevierhealth.com, Fax: (215)238-6445) pp. e18223-e18223 - issn: 0732-183X - wos: WOS:000487345802180 (0) - scopus: (0)

11573/1221448 - 2019 - Modeling strategies to improve parameter estimates in prognostic factors analyses with patient-reported outcomes in oncology
Cottone, Francesco; Deliu, Nina; Collins, Gary S.; Anota, Amelie; Bonnetain, Franck; Van Steen, Kristel; Cella, David; Efficace, Fabio - 01a Articolo in rivista
rivista: QUALITY OF LIFE RESEARCH (Kluwer Academic Publishers:Journals Department, PO Box 322, 3300 AH Dordrecht Netherlands:011 31 78 6576050, EMAIL: frontoffice@wkap.nl, kluweronline@wkap.nl, INTERNET: http://www.kluwerlaw.com, Fax: 011 31 78 6576254) pp. 1315-1325 - issn: 0962-9343 - wos: WOS:000466734100019 (3) - scopus: 2-s2.0-85060202000 (4)

11573/1653958 - 2019 - Clinician-reported symptomatic adverse events in cancer trials: are they concordant with patient-reported outcomes?
Sparano, Francesco; Aaronson, Neil K; Cottone, Francesco; Piciocchi, Alfonso; La Sala, Edoardo; Anota, Amelie; Deliu, Nina; Kieffer, Jacobien M; Efficace, Fabio - 01a Articolo in rivista
rivista: JOURNAL OF COMPARATIVE EFFECTIVENESS RESEARCH (London : Future Medicine, 2012-) pp. 279-288 - issn: 2042-6305 - wos: WOS:000465466700002 (3) - scopus: 2-s2.0-85064212142 (4)

11573/1186915 - 2018 - Deep learning to the test. An application to traffic data streams
Deliu, Nina; Brutti, Pierpaolo - 04b Atto di convegno in volume
congresso: 49th Scientific Meeting of the Italian Statistical Society (Palermo; Italia)
libro: Book of Short Papers SIS 2018 - (9788891910233)

11573/1186926 - 2018 - Evaluating methodological quality of Prognostic models Including Patient-reported HeAlth outcomes in oncologY (EPIPHANY): A systematic review protocol
Deliu, Nina; Cottone, Francesco; Collins, Gary S.; Anota, Amélie; Efficace, Fabio - 01a Articolo in rivista
rivista: BMJ OPEN ([London] : BMJ Publishing Group Ltd, 2011-) pp. e025054- - issn: 2044-6055 - wos: WOS:000454739500190 (5) - scopus: 2-s2.0-85055614072 (5)

11573/1213229 - 2018 - Distribution- and anchor-based methods to determine the minimally important difference on patient-reported outcome questionnaires in oncology: a structured review
Ousmen, Ahmad; Touraine, Célia; Deliu, Nina; Cottone, Francesco; Bonnetain, Franck; Efficace, Fabio; Brédart, Anne; Mollevi, Caroline; Anota, Amélie - 01a Articolo in rivista
rivista: HEALTH AND QUALITY OF LIFE OUTCOMES (LONDON, BioMed Central) pp. - - issn: 1477-7525 - wos: WOS:000452861600001 (62) - scopus: 2-s2.0-85058279478 (65)

11573/1213232 - 2018 - PCN368 - CONCORDANCE BETWEEN PATIENTS AND CLINICIANS’ REPORTING OF SYMPTOMATIC ADVERSE EVENTS IN CANCER CLINICAL TRIALS: A SYSTEMATIC REVIEW
Sparano, F.; Aaronson, N.; Cottone, F.; Piciocchi, A.; La Sala, E.; Anota, A.; Deliu, N.; Kieffer, J. M.; Efficace, F. - 04d Abstract in atti di convegno
rivista: VALUE IN HEALTH (New York: Elsevier Malden MA: Blackwell Science, ©1998-) pp. S76-S77 - issn: 1098-3015 - wos: WOS:000459985600391 (0) - scopus: (0)
congresso: ISPOR Europe 2018 (Barcelona)
libro: Value in Health - ()

11573/1186931 - 2017 - Méthodes de détermination de la différence minimale cliniquement importante pour les questionnaires de qualité de vie relative à la santé en cancérologie
Anota, A.; Touraine, C.; Ousmen, A.; Deliu, N.; Efficace, F.; Bonnetain, F.; Brédart, A.; Bascoul-Mollevi, C. - 01h Abstract in rivista
rivista: REVUE D'EPIDEMIOLOGIE ET DE SANTE PUBLIQUE (Masson Editeur:21 rue Camille Desmoulins, 92786 Issy Cedex 9 France:011 33 1 73281634, EMAIL: infos@masson.fr, INTERNET: http://www.masson.fr, Fax: 011 33 1 73281649) pp. S66- - issn: 0398-7620 - wos: (0) - scopus: (0)

11573/1213238 - 2017 - Modelling strategies to improve estimates of prognostic factors analyses with patient reported outcomes: a simulation study
Deliu, Nina; Efficace, Fabio; Collins, Gary; Anota, Amelie; Bonnetain, Franck; Van Steen, Kristel; Cella, David; Cottone, Francesco - 04c Atto di convegno in rivista
rivista: QUALITY OF LIFE RESEARCH (-Dordrecht: Springer -Dordrecht: Kluwer) pp. 34-35 - issn: 1573-2649 - wos: WOS:000432071000086 (1) - scopus: 2-s2.0-85057547800 (0)
congresso: 24th Annual Conference of the International Society for Quality of Life Research (Philadelphia)

11573/1288226 - 2017 - Concordance between patient reported and clinical outcomes in randomized controlled trials (RCTs) of cancer treatment
Efficace, Fabio; Aaronson, Neil K.; Sparano, Francesco; Sprangers, Mirjam; Fayers, Peter; Pusic, Andrea L.; Anota, Amelie; Cottone, Francesco; Rees, Jonathan; Deliu, Nina; Piciocchi, Alfonso; La Sala, Edoardo; Haas, Alissa; Kieffer, Jacobien M.; Wang, Wenna; Pezold, Mike; Fuzesi, Sarah; Isharwal, Sumit; Yeung, John; Wan, Chonghua; Blazeby, Jane - 04c Atto di convegno in rivista
rivista: QUALITY OF LIFE RESEARCH (-Dordrecht: Springer -Dordrecht: Kluwer) pp. 103-103 - issn: 1573-2649 - wos: WOS:000432071000260 (0) - scopus: 2-s2.0-85057547800 (0)
congresso: 24th Annual Conference of the International Society for Quality of Life Research (ISOQOL) (Philadelphia (PA), USA)

© Università degli Studi di Roma "La Sapienza" - Piazzale Aldo Moro 5, 00185 Roma