LORENZO FEDERICI

PhD Graduate

PhD program:: XXXIV


supervisor: Prof. Alessandro Zavoli
advisor: Prof. Guido Colasurdo

Thesis title: Deep Reinforcement Learning for Robust Spacecraft Guidance and Control

This Ph.D. thesis aims to investigate new guidance and control algorithms based on deep neural networks and reinforcement learning, with application in next-generation space missions, which are expected to require greater levels of autonomy and robustness. Unlike traditional optimal control methods, deep reinforcement learning represents a systematic framework to deal with space trajectory optimization problems under any kind of uncertainty, including unknown or unmodeled dynamics, inaccurate initial conditions, control execution errors, and measurement noise. In deep learning approaches to spacecraft guidance, a deep neural network is used to map observations, that is, any combination of measures of the spacecraft state, including raw images taken by on-board optical cameras, to corresponding control actions, which in space applications generally define the modulus and direction of the thrust. In deep reinforcement learning, the optimal control problem is reformulated as a discrete-time Markov decision process, and the network, or agent, is trained by trial-and-error through repeated simulations of the mission scenario. The agent starts with a random control policy and progressively refines it during training, seeking to maximize the rewards received from the environment as a measure of its current performance. The exploratory behavior typical of reinforcement learning algorithms, which learn through a huge number of simulations, is at the base of their inherent robustness against variations in the environment definition. At the end of the training process, besides a reference robust trajectory, the network outputs an optimal observation-feedback control law. For this reason, the trained network can be used on-board the spacecraft to provide it with real-time and autonomous control capabilities during the actual operations. In this thesis, deep neural networks with different architectures are trained by a state-of-the-art reinforcement learning method and applied to a few selected real-world study cases, including interplanetary, multi-body, and proximity-operation space missions. The objective is to assess how the networks perform in terms of optimality, constraint handling, and robustness in different operational scenarios, which feature scattered initial conditions, multiple terminal and path constraints, unmodeled dynamics, control and navigation errors, and partial observability of the spacecraft state.

Research products

11573/1714525 - 2024 - Meta-Reinforcement Learning with Transformer Networks for Space Guidance Applications
Federici, Lorenzo; Furfaro, Roberto - 04b Atto di convegno in volume
conference: AIAA SciTech Forum and Exposition, 2024 (Orlando; Florida (USA))
book: AIAA SciTech Forum and Exposition, 2024 - (9781624107115)

11573/1713983 - 2024 - Robust interplanetary trajectory design under multiple uncertainties via meta-reinforcement learning
Federici, Lorenzo; Zavoli, Alessandro - 01a Articolo in rivista
paper: ACTA ASTRONAUTICA (Elmsford N.Y.: Pergamon Press) pp. 147-158 - issn: 0094-5765 - wos: WOS:001101996400001 (2) - scopus: 2-s2.0-85174901065 (3)

11573/1714832 - 2023 - Robust Waypoint Guidance of a Hexacopter on Mars using Meta-Reinforcement Learning
Federici, Lorenzo; Furfaro, Roberto; Zavoli, Alessandro; De Matteis, Guido - 04b Atto di convegno in volume
conference: AIAA SciTech Forum and Exposition, 2023 (National Harbor, MD (USA))
book: AIAA SciTech Forum and Exposition, 2023 - ()

11573/1640438 - 2023 - Autonomous guidance for cislunar orbit transfers via reinforcement learning
Federici, Lorenzo; Scorsoglio, Andrea; Zavoli, Alessandro; Furfaro, Roberto - 04b Atto di convegno in volume
conference: 2021 AAS/AIAA Astrodynamics Specialist Conference (virtual)
book: JOURNAL OFSPACECRAFT ANDROCKETS - ()

11573/1713986 - 2023 - Autonomous guidance between quasiperiodic orbits in cislunar space via deep reinforcement learning
Federici, Lorenzo; Scorsoglio, Andrea; Zavoli, Alessandro; Furfaro, Roberto - 01a Articolo in rivista
paper: JOURNAL OF SPACECRAFT AND ROCKETS (American Institute of Aeronautics & Astronautics:1801 Alexander Bell Drive, Suite 500:Reston, VA 20191:(800)639-2422, (703)264-7500, EMAIL: custserv@aiaa.org, INTERNET: http://www.aiaa.org/, Fax: (703)264-7657) pp. 1954-1965 - issn: 0022-4650 - wos: WOS:001080091500001 (2) - scopus: 2-s2.0-85178657244 (2)

11573/1714527 - 2023 - Comparative Analysis of Reinforcement Learning Algorithms for Robust Interplanetary Trajectory Design
Federici, Lorenzo; Zavoli, Alessandro; Furfaro, Roberto - 04b Atto di convegno in volume
conference: 2nd International Conference on Applied Intelligence and Informatics , AII 2022 (Reggio Calabria, Italia)
book: Studies in Computational Intelligence - (9783031257544; 9783031257551)

11573/1713989 - 2023 - Evolutionary optimization for active debris removal mission planning
Zona, Danilo; Zavoli, Alessandro; Federici, Lorenzo; Avanzini, Giulio - 01a Articolo in rivista
paper: IEEE ACCESS (Piscataway NJ: Institute of Electrical and Electronics Engineers) pp. 41019-41033 - issn: 2169-3536 - wos: WOS:000981873000001 (2) - scopus: 2-s2.0-85153802350 (2)

11573/1615402 - 2022 - Image-based meta-reinforcement learning for autonomous terminal guidance of an impactor in a binary asteroid system
Federici, L.; Scorsoglio, A.; Ghilardi, L.; D'ambrosio, A.; Benedikter, B.; Zavoli, A.; Furfaro, R. - 04b Atto di convegno in volume
conference: AIAA Science and Technology Forum and Exposition, AIAA SciTech Forum 2022 (San Diego; CA (USA))
book: AIAA Science and Technology Forum and Exposition, AIAA SciTech Forum 2022 - (978-1-62410-631-6)

11573/1714827 - 2022 - Robust design of interplanetary trajectories under severe uncertainty via meta-reinforcement learning
Federici, L.; Zavoli, A. - 04b Atto di convegno in volume
conference: 73rd International Astronautical Congress, IAC 2022 (Paris; France)
book: Proceedings of the International Astronautical Congress, IAC - ()

11573/1713992 - 2022 - Image-based meta-reinforcement learning for autonomous guidance of an asteroid impactor
Federici, Lorenzo; Scorsoglio, Andrea; Ghilardi, Luca; D'ambrosio, Andrea; Benedikter, Boris; Zavoli, Alessandro; Furfaro, Roberto - 01a Articolo in rivista
paper: JOURNAL OF GUIDANCE CONTROL AND DYNAMICS (American Institute of Aeronautics & Astronautics:1801 Alexander Bell Drive, Suite 500:Reston, VA 20191:(800)639-2422, (703)264-7500, EMAIL: custserv@aiaa.org, INTERNET: http://www.aiaa.org/, Fax: (703)264-7657) pp. 2013-2028 - issn: 0731-5090 - wos: WOS:000837958400001 (18) - scopus: 2-s2.0-85141138383 (20)

11573/1713991 - 2022 - Meta-reinforcement learning for adaptive spacecraft guidance during finite-thrust rendezvous missions
Federici, Lorenzo; Scorsoglio, Andrea; Zavoli, Alessandro; Furfaro, Roberto - 01a Articolo in rivista
paper: ACTA ASTRONAUTICA (Elmsford N.Y.: Pergamon Press) pp. 129-141 - issn: 0094-5765 - wos: WOS:000876339500002 (14) - scopus: 2-s2.0-85138828635 (16)

11573/1667073 - 2022 - Surrogate neural network for rapid flight performance evaluation of hybrid rocket engines
Zavoli, A; Zolla, Pm; Federici, L; Migliorino, Mt; Bianchi, D - 01a Articolo in rivista
paper: JOURNAL OF SPACECRAFT AND ROCKETS (American Institute of Aeronautics & Astronautics:1801 Alexander Bell Drive, Suite 500:Reston, VA 20191:(800)639-2422, (703)264-7500, EMAIL: custserv@aiaa.org, INTERNET: http://www.aiaa.org/, Fax: (703)264-7657) pp. 2003-2016 - issn: 0022-4650 - wos: WOS:000834261800001 (4) - scopus: 2-s2.0-85142457017 (7)

11573/1714828 - 2022 - Optimization of chaser trajectory for active debris removal missions
Zona, D.; Avanzini, G.; Federici, L.; Zavoli, A. - 04b Atto di convegno in volume
conference: 73rd International Astronautical Congress, IAC 2022 (Paris; France)
book: Proceedings of the International Astronautical Congress, IAC - ()

11573/1615396 - 2021 - Deep learning techniques for autonomous spacecraft guidance during proximity operations
Federici, L.; Benedikter, B.; Zavoli, A. - 01a Articolo in rivista
paper: JOURNAL OF SPACECRAFT AND ROCKETS (American Institute of Aeronautics & Astronautics:1801 Alexander Bell Drive, Suite 500:Reston, VA 20191:(800)639-2422, (703)264-7500, EMAIL: custserv@aiaa.org, INTERNET: http://www.aiaa.org/, Fax: (703)264-7657) pp. 1774-1785 - issn: 0022-4650 - wos: WOS:000778561600018 (40) - scopus: 2-s2.0-85120848786 (55)

11573/1640430 - 2021 - Meta-reinforcement learning for adaptive spacecraft guidance during multi-target missions
Federici, L.; Scorsoglio, A.; Zavoli, A.; Furfaro, R. - 04b Atto di convegno in volume
conference: IAF Astrodynamics symposium 2021 at the 72nd International astronautical congress, IAC 2021 (Dubai; UAE)
book: Proceedings of the international astronautical congress, IAC - ()

11573/1557791 - 2021 - Evolutionary optimization of multirendezvous impulsive trajectories
Federici, L.; Zavoli, A.; Colasurdo, G. - 01a Articolo in rivista
paper: INTERNATIONAL JOURNAL OF AEROSPACE ENGINEERING (Cairo : Hindawi Publishing Corporation) pp. 1-19 - issn: 1687-5966 - wos: WOS:000669030300003 (7) - scopus: 2-s2.0-85107694338 (16)

11573/1564207 - 2021 - On the use of A* search for active debris removal mission planning
Federici, L.; Zavoli, A.; Colasurdo, G. - 01a Articolo in rivista
paper: JOURNAL OF SPACE SAFETY ENGINEERING (New York: Elsevier Inc. Noordwijk: IAASS) pp. 245-255 - issn: 2468-8975 - wos: WOS:001117486800003 (3) - scopus: 2-s2.0-85110671209 (3)

11573/1482213 - 2021 - Machine learning techniques for autonomous spacecraft guidance during proximity operation
Federici, Lorenzo; Benedikter, Boris; Zavoli, Alessandro - 04b Atto di convegno in volume
conference: AIAA Scitech 2021 Forum (Virtual)
book: AIAA Scitech 2021 Forum - (9781624106095)

11573/1557800 - 2021 - Integrated optimization of first-stage SRM and ascent trajectory of multistage launch vehicles
Federici, Lorenzo; Zavoli, Alessandro; Colasurdo, Guido; Mancini, Lucandrea; Neri, Agostino - 01a Articolo in rivista
paper: JOURNAL OF SPACECRAFT AND ROCKETS (American Institute of Aeronautics & Astronautics:1801 Alexander Bell Drive, Suite 500:Reston, VA 20191:(800)639-2422, (703)264-7500, EMAIL: custserv@aiaa.org, INTERNET: http://www.aiaa.org/, Fax: (703)264-7657) pp. 786-797 - issn: 0022-4650 - wos: WOS:000672531200018 (8) - scopus: 2-s2.0-85110225621 (13)

11573/1644176 - 2021 - Machine learning techniques for flight performance prediction of hybrid rocket engines
Zavoli, A.; Zolla, P. M.; Federici, L.; Migliorino, M. T.; Bianchi, D. - 04b Atto di convegno in volume
conference: AIAA Propulsion and Energy Forum, 2021 (Virtual, Online)
book: AIAA Propulsion and energy forum, 2021 - (9781624106118)

11573/1452839 - 2021 - Reinforcement learning for low-thrust trajectory design of interplanetary missions
Zavoli, Alessandro; Federici, Lorenzo - 04b Atto di convegno in volume
conference: 2020 AAS/AIAA Astrodynamics Specialist Conference (Virtual Event)
book: Advances in the Astronautical Sciences - (9780877036753)

11573/1560071 - 2021 - Reinforcement learning for robust trajectory design of interplanetary missions
Zavoli, Alessandro; Federici, Lorenzo - 01a Articolo in rivista
paper: JOURNAL OF GUIDANCE CONTROL AND DYNAMICS (American Institute of Aeronautics & Astronautics:1801 Alexander Bell Drive, Suite 500:Reston, VA 20191:(800)639-2422, (703)264-7500, EMAIL: custserv@aiaa.org, INTERNET: http://www.aiaa.org/, Fax: (703)264-7657) pp. 1440-1453 - issn: 0731-5090 - wos: WOS:000674868100003 (52) - scopus: 2-s2.0-85114431509 (85)

11573/1640442 - 2021 - Preliminary design of multi-chaser active debris removal missions with evolutionary algorithms
Zona, Danilo; Federici, Lorenzo; Zavoli, Alessandro - 04b Atto di convegno in volume
conference: 2021 AAS/AIAA Astrodynamics Specialist Conference (virtual)
book: Advances in the Astronautical Sciences - (978-0-87703-683-8; 978-0-87703-684-5)

11573/1563349 - 2020 - Analysis of 3GM Callisto gravity experiment of the JUICE mission
Di Benedetto, Mauro; Cappuccio, Paolo; Molli, Serena; Federici, Lorenzo; Zavoli, Alessandro - 04b Atto di convegno in volume
conference: AAS/AIAA Astrodynamics Specialist Conference 2020 (Virtual Event)
book: Astrodynamics 2020 - (978-0-87703-676-0; 978-0-87703-675-3)

11573/1452827 - 2020 - EOS: a parallel, self-adaptive, multi-population evolutionary algorithm for constrained global optimization
Federici, L.; Benedikter, B.; Zavoli, A. - 04b Atto di convegno in volume
conference: 2020 IEEE Congress on Evolutionary Computation, CEC 2020 (virtual)
book: 2020 IEEE Congress on Evolutionary Computation, CEC 2020 - Conference Proceedings - (978-1-7281-6929-3)

11573/1348939 - 2020 - A time-dependent tsp formulation for the design of an active debris removal mission using simulated annealing
Federici, Lorenzo; Zavoli, Alessandro; Colasurdo, Guido - 04b Atto di convegno in volume
conference: AAS/AIAA Astrodynamics Specialist Conference (Portland; Maine (USA))
book: AAS/AIAA Astrodynamics Conference - (978-087703665-4)

11573/1452850 - 2020 - On the use of A* search for active debris removal mission planning
Federici, Lorenzo; Zavoli, Alessandro; Colasurdo, Guido - 04b Atto di convegno in volume
conference: 71th International astronautical congress - IAC 2020 (Virtual Conference)
book: Proceedings of the 71th international astronautical congress - ()

11573/1466970 - 2020 - GTOC X: Solution approach of team sapienza-polito
Zavoli, A.; Federici, L.; Benedikter, B.; Casalino, L.; Colasurdo, G. - 04b Atto di convegno in volume
conference: AAS/AIAA Astrodynamics Specialist Conference, 2019 (Portland; Maine (USA))
book: Advances in the Astronautical Sciences - ()

11573/1321643 - 2019 - Integrated optimization of ascent trajectory and srm design of multistage launch vehicles
Federici, L.; Zavoli, A.; Colasurdo, G.; Mancini, L.; Neri, A. - 04b Atto di convegno in volume
conference: 29th AAS/AIAA Space flight mechanics meeting, 2019 (Maui; United States)
book: Spaceflight mechanics 2019 : proceedings of the AAS/AIAA Space Flight Mechanics Meeting - (978-087703659-3)

11573/1321649 - 2019 - Impulsive multi-rendezvous trajectory design and optimization
Federici, Lorenzo; Zavoli, Alessandro; Colasurdo, Guido - 04b Atto di convegno in volume
conference: 8 TH EUROPEAN CONFERENCE FOR AERONAUTICS AND AEROSPACE SCIENCES (EUCASS) (Madrid, Spagna)
book: Proceedings of 8 TH EUROPEAN CONFERENCE FOR AERONAUTICS AND AEROSPACE SCIENCES (EUCASS) - ()

11573/1414100 - 2019 - Comparative analysis of genetic crossover operators for the optimization of impulsive multi-rendezvous trajectories
Zavoli, A.; Federici, L.; Benedikter, B.; Colasurdo, G. - 04b Atto di convegno in volume
conference: Italian Association of Aeronautics and Astronautics XXV International Congress (Rome, Italy)
book: XXV International Congress of the Italian Association of Aeronautics and Astronautics - Proceedings - (978-88-943960-1-0)

11573/1178544 - 2018 - Preliminary capture trajectory design for Europa tomography probe
Federici, Lorenzo; Zavoli, Alessandro; Colasurdo, Guido - 01a Articolo in rivista
paper: INTERNATIONAL JOURNAL OF AEROSPACE ENGINEERING (Cairo : Hindawi Publishing Corporation) pp. - - issn: 1687-5966 - wos: WOS:000441924000001 (8) - scopus: 2-s2.0-85059102182 (14)

11573/1283553 - 2018 - A preliminary design of a mission to Triton: a concurrent engineering approach
Pollice, L.; Cascioli, G.; Federici, L.; Iannelli, P.; Di Stefano, Ivan; Ciallella, M.; Casini, S.; De Gasperis, S.; Corallo, F.; Rasoni, C. A.; Filice, V.; Eugeni, M.; Palermo, G.; Gaudenzi, P. - 04b Atto di convegno in volume
conference: 69th International Astronautical Congress: #InvolvingEveryone, IAC 2018 (Bremen; Germany)
book: Proceedings of the International Astronautical Congress, IAC - ()

11573/1090956 - 2017 - Preliminary design of a mission to triton based on a concurrent engineering approach
Cascioli, Gael; Federici, L.; Iannelli, P.; Di Stefano, I.; Ciallella, Mirco; Casini, Stefano; De Gasperis, Simone; Corallo, F.; Rasoni, C. A.; Filice, V.; Palermo, G.; Pollice, L.; Eugeni, M.; Gaudenzi, P. - 04b Atto di convegno in volume
conference: ITALIAN ASSOCIATION OF AERONAUTICS AND ASTRONAUTICS XXIV INTERNATIONAL CONFERENCE (PALERMO-ENNA, ITALY)
book: Proceedings of the XXIV International Conference of the Italian Association of Aeronautics and Astronautics - ()

© Università degli Studi di Roma "La Sapienza" - Piazzale Aldo Moro 5, 00185 Roma