VALERIA BELLONI

Dottoressa di ricerca

ciclo: XXXIII


supervisore: Prof. Mattia Crespi

Titolo della tesi: Innovative approaches for infrastructure monitoring through photogrammetry and deep learning

Nowadays, civil infrastructures are getting older and older all over the world. For this reason, innovative and more efficient monitoring systems are required to determine the health and safety level of these infrastructures. In the last decades, different techniques have been investigated in the field of material testing and infrastructure monitoring. Material testing is usually performed in the laboratory to understand the response of building materials. The principle of material testing is based on the mechanical loading of a material up to a certain level of deformation. During testing, the applied force and material deformation are measured; the aim is to retrieve the rheology of the material which is responsible for its behaviour. During testing, standard pointwise sensors such as strain gauges and Linear Variable Differential Transducers (LVDT) and image-based methods (e.g. Digital Image Correlation - DIC) have been widely adopted to measure material deformation and crack propagation. In particular, DIC can measure displacements, strains and cracks by comparing the positions of points in images acquired using a camera with a fixed position. On the other hand, in the field of infrastructure monitoring, cracks are early indicators of damage; crack detection and measurement represent, indeed, key parameters in evaluating the safety and durability of structural components. Therefore, cracks should be detected as soon as possible and monitored over time to assess the condition of the infrastructure and identify the necessary countermeasures. Traditionally, cracks have been routinely measured during visual infrastructure inspections all over the world. The procedure consists of visually estimating the crack width using graduated cards provided with a set of thickness lines. This method is time-consuming, expensive and prone to human errors. Furthermore, the inspections are typically conducted at night since the infrastructure needs to be closed during inspections. While this minimizes the impact of infrastructure downtime, it provides limited time to conduct the inspections in what often are long infrastructure systems. As a result, the efficiency and thoroughness of such inspections have been questioned as the risk of missing cracks is high. To address the limitations of visual inspections, contact and non-contact techniques have been extensively investigated. However, contact sensors provide only local measurements whereas DIC requires a fixed camera to acquire the images, which becomes a challenge for long-term monitoring outside the controlled laboratory conditions. Nowadays, cameras can be placed on vehicles or drones which enable an automatic procedure to collect imagery from infrastructures. An important task for the inspections is to determine whether or not the length or width of the cracks has propagated since the last inspection. However, due to the limitations of standard DIC, crack propagation cannot be measured from these types of images since the position of the camera between the inspections differs. Therefore, the information from the images acquired using mobile mapping systems can only be adopted to generate a digital representation of the infrastructure (the so-called digital twin) and manually compare the characteristics of the cracks between different inspections. Due to the large amount of collected data, this approach is still time-consuming, inefficient and affected by human errors. The aim of this thesis is to investigate image-based techniques for accurate measurements during laboratory tests and infrastructure monitoring. Specifically, this thesis investigates standard and innovative DIC-based approaches to measure displacements, strains and crack propagation at the laboratory scale and innovative image-based techniques (based on the combination of photogrammetry and deep learning) for infrastructure monitoring. For laboratory measurement and material characterization, the DIC technique is investigated. The developed local 2D DIC software, named Py2DIC, is tested comparing its performances with those of other open-source and commercial software applications (Ncorr, DICe and Vic-2D) and the strain gauge sensor. Three different datasets acquired during laboratory tests using a camera with a fixed position are analysed. For software comparisons, the displacement and strain fields are compared globally and locally, whereas, for the strain gauge, the DIC results are retrieved around the area covered by the sensor and then averaged. The results highlight an agreement of a few microns between Py2DIC, the other software packages and the strain gauge. The analyses confirm the potential of the implemented software in measuring displacements, strains and cracks. To overcome the DIC limitations, a new methodology is also presented in this thesis. The approach can measure the propagation of cracks in images captured using cameras with a not fixed position. The innovative model, named Deformation from Motion (DfM), exploits the homography transformation between image planes and template matching algorithms. The model, completely implemented in Python and based on OpenCV libraries, is validated using synthetic data and images acquired with moving cameras during a laboratory test. The results are compared with the standard DIC technique and the LVDT sensor glued on the sample surface. The comparison shows an agreement of a few hundredths of millimetres between DfM and the LVDT. The methodology, tested in the laboratory, can compute the crack propagation using a time series of images; it is, therefore, suitable for processing images captured with mobile mapping systems. For infrastructure monitoring, the TACK (Tunnel and bridge Automatic CracK monitoring using deep learning and photogrammetry) project is presented and discussed. The project, funded by the Swedish Innovation Agency and the Swedish Transport Administration, aims at investigating a new methodology for efficient and accurate infrastructure monitoring. The methodology is based on the LiDAR and close-range photogrammetric techniques for 3D infrastructure modelling, deep learning approaches (Convolutional Neural Networks - CNNs) for crack detection and the DfM model for crack measurement. In this thesis, the use of LiDAR data acquired in a tunnel and images of a bridge are investigated. Specifically, the images are processed using Pix4Dmapper to investigate a low-cost approach (i.e. close-range photogrammetry) for 3D modelling of bridges. Furthermore, the potentialities of CNNs are investigated using different datasets of cracks. Preliminary results obtained using a U-Net based architecture (CrackSegNet) trained with a small dataset show Intersection over Union (IoU) of 51%, Precision of 67%, Recall of 71%, F1 of 68% and Accuracy of 99% on the test set. The results also demonstrate the possibility of detecting cracks in unseen images acquired in a tunnel. Unfortunately, other objects with a similar shape can be erroneously detected as cracks. For this reason, a new processing pipeline based on the U-Net segmentation model combined with different backbones is developed and trained using a larger dataset. The trained model achieves better results on the testing set and the unseen tunnel dataset. The result assessment using U-Net combined with the VGG (Visual Geometry Group) 19 backbone highlights IoU of 77%, Precision of 88%, Recall of 91%, F1 of 87% and Accuracy of 99% on the test set. The trained model can also better distinguish cracks from similarly-looking objects. The performed experiments demonstrate the possibility to adopt deep learning architectures to detect cracks in images of tunnels increasing, in this way, the efficiency of infrastructure monitoring.

Produzione scientifica

11573/1693291 - 2023 - High-resolution high-accuracy orthophoto map and digital surface model of Forni Glacier tongue (Central Italian Alps) from UAV photogrammetry
Belloni, Valeria; Di Rita, Martina; Fugazza, Davide; Traversa, Giacomo; Hanson, Kevin; Diolaiuti And Mattia Crespi, Guglielmina - 01a Articolo in rivista
rivista: JOURNAL OF MAPS (Abingdon U.K.: Taylor & Francis Surrey U.K.: School of Earth Sciences and Geography Kingston University, 2005-) pp. - - issn: 1744-5647 - wos: WOS:001013269100001 (1) - scopus: 2-s2.0-85163651614 (2)

11573/1693287 - 2023 - Crack Monitoring from Motion (CMfM): Crack detection and measurement using cameras with non-fixed positions
Belloni, Valeria; Sjölander, Andreas; Ravanelli, Roberta; Crespi, Mattia; Nascetti, Andrea - 01a Articolo in rivista
rivista: AUTOMATION IN CONSTRUCTION (Amsterdam ; Tokyo ; Oxford ; New York : Elsevier Science Publishers) pp. - - issn: 0926-5805 - wos: WOS:001077637400001 (1) - scopus: 2-s2.0-85172935027 (1)

11573/1682825 - 2023 - Water reservoirs monitoring through Google Earth Engine: application to Sentinel and Landsat imagery
Bocchino, Filippo; Ravanelli, Roberta; Belloni, Valeria; Mazzucchelli, Paolo; Crespi, Mattia - 04c Atto di convegno in rivista
rivista: INTERNATIONAL ARCHIVES OF THE PHOTOGRAMMETRY, REMOTE SENSING AND SPATIAL INFORMATION SCIENCES (ISPRS Council) pp. 41-47 - issn: 1682-1750 - wos: (0) - scopus: 2-s2.0-85156213720 (1)
congresso: 39th International Symposium on Remote Sensing of Environment, ISRSE 2023 (Turchia; Adalia)

11573/1696649 - 2023 - Earth Observation Big Data Exploitation for Water Reservoirs Continuous Monitoring. The Potential of Sentinel-2 Data and HPC
Ravanelli, Roberta; Mazzucchelli, Paolo; Belloni, Valeria; Bocchino, Filippo; Morselli, Laura; Fiorino, Andrea; Gerace, Fabio; Crespi, Mattia - 04b Atto di convegno in volume
congresso: Workshop at the 2022, 2nd International Conference on Applied Intelligence and Informatics (Reggio Calabria)
libro: The Use of Artificial Intelligence for Space Applications - (978-3-031-25754-4; 978-3-031-25755-1)

11573/1693285 - 2023 - Towards Automated Inspections of Tunnels. A Review of Optical Inspections and Autonomous Assessment of Concrete Tunnel Linings
Sjölander, Andreas; Belloni, Valeria; Ansell, Anders; Nordström, Erik - 01a Articolo in rivista
rivista: SENSORS (Basel : Molecular Diversity Preservation International (MDPI), 2001-) pp. - - issn: 1424-8220 - wos: WOS:000958663200001 (3) - scopus: 2-s2.0-85151195203 (3)

11573/1693283 - 2023 - Experimental dataset to assess the structural performance of cracked reinforced concrete using Digital Image Correlation techniques with fixed and moving cameras
Sjölander, Andreas; Belloni, Valeria; Peterson, Viktor; Ledin, Jonatan - 01a Articolo in rivista
rivista: DATA IN BRIEF (New York : Elsevier Inc.) pp. - - issn: 2352-3409 - wos: WOS:001105272000001 (0) - scopus: 2-s2.0-85175268040 (0)

11573/1673506 - 2022 - Digitalization and defects analysis for the maintenance of mechanized tunnels
Foria, Federico; Miceli, Gabriele; Nascetti, Andrea; Loprencipe, Giuseppe; Crespi, Mattia Giovanni; Belloni, Valeria; Ravanelli, Roberta; Cordaro, Sara - 04b Atto di convegno in volume
congresso: World Tunnel Congress 2022 Underground solutions for a world in change (Copenhagen, Denmark)
libro: Proceedings World Tunnel Congress, WTC2022 - (978-2-9701436-7-3)

11573/1495932 - 2021 - DSM Generation from Single and Cross-Sensor Multi-View Satellite Images Using the New Agisoft Metashape: The Case Studies of Trento and Matera (Italy)
Lastilla, Lorenzo; Belloni, Valeria; Ravanelli, Roberta; Crespi, Mattia - 01a Articolo in rivista
rivista: REMOTE SENSING (Basel : Molecular Diversity Preservation International) pp. - - issn: 2072-4292 - wos: WOS:000624428600001 (11) - scopus: 2-s2.0-85100760438 (14)

11573/1581676 - 2021 - Sharing soil and building geophysical data for seismic characterization of cities using CLARA Webgis. A case study of Matera (southern Italy)
Tragni, N.; Calamita, G.; Lastilla, L.; Belloni, V.; Ravanelli, R.; Lupo, M.; Salvia, V.; Gallipoli, M. R. - 01a Articolo in rivista
rivista: APPLIED SCIENCES (Basel: MDPI AG, 2011-) pp. - - issn: 2076-3417 - wos: WOS:000649954700001 (4) - scopus: 2-s2.0-85106653223 (4)

11573/1558573 - 2020 - COSMO-SkyMed Range Measurements for Displacement Monitoring Using Amplitude Persistent Scatterers
Belloni, V.; Di Tullio, M.; Ravanelli, R.; Fratarcangeli, F.; Nascetti, A.; Crespi, M. - 04b Atto di convegno in volume
congresso: 2020 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2020, Virtual, Waikoloa, 26 September 2020 - 2 October 2020, (Waikoloa, (Hawaii, USA))
libro: International Geoscience and Remote Sensing Symposium (IGARSS) - (978-1-7281-6374-1)

11573/1458000 - 2020 - Tack project. Tunnel and bridge automatic crack monitoring using deep learning and photogrammetry
Belloni, V.; Sjölander, A.; Ravanelli, R.; Crespi, M.; Nascetti, A. - 04c Atto di convegno in rivista
rivista: INTERNATIONAL ARCHIVES OF THE PHOTOGRAMMETRY, REMOTE SENSING AND SPATIAL INFORMATION SCIENCES ([Göttingen] : Copernicus Publ.) pp. 741-745 - issn: 2194-9034 - wos: (0) - scopus: 2-s2.0-85104784898 (8)
congresso: XXIV ISPRS Congress (Nice (France) - virtual)

11573/1457996 - 2020 - Glacier volume change monitoring from UAV observation. Issues and potentials of state-of-the-art techniques
Di Rita, M.; Fugazza, D.; Belloni, V.; Diolaiuti, G.; Scaioni, M.; Crespi, M. - 04c Atto di convegno in rivista
rivista: INTERNATIONAL ARCHIVES OF THE PHOTOGRAMMETRY, REMOTE SENSING AND SPATIAL INFORMATION SCIENCES (ISPRS Council) pp. 1041-1048 - issn: 1682-1750 - wos: (0) - scopus: 2-s2.0-85091109631 (8)
congresso: 2020 24th ISPRS Congress - Technical Commission II (Nice (France) - virtual)

11573/1350209 - 2019 - py2DIC. A New Free and Open Source Software for Displacement and Strain Measurements in the Field of Experimental Mechanics
Belloni, Valeria; Ravanelli, Roberta; Nascetti, Andrea; Di Rita, Martina; Mattei, Domitilla; Crespi, Mattia - 01a Articolo in rivista
rivista: SENSORS (Basel : Molecular Diversity Preservation International (MDPI), 2001-) pp. - - issn: 1424-8220 - wos: WOS:000489187800018 (20) - scopus: 2-s2.0-85071896530 (32)

11573/1122645 - 2018 - Digital image correlation from commercial to FOS software. A mature technique for full-field displacement measurements
Belloni, Valeria; Ravanelli, Roberta; Nascetti, Andrea; Di Rita, Martina; Mattei, Domitilla; Crespi, Mattia - 04c Atto di convegno in rivista
rivista: INTERNATIONAL ARCHIVES OF THE PHOTOGRAMMETRY, REMOTE SENSING AND SPATIAL INFORMATION SCIENCES (ISPRS Council) pp. 91-95 - issn: 1682-1750 - wos: (0) - scopus: 2-s2.0-85048373789 (9)
congresso: 2018 ISPRS TC II Mid-term Symposium "Towards Photogrammetry 2020" (Riva Del Garda, Italia)

11573/1015713 - 2017 - New trends In geomatics, in the era of low-cost sensors, free and open source software and HPC geoBigData infrastructures
Ravanelli, Roberta; Di Rita, Martina; Belloni, Valeria; Nascetti, Andrea; Mazzoni, Augusto; Crespi, Mattia Giovanni - 01a Articolo in rivista
rivista: GEOMEDIA (-Rimini : Maggioli -Roma : Geo4all, 2003-) pp. 6-9 - issn: 1128-8132 - wos: WOS:000416287800002 (0) - scopus: (0)

11573/1006776 - 2017 - A new digital image correlation software for displacements field measurement in structural applications
Ravanelli, Roberta; Nascetti, Andrea; Di Rita, Martina; Belloni, Valeria; Mattei, Domitilla; Nistico', Nicola; Crespi, Mattia Giovanni - 04c Atto di convegno in rivista
rivista: INTERNATIONAL ARCHIVES OF THE PHOTOGRAMMETRY, REMOTE SENSING AND SPATIAL INFORMATION SCIENCES (ISPRS Council) pp. 139-145 - issn: 1682-1750 - wos: WOS:000582726700021 (6) - scopus: 2-s2.0-85027227892 (15)
congresso: 2017 Free and Open Source Software for Geospatial Conference-Europe, FOSS4G-Europe 2017 (Marne La Vallee, (Ecole Nationale des Sciences Geographiques ENSG))
libro: International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives - ()

11573/1015714 - 2016 - Protocollo operativo per la validazione geometrica di immagini satellitari ad alta risoluzione
Crespi, Mattia; De Paulis Riccardo, ; Pellegri, Francesco; Capaldo, Paola; Fratarcangeli, Francesca; Gini, Rossana; Nascetti, Andrea; Selva, Federica; Ravanelli, Roberta; Di Rita, Martina; Belloni, Valeria; Augusto, Mazzoni - 01a Articolo in rivista
rivista: GEOMEDIA (-Rimini : Maggioli -Roma : Geo4all, 2003-) pp. 6-9 - issn: 1128-8132 - wos: WOS:000388710300001 (0) - scopus: (0)

© Università degli Studi di Roma "La Sapienza" - Piazzale Aldo Moro 5, 00185 Roma