Buildings undergo various kinds of structural damage during earthquakes,and damage detection and functional assessment of these structures in the aftermath of the events have been challenging issues.Under these circum...Buildings undergo various kinds of structural damage during earthquakes,and damage detection and functional assessment of these structures in the aftermath of the events have been challenging issues.Under these circumstances,computer vision techniques offer a promising solution by automating the inspection process.This study presents an effective methodology for automatic structural components and damage detection using unmanned aerial vehicle(UAV)images of damaged buildings.Two types of neural network architectures are considered for appropriate feature extractions in different task detections.The feature pyramid network(FPN)is employed for crack,spall,rebar,and component damage segmentation,while the UNet++network is utilized for the damage state.For network training and validation,a total of 3805 original images of size 1920×1080 pixels are processed by the proposed method and reduced the image pixels.From the FPN,the achieved highest intersection over unions(IoUs)were 0.59,0.93,0.42,and 0.99 for crack,spall,rebar,and components,respectively.These predicted labels were found in close agreement with the labels.Similarly,the UNet++recognized the semantic information and damage state with an IoU of 0.72.This demonstrated the applicability of the proposed method for automated post-earthquake building inspection process accurately without information loss from the original images.展开更多
Recent studies for computer vision and deep learning-based,post-earthquake inspections on RC structures mainly perform well for specific tasks,while the trained models must be fine-tuned and re-trained when facing new...Recent studies for computer vision and deep learning-based,post-earthquake inspections on RC structures mainly perform well for specific tasks,while the trained models must be fine-tuned and re-trained when facing new tasks and datasets,which is inevitably time-consuming.This study proposes a multi-task learning approach that simultaneously accomplishes the semantic segmentation of seven-type structural components,three-type seismic damage,and four-type deterioration states.The proposed method contains a CNN-based encoder-decoder backbone subnetwork with skip-connection modules and a multi-head,task-specific recognition subnetwork.The backbone subnetwork is designed to extract multi-level features of post-earthquake RC structures.The multi-head,task-specific recognition subnetwork consists of three individual self-attention pipelines,each of which utilizes extracted multi-level features from the backbone network as a mutual guidance for the individual segmentation task.A synthetical loss function is designed with real-time adaptive coefficients to balance multi-task losses and focus on the most unstably fluctuating one.Ablation experiments and comparative studies are further conducted to demonstrate their effectiveness and necessity.The results show that the proposed method can simultaneously recognize different structural components,seismic damage,and deterioration states,and that the overall performance of the three-task learning models gains general improvement when compared to all single-task and dual-task models.展开更多
文摘Buildings undergo various kinds of structural damage during earthquakes,and damage detection and functional assessment of these structures in the aftermath of the events have been challenging issues.Under these circumstances,computer vision techniques offer a promising solution by automating the inspection process.This study presents an effective methodology for automatic structural components and damage detection using unmanned aerial vehicle(UAV)images of damaged buildings.Two types of neural network architectures are considered for appropriate feature extractions in different task detections.The feature pyramid network(FPN)is employed for crack,spall,rebar,and component damage segmentation,while the UNet++network is utilized for the damage state.For network training and validation,a total of 3805 original images of size 1920×1080 pixels are processed by the proposed method and reduced the image pixels.From the FPN,the achieved highest intersection over unions(IoUs)were 0.59,0.93,0.42,and 0.99 for crack,spall,rebar,and components,respectively.These predicted labels were found in close agreement with the labels.Similarly,the UNet++recognized the semantic information and damage state with an IoU of 0.72.This demonstrated the applicability of the proposed method for automated post-earthquake building inspection process accurately without information loss from the original images.
基金National Key R&D Program of China under Grant No.2019YFC1511005the National Natural Science Foundation of China under Grant Nos.51921006,52192661 and 52008138+2 种基金the China Postdoctoral Science Foundation under Grant Nos.BX20190102 and 2019M661286the Heilongjiang Natural Science Foundation under Grant No.LH2022E070the Heilongjiang Province Postdoctoral Science Foundation under Grant Nos.LBH-TZ2016 and LBH-Z19064。
文摘Recent studies for computer vision and deep learning-based,post-earthquake inspections on RC structures mainly perform well for specific tasks,while the trained models must be fine-tuned and re-trained when facing new tasks and datasets,which is inevitably time-consuming.This study proposes a multi-task learning approach that simultaneously accomplishes the semantic segmentation of seven-type structural components,three-type seismic damage,and four-type deterioration states.The proposed method contains a CNN-based encoder-decoder backbone subnetwork with skip-connection modules and a multi-head,task-specific recognition subnetwork.The backbone subnetwork is designed to extract multi-level features of post-earthquake RC structures.The multi-head,task-specific recognition subnetwork consists of three individual self-attention pipelines,each of which utilizes extracted multi-level features from the backbone network as a mutual guidance for the individual segmentation task.A synthetical loss function is designed with real-time adaptive coefficients to balance multi-task losses and focus on the most unstably fluctuating one.Ablation experiments and comparative studies are further conducted to demonstrate their effectiveness and necessity.The results show that the proposed method can simultaneously recognize different structural components,seismic damage,and deterioration states,and that the overall performance of the three-task learning models gains general improvement when compared to all single-task and dual-task models.
