Extracting roof sub-components from orthophotos using deep-learning -based semantic segmentation

Best practice for the detection and annotation of visible defects in slated roofs is by annotation of photos, ideally or-thophotos. If such a process is to be effectively automated in support of emerging Digital Twinning solutions, it is necessary to first recognise the external sub-components of the roof in the orthophotos, in particular the slated and leadwork areas. Using a dataset composed of many photos from two historic buildings, this study develops and compares different deep-learning -based semantic segmentation models to segment roof orthophotos into slated areas, leadwork, and ‘other’ areas. Since orthophotos typically contain pixels which do not belong to the roof panel (black ‘background’ pixels), the method employs a subsequent ‘background’ label correction step. The best-performing model is found to be PointRend with Focal Loss: overall aAcc = 99, mIoU = 88.91, and mAcc = 92.77; for slate class, IoU and Acc is nearly 100; for leadwork class, IoU and Acc is around 90.