Semantic segmentation of roof materials in urban environment by utilizing hyperspectral and LiDAR data

Английский. Mapping areas in an urban environment can be challenging due to various materials and manufactured structures. The urban environment is a mix of natural andartificial materials, and finding the right object of a specific material is a challengeeven for the trained eye. Therefore, by applying high spectral resolution hyperspectral imagery it is possible to examine surface materials based on spectral signature.Combined with LiDAR, it is also feasible to detect the geometrical structure of thesurface. These data can be exposed to a machine learning algorithm to recognizeobjects automatically. In this study machine learning algorithms are exposed toairborne images of roof materials.This thesis presents an application of semantic segmentation for roof materials based on fused hyperspectral (HySpex VNIR-1800 and SWIR-384) and LiDAR(Riegl VQ-560i) data acquired from 2021 over Bærum municipality near Oslo inNorway. The machine learning algorithm is a semantic segmentation model namedRes-U-net with a U-net architecture and a ResNet34 backbone. The Res-U-Net isa supervised neural network with high capacity to learn high-dimensional airbornedata. The model returns a mask of the urban area that pinpoints the roofs’ positionand materials. The ground truth is generated with information from field work,a geographical database and the watershed algorithm for object detection. Thisground truth consists of nine different roof materials and background.The semantic segmentation model is optimized by testing different model configurations for this specific problem. The best model scores 0.903, 0.896, and 0.579in accuracy score, F1 score weighted and Matthews Correlation Coefficient. For thebinary problem of detecting roof the model scores 0.948, 0.946, and 0.767 on thesame metrics. This study demonstrates that semantic segmentation is viable forlocalizing and classifying roof materials with fused hyperspectral and LiDAR data.Such an analysis can potentially automate several mapping chores and manual assignments by systemically processing a larger area in a short time to free humancapacity.

M-MF