HSTNet: An Iterative Optimization Network for Semantic Segmentation of High-Resolution Remote Sensing Images

Tingkai Wang

Abstract: How to achieve accurate semantic segmentation of high-resolution remote sensing images is a current focal point in image semantic segmentation tasks. However, the information contained in high-resolution images is typically complex, and due to the large size of the images, they are constrained by the receptive field size of convolutional networks, making accurate semantic segmentation challenging. Significant errors exist in both local edge and overall image segmentation results. This paper presents HSTNet, a semantic segmentation network for high-resolution remote sensing images with an iterative structure. HSTNet adopts an encoder-decoder architecture similar to Unet. In HSTNet, we employ Swin-Transformer modules to learn and correlate feature tensors at different scales, aiming to capture the overall structure of high-resolution images and associate long-range geographic information across the images as much as possible. Furthermore, we devised an iterative optimization framework that progressively enhances the semantic segmentation results of the network. We observed that preliminary semantic segmentation outputs can serve as cues to facilitate the network in achieving more accurate segmentation. These initial semantic segmentation results encapsulate relationships among various semantic objects within different regions of the image, thereby reducing the cost of the network learning image features during subsequent iterations and assisting the network in achieving improved outcomes. We compared our approach with several state-of-the-art methods on the Potsdam dataset from ISPRS. The final results indicate that our method achieves outstanding performance.

Keywords: Deep learning, Semantic segmentation, Image processing