Exploring the Capability of Text-to-Image Diffusion Models With Structural Edge Guidance for Multispectral Satellite Image Inpainting
Mikolaj Czerkawski, Christos Tachtatzis
Abstract
The paper investigates the utility of text-to-image inpainting models for satellite image data. Two technical challenges of injecting structural guiding signals into the generative process as well as translating the inpainted RGB pixels to a wider set of MSI bands are addressed by introducing a novel inpainting framework based on StableDiffusion and ControlNet as well as a novel method for RGB-to-MSI translation. The results on a wider set of data suggest that the inpainting synthesized via StableDiffusion suffers from undesired artefacts and that a simple alternative of self-supervised internal inpainting achieves a higher quality of synthesis.
Topics & Concepts
Multispectral imageInpaintingArtificial intelligenceComputer visionComputer scienceImage (mathematics)Satellite imageSatelliteEnhanced Data Rates for GSM EvolutionPattern recognition (psychology)EngineeringAerospace engineeringGenerative Adversarial Networks and Image SynthesisImage Retrieval and Classification Techniques