A robust gap-filling method for predicting missing observations in daily Black Marble nighttime light data

ABSTRACTNighttime light (NTL) remote sensing data plays a crucial role in comprehending changes in human activities. The availability of the daily lunar BRDF-corrected Black Marble NTL product (VNP46A2) enables the use of NTL data to detect and assess the impact of short-term emergencies. However, daily NTL data often experience missing values due to cloud cover and low-quality signals. To address this issue, many studies utilize monthly or annual time-composite NTL products, which restrict the timeliness and potential application scenarios of NTL data usage. Therefore, it is necessary to generate the gap-filled daily NTL product. This study presented a novel NTL gap-filling method comprising rough reconstruction based on spatiotemporal weighting and refined gap-filling using a Bidirectional Long Short-Term Memory (Bi-LSTM) model. We evaluate the accuracy of the proposed method using the “remove-reconstruct-compare” approach, which randomly removes some original data from the complete image, fills the gaps with the proposed gap-filling method, and compares the reconstructed NTL data with the original observations in Beijing, Shanghai, Xi’an and New York. The results reveal that when the rate of missing values in Beijing is 40% and 50%, the proposed gap-filling method achieves accuracy with mean coefficient of determination (R2) values of 0.834 and 0.841, accompanied by corresponding root mean square (RMSE) values of 7.793 and 7.171, respectively. Furthermore, the gap-filling accuracy was evaluated quantitatively, and our proposed gap-filling method performed better than the Spatial and Temporal Adaptive Reflectance Fusion Model (STARFM). Our proposed gap-filling method had R2 values of 0.685, 0.781, 0.720 and 0.642, which were higher than those for STARFM (0.430, 0.662, 0.221 and 0.345). The RMSE values of our gap-filling method were 9.628, 12.083, 10.963 and 19.882 for the four sites, while those of STARFM were 12.953, 14.872, 18.280 and 26.990, respectively. The temporal and spatial analysis results demonstrate that this model is robust, capturing city boundaries and NTL high-brightness hotspots with high accuracy and stability. The gap-filling model proposed in this study provides a new technique for expanding the potential applications and reliability of NASA’s daily Black Marble product (VNP46A2) in remote sensing.