Multi-source knowledge graph reasoning for ocean oil spill detection from satellite SAR images

• Published in: International journal of applied earth observation and geoinformation 2023-02, Vol.116, p.103153, Article 103153
• Main Authors: Liu, Xiaojian, Zhang, Yongjun, Zou, Huimin, Wang, Fei, Cheng, Xin, Wu, Wenpin, Liu, Xinyi, Li, Yansheng
• Format: Article
• Language: English
• Subjects: Knowledge graph reasoning; Multi-source knowledge graph; SAR image oil spill detection
• ISSN: 1569-8432; 1872-826X
• DOI: 10.1016/j.jag.2022.103153

Marine oil spills can cause severe damage to the marine environment and biological resources. Using satellite remote sensing technology is one of the best ways to monitor the sea surface in near real-time to obtain oil spill information. The existing methods in the literature either use deep convolutional neural networks in synthetic aperture radar (SAR) images to directly identify oil spills or use traditional methods based on artificial features sequentially to distinguish oil spills from sea surface. However, both approaches currently only use image information and ignore some valuable auxiliary information, such as marine weather conditions, distances from oil spill candidates to oil spill sources, etc. In this study, we proposed a novel method to help detect marine oil spills by constructing a multi-source knowledge graph, which was the first one specifically designed for oil spill detection in the remote sensing field. Our method can rationally organize and utilize various oil spill-related information obtained from multiple data sources, such as remote sensing images, vectors, texts, and atmosphere-ocean model data, which can be stored in a graph database for user-friendly query and management. In order to identify oil spills more effectively, we also proposed 13 new dark spot features and then used a feature selection technique to create a feature subset that was favorable to oil spill detection. Furthermore, we proposed a knowledge graph-based oil spill reasoning method that combines rule inference and graph neural network technology, which pre-inferred and eliminated most non-oil spills using statistical rules to alleviate the problem of imbalanced data categories (oil slick and non-oil slick). Entity recognition is ultimately performed on the remaining oil spill candidates using a graph neural network algorithm. To verify the effectiveness of our knowledge graph approach, we collected 35 large SAR images to construct a new dataset, for which the training set contained 110 oil slicks and 66264 non-oil slicks from 18 SAR images, the validation set contained 35 oil slicks and 69005 non-oil slicks from 10 SAR images, and the testing set contained 36 oil slicks and 36281 non-oil slicks from the remaining 7 SAR images. The results showed that some traditional oil spill detection methods and deep learning models failed when the dataset suffered a severe imbalance, while our proposed method identified oil spills with a sensitivity of 0.8428, specificity