SmokeSeger: A Transformer-CNN Coupled Model for Urban Scene Smoke Segmentation

• Published in: IEEE transactions on industrial informatics 2024-02, Vol.20 (2), p.1385-1396
• Main Authors: Jing, Tao, Meng, Qing-Hao, Hou, Hui-Rang
• Format: Article
• Language: English
• Subjects: Annotations; Artificial neural networks; Convolutional neural network; Convolutional neural networks; Datasets; Decoding; dual-branch encoder; Image segmentation; Semantic segmentation; Semantics; Smoke; smoke semantic segmentation; Surveillance systems; Task analysis; transformer; Transformers; urban smoke scene; Visualization
• ISSN: 1551-3203; 1941-0050
• DOI: 10.1109/TII.2023.3271441

Smoke is an informative indicator of early fire and gas leakage. Segmenting the smoke from images can provide detailed information about the smoke volume, dispersion direction, and source location, which has significant implications considering the proliferation of video surveillance systems in cities. Focusing on smoke segmentation in the urban scene, we designed a dual-branch segmentation model, named SmokeSeger, which couples a transformer branch and a convolutional neural network (CNN) branch to enhance the representation of both global and local features. To address the lack of real-scene smoke datasets, we built an urban scene smoke segmentation dataset containing 3217 images of fire smoke and exhaust emissions with accurate annotations. Experiments validate that the SmokeSeger outperforms other mainstream segmentation methods on the proposed dataset. Visualization of attention maps reveals that the model could effectively capture the semantic relationship between the smoke and the corresponding source, which benefits the discrimination between smoke and smoke-like objects. More details available at https://github.com/VisAcademic/SmokeSeger .