FormerLeaf: An efficient vision transformer for Cassava Leaf Disease detection

• Published in: Computers and electronics in agriculture 2023-01, Vol.204, p.107518, Article 107518
• Main Authors: Thai, Huy-Tan, Le, Kim-Hung, Nguyen, Ngan Luu-Thuy
• Format: Article
• Language: English
• Subjects: Deep learning; Efficient vision transformer; Image classification; Leaf Disease detection; Precision agriculture
• ISSN: 0168-1699; 1872-7107
• DOI: 10.1016/j.compag.2022.107518

Leaf diseases have become more prevalent in recent years due to climate change, increased growth of outdoor air pollutants, and global warming. They may severely damage crop yield, leading to detrimental effects on global food security. The timely and precise detection of leaf diseases is thus crucial for preventing their spread and ensuring the sustainability of agricultural production. In this paper, we introduce a transformer-based leaf disease detection model, namely FormerLeaf along with two optimization methods to enhance the model performance. In more detail, we propose the Least Important Attention Pruning (LeIAP) algorithm to select the most important attention heads of each layer in the Transformer model. It could reduce the model size up to 28% and accelerate the evaluation speed by 15% with about 3% accuracy enhancement. In addition, we employ the sparse matrix-matrix multiplication (SPMM) to calculate matrix correlation in the model. This reduces the model’s complexity from O(n2) to O(n2p), resulting in lowering training time by 10% while keeping a similar performance. The evaluation results on the Cassava Leaf Disease Dataset show that our proposal outperforms the state-of-the-art models in most cases. •Proposing an efficient transformer-based model to precisely detect leaf diseases.•Designing a pruning algorithm to optimize attention heads of each model layer.•Employing sparse matrix-matrix multiplication to reduce the training time.