Logo PTI Logo FedCSIS

Proceedings of the 21st Conference on Computer Science and Intelligence Systems (FedCSIS)

Annals of Computer Science and Information Systems, Volume 47

Diffusion–CLIP Guided Dual-Branch Framework for Infected Region Segmentation in Fruit and Leaf Images

, , , , , ,

DOI: http://dx.doi.org/10.15439/2026F5271

Citation: Poornima Basatti Hanuma Gowda, , , , , ,

Full text

Abstract. Accurate segmentation of plant diseases caused by viruses, bacteria, and fungi is a critical step for automated crop monitoring and precision agriculture. However, variations in disease appearance, illumination, and background clutter make pixel-level delineation highly challenging. In this paper, we propose a novel diffusion--CLIP-guided dual-branch segmentation framework that explicitly models the relationship between healthy and infected tissue regions. The proposed architecture integrates a diffusion-based normality deviation module to highlight abnormal regions, a CLIP-based semantic interpretation module to provide high-level contextual guidance, and a dual-branch encoder that separately learns representations for healthy and infected regions in the input image. A cross-region reasoning mechanism further enhances relational understanding between normal and infected regions, while multi-scale feature fusion ensures robust segmentation across varying disease scales. Experimental evaluations on fruit and leaf disease datasets demonstrate that the proposed method consistently outperformed existing approaches and achieved Dice scores of 93.10\% on fruit images and 90.08\% on leaf images.

References

  1. E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo, “SegFormer: Simple and efficient design for semantic segmentation with transformers,” in Advances in Neural Information Processing Systems, vol. 34, pp. 12077–12090, 2021, https://dx.doi.org/10.48550/arXiv.2105.15203.
  2. B. Cheng, I. Misra, A. G. Schwing, A. Kirillov, and R. Girdhar, “Masked-attention mask transformer for universal image segmentation,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2022, pp. 1290–1299, https://dx.doi.org/10.1109/CVPR52688.2022.00135.
  3. F. Li, H. Zhang, H. Xu, S. Liu, L. Zhang, L. M. Ni, and H. Y. Shum, “Mask DINO: Towards a unified transformer-based framework for object detection and segmentation,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2023, pp. 3041–3051, https://dx.doi.org/10.1109/CVPR52729.2023.00297.
  4. Y. Wu, Z. Xi, F. Liu, W. Hu, H. Feng, and Q. Zhang, “A deep semantic network-based image segmentation of soybean rust pathogens,” Front. Plant Sci., vol. 15, Art. no. 1340584, 2024, https://dx.doi.org/10.3389/fpls.2024.1340584.
  5. V. Badrinarayanan, A. Kendall, and R. Cipolla, “SegNet: A deep convolutional encoder–decoder architecture for image segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 12, pp. 2481–2495, 2017, https://dx.doi.org/10.1109/TPAMI.2016.2644615.
  6. G. Singh, A. A. Al-Huqail, A. Almogren, S. Kaur, K. Joshi, and S. Bharany, “Enhanced leaf disease segmentation using U-Net architecture for precision agriculture: A deep learning approach,” Food Sci. Nutr., vol. 13, no. 7, Art. no. e70594, 2025, https://dx.doi.org/10.1002/fsn3.70594.
  7. G. Zheng, Z. Jiang, X. Zhang, and D. Jiang, “Multi-scale feature fusion-based semantic segmentation network for agricultural remote sensing images,” Chem. Biol. Technol. Agric., vol. 12, 2025, https://dx.doi.org/10.1186/s40538-025-00833-8.
  8. K. Sahu, S. Tiwari, M. K. Singh, J. S. P., H. K. Shakya, and G. Kumar, “Plant disease detection using a hybrid dilated CNN with attention mechanisms and optimized Mask R-CNN segmentation,” Sci. Rep., vol. 15, Art. no. 42008, 2025, https://dx.doi.org/10.1038/s41598-025-26192.
  9. M. K. Surehli, N. Aggarwal, G. Joshi, and H. Nayyar, “Semantic segmentation of plant structures with deep learning and channel-wise attention mechanism,” J. Telecommun. Inf. Technol., vol. 99, no. 1, pp. 56–66, 2025, https://dx.doi.org/10.26636/jtit.2025.1.1853.
  10. H. Xu, J. Song, and Y. Zhu, “Evaluation and comparison of semantic segmentation networks for rice identification based on Sentinel-2 imagery,” Remote Sens., vol. 15, no. 6, Art. no. 1499, 2023, https://dx.doi.org/10.3390/rs15061499.
  11. D. Sotoude, M. Hoseinkhani, and A. Amiri Tehranizadeh, “Context-aware fusion of transformers and CNNs for medical image segmentation,” Informat. Med. Unlocked, vol. 43, Art. no. 101396, 2023, https://dx.doi.org/10.1016/j.imu.2023.101396.
  12. S. Minaee, Y. Boykov, F. Porikli, A. Plaza, N. Kehtarnavaz, and D. Terzopoulos, “Image segmentation using deep learning: A survey,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 7, pp. 3523–3542, 2022, https://dx.doi.org/10.1109/TPAMI.2021.3059968.
  13. L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam, “Encoder–decoder with atrous separable convolution for semantic image segmentation,” in Proc. Eur. Conf. Comput. Vis. (ECCV), 2018, pp. 801–818, https://dx.doi.org/10.1007/978-3-030-01234-2_49.
  14. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2015, https://dx.doi.org/10.48550/arXiv.1409.1556.
  15. L. Lei, Q. Yang, L. Yang, T. Shen, R. Wang, and C. Fu, “Deep learning implementation of image segmentation in agricultural applications: A comprehensive review,” Artif. Intell. Rev., vol. 57, Art. no. 149, 2024, https://dx.doi.org/10.1007/s10462-024-10775-6.
  16. T. Zhou and W. Wang, “Prototype-based semantic segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 46, no. 10, pp. 6858–6872, Oct. 2024, https://dx.doi.org/10.1109/TPAMI.2024.3387116.
  17. S. Zhang, Y. Zhao, and E. Y. Lam, “Semi-supervised semantic segmentation for light field images using disparity information,” IEEE Trans. Image Process., 2024, https://dx.doi.org/10.1109/TIP.2024.3441930.
  18. Z. Zheng, G. Huang, X. Yuan, C.-M. Pun, H. Liu, and W.-K. Ling, “Quaternion-valued correlation learning for few-shot semantic segmentation,” IEEE Trans. Circuits Syst. Video Technol., vol. 33, no. 5, pp. 2102–2115, May 2023, https://dx.doi.org/10.1109/TCSVT.2022.3223150.
  19. A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, et al., “Segment Anything,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), Paris, France, 2023, pp. 3992–4003, https://dx.doi.org/10.48550/arXiv.2304.02643.
  20. T. Yang, Y. Wang, and J. Lian, “Plant diseased lesion image segmentation and recognition based on improved multi-scale attention network,” Appl. Sci., vol. 14, no. 5, Art. no. 1716, 2024, https://dx.doi.org/10.3390/app14051716.
  21. C. Zhao, C. Li, X. Wang, X. Wu, Y. Du, H. Chai, T. Cai, H. Xiang, and Y. Jiao, “Plant disease segmentation networks for fast automatic severity estimation under natural field scenarios,” Agriculture, vol. 15, no. 6, Art. no. 583, 2025, https://dx.doi.org/10.3390/agriculture15060583.
  22. R. Strudel, R. Garcia, I. Laptev, and C. Schmid, “Segmenter: Transformer for semantic segmentation,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), 2021, pp. 7262–7272, https://dx.doi.org/10.48550/arXiv.2105.05633.
  23. Y. Rao, W. Zhao, G. Chen, Y. Tang, Z. Zhu, G. Huang, J. Zhou, and J. Lu, “DenseCLIP: Language-guided dense prediction with context-aware prompting,” arXiv preprint https://arxiv.org/abs/2112.01518, 2022, https://dx.doi.org/10.48550/arXiv.2112.01518.
  24. A. Das, F. Pathan, J. R. Jim, M. M. Kabir, and M. F. Mridha, “Deep learning-based classification, detection, and segmentation of tomato leaf diseases: A state-of-the-art review,” Artif. Intell. Agric., vol. 15, pp. 192–220, 2025, https://dx.doi.org/10.1016/j.aiia.2025.02.006.
  25. X. Zhang, F. Li, H. Zheng, and W. Mu, “UPFormer: U-shaped perception lightweight transformer for segmentation of field grape leaf diseases,” Expert Syst. Appl., vol. 249, Art. no. 123546, 2024, https://dx.doi.org/10.1016/j.eswa.2024.123546.
  26. H. Mao and J. Lazaro, “Remote sensing image segmentation methods based on deep learning models,” J. Comput. Signal Syst. Res., vol. 2, no. 2, pp. 124–133, 2025, https://dx.doi.org/10.71222/15ra5x93.
  27. J. Cheng, H. Li, D. Li, S. Hua, and V. S. Sheng, “A survey on image semantic segmentation using deep learning techniques,” Comput. Mater. Continua, vol. 74, no. 1, pp. 1941–1957, 2023, https://dx.doi.org/10.32604/cmc.2023.032757.
  28. A. K. Singh, A. Rao, P. Chattopadhyay, R. Maurya, and L. Singh, “Effective plant disease diagnosis using vision transformer trained with leafy-generative adversarial network-generated images,” Expert Syst. Appl., vol. 254, Art. no. 124387, 2024, https://dx.doi.org/10.1016/j.eswa.2024.124387.
  29. Z. Xu, D. Wu, C. Yu, X. Chu, N. Sang, and C. Gao, “SCTNet: Single-branch CNN with transformer semantic information for real-time segmentation,” IEEE Access, vol. 12, pp. 98765–98784, 2024, https://dx.doi.org/10.1609/aaai.v38i6.28457.
  30. Y. Zhang, Z. Qiu, T. Yao, D. Liu, and T. Mei, “Fully convolutional adaptation networks for semantic segmentation,” arXiv preprint https://arxiv.org/abs/1804.08286, 2018, https://dx.doi.org/10.48550/arXiv.1804.08286.
  31. J. Wu, R. Fu, H. Fang, Y. Zhang, Y. Yang, H. Xiong, H. Liu, and Y. Xu, “MedSegDiff: Medical image segmentation with diffusion probabilistic model,” arXiv preprint https://arxiv.org/abs/2211.00611, 2023, https://dx.doi.org/10.48550/arXiv.2211.00611.
  32. H. Yu, H. Li, W. Yang, L. Yu, and G.-S. Xia, “Detecting line segments in motion-blurred images with events,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 46, no. 5, pp. 2866–2881, May 2024, https://dx.doi.org/10.1109/TPAMI.2023.3334877.
  33. Z. Huang, X. Wang, L. Huang, S. Huang, Y. Wei, and W. Liu, “CCNet: Criss-cross attention for semantic segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 9, pp. 5489–5503, 2022, https://dx.doi.org/10.1109/TPAMI.2020.3007032.
  34. L. Sun, Z. Chen, X. Zhu, Y. Bi, C. Liu, and Q. M. J. Wu, “TSNUNet: Two-stage nested U-network for salient object detection,” J. Vis. Commun. Image Represent., vol. 116, Art. no. 104737, 2026, https://dx.doi.org/10.1016/j.jvcir.2026.104737.
  35. M. A. Bhatti, Z. Zeeshan, S. M. Syam, U. Bhatti, A. Khan, Y. Ghadi, S. Alsenan, Y. Li, M. Asif, and T. Afzal, “Advanced plant disease segmentation in precision agriculture using optimal dimensionality reduction with fuzzy C-means clustering and deep learning,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., 2024, https://dx.doi.org/10.1109/JSTARS.2024.3437469.