Logo PTI Logo FedCSIS

Proceedings of the 17th Conference on Computer Science and Intelligence Systems

Annals of Computer Science and Information Systems, Volume 30

An Automated Algorithm for Fruit Image Dataset Building

DOI: http://dx.doi.org/10.15439/2022F58

Citation: Proceedings of the 17th Conference on Computer Science and Intelligence Systems, M. Ganzha, L. Maciaszek, M. Paprzycki, D. Ślęzak (eds). ACSIS, Vol. 30, pages 103–107 ()

Full text

Abstract. This paper introduces a new algorithm that utilises images from the Fruits-360 dataset, superimposes them of various backgrounds and creates associated annotation files with the coordinates of the bounding boxes surrounding the fruits. The main challenge of this task was accounting for the variations in lighting and occlusion associated with outdoor locations. The utility and application of such an algorithm is to reduce the need to collect real world data for training, accelerating the speed at which new models are developed. Using 3000 images generated by this algorithm we train a single shot multibox detector (SSD) to study the feasibility of using generated data during training. We then test the trained model on 70 real world images of apples (65 images of apples on trees and 5 images of apples in bunches) and obtain a mean average precision of 0.750 and we compare our results with those obtained by other state of the art models.

References

  1. N. C. Eli-Chukwu, “Applications of artificial intelligence in agriculture: A review,” Engineering, Technology & Applied Science Research, vol. 9, no. 4, pp. 4377–4383, 2019. http://dx.doi.org/10.48084/etasr.2756
  2. O. Apolo-Apolo, J. Martı́nez-Guanter, G. Egea, P. Raja, and M. Pérez-Ruiz, “Deep learning techniques for estimation of the yield and size of citrus fruits using a uav,” European Journal of Agronomy, vol. 115, p. 126030, 2020. http://dx.doi.org/10.1016/j.eja.2020.126030
  3. S. Bargoti and J. P. Underwood, “Image segmentation for fruit detection and yield estimation in apple orchards,” Journal of Field Robotics, vol. 34, no. 6, pp. 1039–1060, 2017. http://dx.doi.org/10.48550/arXiv.1610.08120
  4. H. Kang and C. Chen, “Fast implementation of real-time fruit detection in apple orchards using deep learning,” Computers and Electronics in Agriculture, vol. 168, p. 105108, 2020. http://dx.doi.org/10.1016/j.compag.2019.105108. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0168169919314395
  5. H. Yu, S. Song, S. Ma, and R. O. Sinnott, “Estimating fruit crop yield through deep learning,” in Proceedings of the 6th IEEE/ACM International Conference on Big Data Computing, Applications and Technologies, 2019. http://dx.doi.org/10.1145/3365109.3368766 pp. 145–148.
  6. R. Kestur, A. Meduri, and O. Narasipura, “Mangonet: A deep semantic segmentation architecture for a method to detect and count mangoes in an open orchard,” Engineering Applications of Artificial Intelligence, vol. 77, pp. 59 – 69, 2019. http://dx.doi.org/10.1016/j.engappai.2018.09.011. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0952197618301970
  7. A. Koirala, K. B. Walsh, Z. Wang, and C. McCarthy, “Deep learning–method overview and review of use for fruit detection and yield estimation,” Computers and Electronics in Agriculture, vol. 162, pp. 219–234, 2019. http://dx.doi.org/10.1016/j.compag.2019.04.017
  8. M. Rahnemoonfar and C. Sheppard, “Real-time yield estimation based on deep learning,” in Autonomous Air and Ground Sensing Systems for Agricultural Optimization and Phenotyping II, J. A. Thomasson, M. McKee, and R. J. Moorhead, Eds., vol. 10218, International Society for Optics and Photonics. SPIE, 2017. http://dx.doi.org/10.1117/12.2263097 pp. 59 – 65.
  9. H. Mureşan and M. Oltean, “Fruit recognition from images using deep learning,” Acta Universitatis Sapientiae, Informatica, vol. 10, no. 1, pp. 26 – 42, 2018. http://dx.doi.org/10.48550/arXiv.1712.00580. [Online]. Available: https://content.sciendo.com/view/journals/ausi/10/1/article-p26.xml
  10. M. Oltean and H. Muresan, “Fruits 360 dataset on github,” 2017, [Online; accessed 16.09.2021]. [Online]. Available: https://github.com/Horea94/Fruit-Images-Dataset
  11. W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. E. Reed, C. Fu, and A. C. Berg, “SSD: single shot multibox detector,” CoRR, vol. abs/1512.02325, 2015. http://dx.doi.org/10.1007/978-3-319-46448-0 2. [Online]. Available: http://arxiv.org/abs/1512.02325
  12. C. Szegedy, S. Ioffe, and V. Vanhoucke, “Inception-v4, inceptionresnet and the impact of residual connections on learning,” CoRR, vol. abs/1602.07261, 2016. http://dx.doi.org/10.48550/arXiv.1602.07261. [Online]. Available: http://arxiv.org/abs/1602.07261
  13. H. Mureşan, A. Călin, and A. Coroiu, “Overview of recent deep learning methods applied in fruit counting for yield estimation,” Studia Universitatis Babeş-Bolyai Informatica, vol. 65, no. 2, pp. 50–65, 2020. http://dx.doi.org/10.24193/subbi.2020.2.04. [Online]. Available: http://www.cs.ubbcluj.ro/~studia-i/journal/journal/article/view/58
  14. J. Fourie, J. Hsaio, and A. Werner, “Crop yield estimation using deep learning,” in 7th Asian-Australasian Conference on Precision Agriculture, 2017. http://dx.doi.org/10.5281/zenodo.893710 pp. 1–10.
  15. S. W. Chen, S. S. Shivakumar, S. Dcunha, J. Das, E. Okon, C. Qu, C. J. Taylor, and V. Kumar, “Counting apples and oranges with deep learning: A data-driven approach,” IEEE Robotics and Automation Letters, vol. 2, no. 2, pp. 781–788, 2017. http://dx.doi.org/10.1109/LRA.2017.2651944
  16. Q. Xiang, X. Wang, R. Li, G. Zhang, J. Lai, and Q. Hu, “Fruit image classification based on mobilenetv2 with transfer learning technique,” in Proceedings of the 3rd International Conference on Computer Science and Application Engineering, 2019. http://dx.doi.org/10.1145/3331453.3361658 pp. 1–7.
  17. M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018. http://dx.doi.org/10.48550/arXiv.1801.04381 pp. 4510–4520.
  18. R. Siddiqi, “Effectiveness of transfer learning and fine tuning in automated fruit image classification,” in Proceedings of the 2019 3rd International Conference on Deep Learning Technologies, 2019. http://dx.doi.org/10.1145/3342999.3343002 pp. 91–100.
  19. P. Jiang, Y. Chen, B. Liu, D. He, and C. Liang, “Real-time detection of apple leaf diseases using deep learning approach based on improved convolutional neural networks,” IEEE Access, vol. 7, pp. 59 069–59 080, 2019. http://dx.doi.org/10.1109/ACCESS.2019.2914929
  20. P. Ferrari, “Ssd keras implementation,” 2018, [Online; accessed 16.09.2021]. [Online]. Available: https://github.com/pierluigiferrari/ssd keras
  21. TensorFlow, “Tensorflow,” 2015, [Online; accessed 16.09.2021]. [Online]. Available: https://www.tensorflow.org
  22. K. Wada, “labelme,” 2010, [Online; accessed 16.09.2021]. [Online]. Available: https://github.com/wkentaro/labelme
  23. WikimediaCommons, “Wikimedia commons - freely usable media files,” 2004, [Online; accessed 16.09.2021]. [Online]. Available: https://commons.wikimedia.org/wiki/Main Page