Logo PTI Logo FedCSIS

Communication Papers of the 18th Conference on Computer Science and Intelligence Systems

Annals of Computer Science and Information Systems, Volume 37

FAS-CT: FPGA-Based Acceleration System with Continuous Training

, , , , , ,

DOI: http://dx.doi.org/10.15439/2023F7674

Citation: Communication Papers of the 18th Conference on Computer Science and Intelligence Systems, M. Ganzha, L. Maciaszek, M. Paprzycki, D. Ślęzak (eds). ACSIS, Vol. 37, pages 131–139 ()

Full text

Abstract. This paper presents FAS-CT a novel approach to a distributed low-latency Deep Learning inference system based on a Field Programmable Gate Array (FPGA). The system incorporates continuous training capabilities based on Concept Drift Detection, where each model prediction is compared with the ground truth to detect a change in the data patterns that the model requires to adapt to. FAS-CT is formed by two main execution pipelines. First, is the Inference pipeline powered with Xilinx® Zynq® UltraScale+™ MPSoC FPGA and where low latency is the target. Second, is the Retraining pipeline its objective is to adapt the model as soon as possible when Concept Drift is detected. A complete characterization of FAS-CT is provided in this article using a neural network model and an experimental setup. The latency of the Prediction pipeline achieved was 5.79 ms. The total degradation of the model when continuous training is activated is 57\% in contrast to when is deactivated which is 1609\%. These results demonstrate that FAS-CT is suited for real-time Deep Learning inference and can be automatically adapted to evolving data environments.

References

  1. Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, May 2015. http://dx.doi.org/10.1038/nature14539
  2. B. Varghese and R. Buyya, “Next generation cloud computing: New trends and research directions,” Future Generation Computer Systems, vol. 79, pp. 849–861, Feb. 2018. http://dx.doi.org/10.1016/j.future.2017.09.020
  3. A. Arpteg, B. Brinne, L. Crnkovic-Friis, and J. Bosch, “Software Engineering Challenges of Deep Learning,” in 2018 44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), Aug. 2018. http://dx.doi.org/10.1109/SEAA.2018.00018 pp. 50–59.
  4. M. P. Véstias, R. P. Duarte, J. T. de Sousa, and H. C. Neto, “Moving Deep Learning to the Edge,” Algorithms, vol. 13, no. 5, p. 125, May 2020. http://dx.doi.org/10.3390/a13050125
  5. A. Gholami, S. Kim, Z. Dong, Z. Yao, M. W. Mahoney, and K. Keutzer, “A Survey of Quantization Methods for Efficient Neural Network Inference,” Jun. 2021, https://arxiv.org/abs/2103.13630 [cs].
  6. J. Lu, A. Liu, F. Dong, F. Gu, J. Gama, and G. Zhang, “Learning under Concept Drift: A Review,” IEEE Transactions on Knowledge and Data Engineering, vol. 31, no. 12, pp. 2346–2363, Dec. 2019. http://dx.doi.org/10.1109/TKDE.2018.2876857
  7. E. A. Castillo and A. Ahmadinia, “A Distributed Smart Camera System Based on an Edge Orchestration Architecture,” Journal of Circuits, Systems and Computers, vol. 30, no. 04, p. 2150059, Mar. 2021. http://dx.doi.org/10.1142/S0218126621500596
  8. L. D. Biasi, A. A. Citarella, M. Risi, and G. Tortora, “A Cloud Approach for Melanoma Detection Based on Deep Learning Networks,” IEEE Journal of Biomedical and Health Informatics, vol. 26, no. 3, pp. 962–972, Mar. 2022. http://dx.doi.org/10.1109/JBHI.2021.3113609
  9. H. K. Fatlawi and A. Kiss, “An Adaptive Classification Model for Predicting Epileptic Seizures Using Cloud Computing Service Architecture,” Applied Sciences, vol. 12, no. 7, p. 3408, Mar. 2022. http://dx.doi.org/10.3390/app12073408
  10. B. Cao, W. Wu, and J. Zhou, “LCFIL: A Loss Compensation Mechanism for Latest Data in Federated Incremental Learning,” in 2022 IEEE 19th International Conference on Mobile Ad Hoc and Smart Systems (MASS). Denver, CO, USA: IEEE, Oct. 2022. http://dx.doi.org/10.1109/MASS56207.2022.00055. ISBN 978-1-66547-180-0 pp. 332–338.
  11. K. Vipin, “ZyNet: Automating Deep Neural Network Implementation on Low-Cost Reconfigurable Edge Computing Platforms,” in 2019 International Conference on Field-Programmable Technology (ICFPT). Tianjin, China: IEEE, Dec. 2019. http://dx.doi.org/10.1109/ICFPT47387.2019.00058. ISBN 978-1-72812-943-3 pp. 323–326.
  12. R. Lozada, J. Ruiz, M. L. González, J. Sedano, J. R. Villar, A. M. García-Vico, and E. S. Skibinsky-Gitlin, “Performance/Resources Comparison of Hardware Implementations on Fully Connected Network Inference,” in Intelligent Data Engineering and Automated Learning – IDEAL 2022, ser. Lecture Notes in Computer Science. Cham: Springer International Publishing, 2022. http://dx.doi.org/10.1007/978-3-031-21753-1_34. ISBN 978-3-031-21753-1 pp. 348–358.
  13. Y. Zheng, B. He, and T. Li, “Research on the Lightweight Deployment Method of Integration of Training and Inference in Artificial Intelligence,” Applied Sciences, vol. 12, no. 13, p. 6616, Jun. 2022. http://dx.doi.org/10.3390/app12136616
  14. J. Violos, S. Tsanakas, T. Theodoropoulos, A. Leivadeas, K. Tserpes, and T. Varvarigou, “Hypertuning GRU Neural Networks for Edge Resource Usage Prediction,” in 2021 IEEE Symposium on Computers and Communications (ISCC). Athens, Greece: IEEE, Sep. 2021. http://dx.doi.org/10.1109/ISCC53001.2021.9631548. ISBN 978-1-66542-744-9 pp. 1–8.
  15. X. Wang, A. Khan, J. Wang, A. Gangopadhyay, C. Busart, and J. Freeman, “An edge–cloud integrated framework for flexible and dynamic stream analytics,” Future Generation Computer Systems, vol. 137, pp. 323–335, Dec. 2022. http://dx.doi.org/10.1016/j.future.2022.07.023
  16. M. L. González, R. Lozada, J. Ruiz, E. S. Skibinsky-Gitlin, A. M. García-Vico, J. Sedano, and J. R. Villar, “Exploring the implementation of LSTM inference on FPGA [Manuscript submitted for publication],” Proc. of the International Conference on Electrical, Computer, Communications and Mechatronics Engineering (ICECCME 2023), 2023.
  17. S. An and U. Y. Ogras, “MARS: mmWave-based Assistive Rehabilitation System for Smart Healthcare,” ACM Transactions on Embedded Computing Systems, vol. 20, no. 5s, pp. 72:1–72:22, Sep. 2021. http://dx.doi.org/10.1145/3477003
  18. A. Pardos, A. Menychtas, and I. Maglogiannis, “On unifying deep learning and edge computing for human motion analysis in exergames development,” Neural Computing and Applications, vol. 34, no. 2, pp. 951–967, Jan. 2022. http://dx.doi.org/10.1007/s00521-021-06181-6
  19. W. Jiang, X. Ye, R. Chen, F. Su, M. Lin, Y. Ma, Y. Zhu, and S. Huang, “Wearable on-device deep learning system for hand gesture recognition based on FPGA accelerator,” Mathematical Biosciences and Engineering, vol. 18, no. 1, pp. 132–153, 2021. http://dx.doi.org/10.3934/mbe.2021007
  20. B. C. Dos Santos Melício, G. Baranyi, Z. Gaál, S. Zidan, and A. Lorincz, “DeepRehab: Real Time Pose Estimation on the Edge for Knee Injury Rehabilitation,” in Artificial Neural Networks and Machine Learning – ICANN 2021, ser. Lecture Notes in Computer Science. Cham: Springer International Publishing, 2021. http://dx.doi.org/10.1007/978-3-030-86365-4_31. ISBN 978-3-030-86365-4 pp. 380–391.
  21. D. Merkel, “Docker: lightweight linux containers for consistent development and deployment,” Linux journal, vol. 2014, no. 239, p. 2, 2014.
  22. “Mqtt specification,” https://mqtt.org/mqtt-specification/.
  23. R. A. Light, “Mosquitto: server and client implementation of the mqtt protocol,” Journal of Open Source Software, vol. 2, no. 13, p. 265, 2017. http://dx.doi.org/10.21105/joss.00265
  24. B. Mishra, B. Mishra, and A. Kertesz, “Stress-testing mqtt brokers: A comparative analysis of performance measurements,” Energies, vol. 14, no. 18, 2021. http://dx.doi.org/10.3390/en14185817
  25. “grpc,” https://grpc.io/.
  26. J. Gama, P. Medas, G. Castillo, and P. Rodrigues, “Learning with Drift Detection,” in Advances in Artificial Intelligence – SBIA 2004, ser. Lecture Notes in Computer Science. Berlin, Heidelberg: Springer, 2004. http://dx.doi.org/10.1007/978-3-540-28645-5_29. ISBN 978-3-540-28645-5 pp. 286–295.
  27. “Mlflow - a platform for the machine learning lifecycle | mlflow,” https: //mlflow.org/.
  28. M. L. González, J. Sedano, A. M. García-Vico, and J. R. Víllar, “A Comparison of Techniques for Virtual Concept Drift Detection,” in 16th International Conference on Soft Computing Models in Industrial and Environmental Applications (SOCO 2021), ser. Advances in Intelligent Systems and Computing. Cham: Springer International Publishing, 2022. http://dx.doi.org/10.1007/978-3-030-87869-6_1. ISBN 978-3-030-87869-6 pp. 3–13.
  29. “Liambindle/mqtt-c: A portable mqtt c client for embedded systems and pcs alike.” https://github.com/LiamBindle/MQTT-C.
  30. M. Baena-Garcıa, R. Gavalda, and R. Morales-Bueno, “Early Drift Detection Method.”