Logo PTI Logo FedCSIS

Communication Papers of the 18th Conference on Computer Science and Intelligence Systems

Annals of Computer Science and Information Systems, Volume 37

Developing an interval method for training denoising autoencoders by bounding the noise

DOI: http://dx.doi.org/10.15439/2023F865

Citation: Communication Papers of the 18th Conference on Computer Science and Intelligence Systems, M. Ganzha, L. Maciaszek, M. Paprzycki, D. Ślęzak (eds). ACSIS, Vol. 37, pages 179–186 ()

Full text

Abstract. This paper discusses prospects of using interval methods to training denoising autoencoders. Advantages and disadvantages of using the interval approach are discussed. It is proposed to formulate the problem of training the proper neural network as a constraint-satisfaction, and not optimization, problem. Pros and cons of this approach are considered. Preliminary numerical experiments are also presented.

References

  1. S. P. Adam, D. A. Karras, G. D. Magoulas, and M. N. Vrahatis, “Solving the linear interval tolerance problem for weight initialization of neural networks,” Neural Networks, vol. 54, pp. 17–37, 2014.
  2. S. Huang, Z. Ma, S. Yu, and Y. Han, “New discrete-time zeroing neural network for solving time-variant underdetermined nonlinear systems under bound constraint,” Neurocomputing, vol. 487, pp. 214–227, 2022.
  3. M. Beheshti, A. Berrached, A. de Korvin, C. Hu, and O. Sirisaengtaksin, “On interval weighted three-layer neural networks,” in Simulation Symposium, 1998. Proceedings. 31st Annual. IEEE, 1998, pp. 188–194.
  4. B. J. Kubica, P. Hoser, and A. Wiliński, “Interval methods for seeking fixed points of recurrent neural networks,” in International Conference on Computational Science. Springer, 2020. http://dx.doi.org/10.1007/978-3-030-50420-5_30 pp. 414–423.
  5. A. Rauh and E. Auer, “Interval extension of neural network models for the electrochemical behavior of high-temperature fuel cells,” Frontiers in Control Engineering, vol. 3, 2022. http://dx.doi.org/10.3389/fcteg.2022.785123
  6. P. V. Saraev, “Numerical methods of interval analysis in learning neural network,” Automation and Remote Control, vol. 73, no. 11, pp. 1865–1876, 2012.
  7. E. de Weerdt, Q. Chu, and J. Mulder, “Neural network output optimization using interval analysis,” IEEE Transactions on Neural Networks, vol. 20, no. 4, pp. 638–653, 2009.
  8. L. V. Utkin, A. V. Podolskaja, and V. S. Zaborovsky, “A robust interval autoencoder,” in 2017 International Conference on Control, Artificial Intelligence, Robotics & Optimization (ICCAIRO). IEEE, 2017, pp. 115–120.
  9. I. Goodfellow, Y. Bengio, and A. Courville, Deep learning. MIT press, 2016.
  10. A. Géron, Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow: Concepts, tools, and techniques to build intelligent systems. O’Reilly Media, 2019.
  11. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning internal representations by error propagation,” Institute for Cognitive Science, University of California, San Diego, Tech. Rep. 8506, 1985.
  12. ——, “Learning representations by back-propagating errors,” Nature, vol. 323, no. 6088, pp. 533–536, 1986.
  13. G. E. Hinton, “Connectionist learning procedures,” in Machine learning, 1990, pp. 555–610.
  14. O. Kosheleva and V. Kreinovich, “Why deep learning methods use KL divergence instead of least squares: A possible pedagogical explanation,” University of Texas at El Paso, Tech. Rep. UTEP-CS-17-95, 2017.
  15. X. Li, S. Feng, N. Hou, R. Wang, H. Li, M. ZGao, and S. Li, “Surface microseismic data denoising based on sparse autoencoder and Kalman filter,” Systems Science & Control Engineering, vol. 10, no. 1, pp. 616–628, 2022.
  16. M. Chen, K. Weinberger, F. Sha, and YoshuaBengio, “Marginalized denoising auto-encoders for nonlinear representations,” in International conference on machine learning. PMLR, 2014, pp. 1476–1484.
  17. D. P. Kingma and M. Welling, “Auto-encoding variational Bayes,” arXiv preprint https://arxiv.org/abs/1312.6114, 2013.
  18. E. Hansen and W. Walster, Global Optimization Using Interval Analysis. Marcel Dekker, New York, 2004.
  19. L. Jaulin, M. Kieffer, O. Didrit, and E. Walter, Applied Interval Analysis. Springer, London, 2001.
  20. R. B. Kearfott, Rigorous Global Search: Continuous Problems. Kluwer, Dordrecht, 1996.
  21. U. Kulisch, Computer Arithmetic and Validity – Theory, Implementation and Applications. De Gruyter, Berlin, New York, 2008.
  22. R. E. Moore, R. B. Kearfott, and M. J. Cloud, Introduction to Interval Analysis. SIAM, Philadelphia, 2009.
  23. S. P. Shary, Finite-dimensional Interval Analysis. Institute of Computational Technologies, Sibirian Branch of Russian Academy of Science, Novosibirsk, 2013.
  24. B. J. Kubica, Interval methods for solving nonlinear constraint satisfaction, optimization and similar problems: From inequalities systems to game solutions, ser. Studies in Computational Intelligence. Springer, 2019, vol. 805.
  25. ——, “Interval methods for solving underdetermined nonlinear equations systems,” Reliable Computing, vol. 15, pp. 207–217, 2011, proceedings of SCAN 2008.
  26. ——, “Presentation of a highly tuned multithreaded interval solver for underdetermined and well-determined nonlinear systems,” Numerical Algorithms, vol. 70, no. 4, pp. 929–963, 2015. http://dx.doi.org/10.1007/s11075-015-9980-y
  27. ——, “Parallelization of a bound-consistency enforcing procedure and its application in solving nonlinear systems,” Journal of Parallel and Distributed Computing, vol. 107, pp. 57–66, 2017. http://dx.doi.org/10.1016/j.jpdc.2017.03.009
  28. ——, “Role of hull-consistency in the HIBA_USNE multithreaded solver for nonlinear systems,” Lecture Notes in Computer Science, vol. 10778, pp. 381–390, 2018. http://dx.doi.org/10.1007/978-3-319-78054-2_36 Proceedings of PPAM 2017.
  29. “HIBA_USNE, C++ library,” https://gitlab.com/bkubica/hiba_usne, 2023.
  30. “HIBA_TANN, C++ library,” https://gitlab.com/bkubica/hiba_tann, 2023.
  31. “Intel TBB,” https://github.com/oneapi-src/oneTBB, 2023.
  32. “OpenBLAS library,” https://www.openblas.net, 2023.
  33. “Iris Species dataset,” https://www.kaggle.com/datasets/uciml/iris?resource=download, 2023.
  34. B. J. Kubica, “Preliminary experiments with an interval Model-Predictive-Control solver,” Lecture Notes in Computer Science, vol. 9574, pp. 464–473, 2016, PPAM 2015 Proceedings.
  35. S. P. Shary, “Algebraic approach to the interval linear static identification, tolerance, and control problems, or one more application of Kaucher arithmetic,” Reliable Computing, vol. 2, no. 1, pp. 3–33, 1996.