Logo PTI
Polish Information Processing Society
Logo FedCSIS

Annals of Computer Science and Information Systems, Volume 11

Proceedings of the 2017 Federated Conference on Computer Science and Information Systems

Determining the significance of features with the use of Sobol' method in probabilistic neural network classification tasks

,

DOI: http://dx.doi.org/10.15439/2017F225

Citation: Proceedings of the 2017 Federated Conference on Computer Science and Information Systems, M. Ganzha, L. Maciaszek, M. Paprzycki (eds). ACSIS, Vol. 11, pages 39–48 ()

Full text

Abstract. In this article, the problem of determining the significance of data features is considered. For this purpose the algorithm is proposed, which with the use of Sobol' method, provides the global sensitivity indices. On the basis of these indices, the aggregated sensitivity coefficients are determined which are used to indicate significant features. Using such an information, the process of features' removal is performed. The results are verified by the probabilistic neural network in the classification of medical data sets by computing model's quality. We show that it is possible to point the least significant features which can be removed from the input space achieving higher classification performance.

References

  1. M. D. Morris, “Factorial sampling plans for preliminary computational experiments,” Technometrics, vol. 33, no. 2, pp. 161–174, 1991.
  2. I. M. Sobol, “Sensitivity estimates for nonlinear mathematical models,” Mathematical Modelling and Computational Experiments, vol. 1, no. 4, pp. 407–414, 1993.
  3. ——, “Global sensitivity indices for nonlinear mathematical models and their monte carlo estimates,” Mathematics and computers in simulation, vol. 55, no. 1, pp. 271–280, 2001.
  4. R. Cukier, C. Fortuin, K. E. Shuler, A. Petschek, and J. Schaibly, “Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. i theory,” The Journal of Chemical Physics, vol. 59, no. 8, pp. 3873–3878, 1973.
  5. A. Saltelli, S. Tarantola, and K.-S. Chan, “A quantitative model-independent method for global sensitivity analysis of model output,” Technometrics, vol. 41, no. 1, pp. 39–56, 1999.
  6. M. Fesanghary, E. Damangir, and I. Soleimani, “Design optimization of shell and tube heat exchangers using global sensitivity analysis and harmony search algorithm,” Applied Thermal Engineering, vol. 29, no. 5, pp. 1026–1031, 2009.
  7. E. Fock, “Global sensitivity analysis approach for input selection and system identification purposes–a new framework for feedforward neural networks,” IEEE Transactions on Neural Networks and Learning Sys tems, vol. 25, no. 8, pp. 1484–1495, 2014.
  8. A. Cosenza, G. Mannina, P. A. Vanrolleghem, and M. B. Neumann, “Global sensitivity analysis in wastewater applications: A comprehensive comparison of different methods,” Environmental modelling & software, vol. 49, pp. 40–52, 2013.
  9. P. A. Kowalski and M. Kusy, “Sensitivity analysis for probabilistic neural network structure reduction,” IEEE Transactions on Neural Networks and Learning Systems, vol. PP, no. 99, pp. 1–14, 2017. http://dx.doi.org/10.1109/TNNLS.2017.2688482
  10. I. Kononenko, “Estimating attributes: analysis and extensions of relief,” in Machine Learning: ECML-94. Springer, 1994, pp. 171–182.
  11. L. Breiman, “Random forests,” Machine learning, vol. 45, no. 1, pp. 5–32, 2001.
  12. M. Szczuka and D. Slezak, “Feedforward neural networks for compound signals,” Theoretical Computer Science, vol. 412, no. 42, pp. 5960–5973, 2011.
  13. A. Janusz and D. Slezak, “Utilization of attribute clustering methods for scalable computation of reducts from high-dimensional data,” in Federated Conference on Computer Science and Information Systems (FedCSIS), 2012, pp. 295–302.
  14. M. Lichman, “UCI machine learning repository,” 2013. [Online]. Available: http://archive.ics.uci.edu/ml
  15. A. Saltelli, S. Tarantola, and K.-S. Chan, “A quantitative model-independent method for global sensitivity analysis of model output,” Technometrics, vol. 41, no. 1, pp. 39–56, 1999.
  16. D. F. Specht, “Probabilistic neural networks,” Neural Networks, vol. 3, no. 1, pp. 109–118, 1990.
  17. ——, “Probabilistic neural networks and the polynomial adaline as complementary techniques for classification,” Neural Networks, IEEE Transactions on, vol. 1, no. 1, pp. 111–121, Mar 1990. http://dx.doi.org/10.1109/72.80210
  18. R. Folland, E. Hines, R. Dutta, P. Boilot, and D. Morgan, “Comparison of neural network predictors in the classification of tracheal–bronchial breath sounds by respiratory auscultation,” Artificial intelligence in medicine, vol. 31, no. 3, pp. 211–220, 2004.
  19. D. Mantzaris, G. Anastassopoulos, and A. Adamopoulos, “Genetic algorithm pruning of probabilistic neural networks in medical disease estimation,” Neural Networks, vol. 24, no. 8, pp. 831–835, 2011.
  20. M. Kusy and R. Zajdel, “Application of reinforcement learning algorithms for the adaptive computation of the smoothing parameter for probabilistic neural network,” Neural Networks and Learning Systems, IEEE Transactions on, vol. 26, no. 9, pp. 2163–2175, 2015.
  21. M. Kusy and R. Zajdel, “Probabilistic neural network training procedure based on q(0)– learning algorithm in medical data classification,” Applied Intelligence, vol. 41, no. 3, pp. 837–854, 2014.
  22. Y. Chtioui, S. Panigrahi, and R. Marsh, “Conjugate gradient and approximate newton methods for an optimal probabilistic neural network for food color classification,” Optical Engineering, vol. 37, no. 11, pp. 3015–3023, 1998.
  23. S. Ramakrishnan and S. Selvan, “Image texture classification using wavelet based curve fitting and probabilistic neural network,” International Journal of Imaging Systems and Technology, vol. 17, no. 4, pp. 266–275, 2007.
  24. X.-B. Wen, H. Zhang, X.-Q. Xu, and J.-J. Quan, “A new watermarking approach based on probabilistic neural network in wavelet domain,” Soft Computing, vol. 13, no. 4, pp. 355–360, 2009.
  25. S. Venkatesh and S. Gopal, “Orthogonal least square center selection technique–a robust scheme for multiple source partial discharge pattern recognition using radial basis probabilistic neural network,” Expert Systems with Applications, vol. 38, no. 7, pp. 8978–8989, 2011.
  26. P. A. Kowalski and P. Kulczycki, “Data sample reduction for classification of interval information using neural network sensitivity analysis,” in Artificial Intelligence: Methodology, Systems, and Applications, ser. Lecture Notes in Computer Science, D. Dicheva and D. Dochev, Eds. Springer Berlin Heidelberg, 2010, vol. 6304, pp. 271–272.
  27. P. A. Kowalski and P. Kulczycki, “Interval probabilistic neural network,” Neural Computing and Applications, vol. 28, no. 4, pp. 817–834, 2017. doi: 10.1007/s00521-015-2109-3. [Online]. Available: http://dx.doi.org/10.1007/s00521-015-2109-3
  28. K. Elenius and H. G. Tråvén, “Multi-layer perceptrons and probabilistic neural networks for phoneme recognition.” in EUROSPEECH, 1993.
  29. T. P. Tran, T. T. S. Nguyen, P. Tsai, and X. Kong, “Bspnn: boosted subspace probabilistic neural network for email security,” Artificial Intelligence Review, vol. 35, no. 4, pp. 369–382, 2011.
  30. T. P. Tran, L. Cao, D. Tran, and C. D. Nguyen, “Novel intrusion detection using probabilistic neural network and adaptive boosting,” International Journal of Computer Science and Information Security, vol. 6, no. 1, pp. 83–91, 2009.
  31. P. A. Kowalski and P. Kulczycki, “A complete algorithm for the reduction of pattern data in the classification of interval information,” International Journal of Computational Methods, vol. 13, no. 03, p. 1650018, 2016. http://dx.doi.org/10.1142/S0219876216500183
  32. M. P. Wand and M. C. Jones, Kernel smoothing. Crc Press, 1994.
  33. B. W. Silverman, Density estimation for statistics and data analysis. CRC press, 1986, vol. 26.
  34. J. Zhang, “Selecting typical instances in instance-based learning,” in Proceedings of the Ninth International Workshop on Machine Learning, ser. ML92. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 1992. ISBN 1-5586-247-X pp. 470–479. [Online]. Available: http://dl.acm.org/citation.cfm?id=141975.142091
  35. G. Brown, Diversity in neural network ensembles. University of Birmingham, 2004.
  36. M. A. Little, P. E. McSharry, E. J. Hunter, J. Spielman, and L. O. Ramig, “Suitability of dysphonia measurements for telemonitoring of parkinson’s disease,” IEEE Transactions on Biomedical Engineering, vol. 56, no. 4, pp. 1015–1022, April 2009. http://dx.doi.org/10.1109/TBME.2008.2005954