Logo PTI
Polish Information Processing Society
Logo FedCSIS

Annals of Computer Science and Information Systems, Volume 8

Proceedings of the 2016 Federated Conference on Computer Science and Information Systems

Applying Mutation Testing for Assessing Test Suites Quality at Model Level

DOI: http://dx.doi.org/10.15439/2016F82

Citation: Proceedings of the 2016 Federated Conference on Computer Science and Information Systems, M. Ganzha, L. Maciaszek, M. Paprzycki (eds). ACSIS, Vol. 8, pages 1593–1596 ()

Full text

Abstract. Models are commonly used in software testing to select test suites. Application of mutation testing at a model level can contribute to reliable and early assessment of the quality of the test suites. It can also support selection of test suites achieving high fault detection rates. The main issue related to using mutation testing at the early development stage is to determine how reliably the quality of test suites can be measured at the model level. The research presented in this paper addresses this problem for object-oriented systems. It focuses on describing an experiment aiming at comparing results of applying mutation testing at a model level with results of applying this technique at an implementation level and presents and discusses the outcomes of the experiment. The paper presents also mutation operators applicable at the model level.

References

  1. B. Aichernig and P. Salas, “Test case generation by ocl mutation and constraint solving,” in 5th International Conference on Quality Software, Melbourne, 2005, pp. 64-71, http://dx.doi.org/10.1109/QSIC.2005.63.
  2. B. Aichernig, H. Brandl, E. Jobstl, W. Krenn, R. Schlick, S. Tiran, “Killing strategies for model-based mutation testing,” Software Testing, Verification and Reliability, vol. 25(8), 2015, pp. 716-748, http://dx.doi.org/10.1002/stvr.1522.
  3. F. Belli, C. J. Budnik, A. Hollmann, T. Tuglular, W. E. Wong, “Model-based mutation testing - Approach and case studies,” Science of Computer Programming, vol 120(1), 2016, pp. 25-48, http://dx.doi.org/10.1016/j.scico.2016.01.003.
  4. P. Black, V. Okun, Y. Yesha, “Mutation operators for specifications,” in 5th IEEE International Conference on Automated Software Engineering, Grenoble, 2000, pp. 81-88, http://dx.doi.org/10.1109/ASE.2000.873653.
  5. A. Brucker, M. Krieger, B. Wolff, “A specification-based test case generation method for uml/ocl,” in International Conference on Models in Software Engineering, Oslo, 2011, pp. 334-348, http://dx.doi.org/10.1007/978-3-642-21210-9-33.
  6. M. Daran and P. Thvenod-Fosse, “Software error analysis: A real case study involving real faults and mutations,” in ACM SIGSOFT international symposium on Software testing and analysis, Mission Beach, CA, 1996, pp. 158-177, http://dx.doi.org/0.1145/229000.226313.
  7. R. A. DeMillo, R. J. Lipton, F. G. Sayward, “Hints on test data selection: Help for the practicing programmer,” Computer, vol. 11(4), 1878, pp. 34-41, http://dx.doi.org/10.1109/C-M.1978.218136.
  8. A. Derezinska, “Object-oriented mutation to asses the quality of tests, in 29th Euromicro Conference, Belek, 2003, pp. 417-420, http://dx.doi.org/10.1109/EURMIC.2003.1231626.
  9. T. Dinh-Trong, S. Ghosh, R. France, B. Baudry, F. Fleurey, “A taxonomy of faults for uml designs,” in 2nd MoDeVa workshop - Model design and ValidationModel Design and Validation Workshop, Montego Bay, 2005.
  10. M. Gogolla, F. Buttner, M. Richters, “Use: A uml-based specification environment for validating uml and ocl,” Science of Computer Programming, vol. 69, 2007, pp. 27-34, http://dx.doi.org/10.1016/j.scico.2007.01.013.
  11. Y. Jia and M. Harman, “An analysis and survey of the development of mutation testing,” IEEE Transaction on Software Engineering, vol. 37(5), 2011, pp. 649-678, http://dx.doi.org/10.1109/TSE.2010.62.
  12. Ying Jiang, Shan-Shan Hou, Jinhui Shan, Lu Zhang, Bing Xie, “An approach to testing black-box components using contract-based muta- tion,” Journal of Software Engineering and Knowledge Engineering, vol. 18(1), 2008, pp. 93-117, http://dx.doi.org/10.1142/S0218194008003556
  13. R. Just, D. Jalali, L. Inozemtseva, M. D. Ernst, R. Holmes, G. Fraser, “Are Mutants a Valid Substitute for Real Faults in Software Testing?,” 22nd ACM SIGSOFT International Symposium on Foun- dations of Software Engineering, Hong Kong, 2014, pp. 654-665, http://dx.doi.org/10.1145/2635868.2635929.
  14. W. Krenn, R. Schlick, S. Tiran, B. Aichernig, E. Jobstl, H. Brandl, “MoMut::UML Model-Based Mutation Testing for UML,” in 8th Inter- national Conference on Software Testing, Verification and Validation, Graz, 2015, pp. 1-8, http://dx.doi.org/10.1109/ICST.2015.7102627
  15. Yu-Seung Ma, A. J. Offutt, Yong-Rae Kwon, “Mujava: An automated class mutation system,” Software Testing, Verification and Reliability, vol. 15(2), 2005, 97-133, http://dx.doi.org/10.1002/stvr.v15:2.
  16. R. Schlick, W. Herzner, E. Jobstl, “Fault-based generation of test cases from uml-models approach and some experiences,” in 30th International Conference on Computer safety, Reliability, and Security, Naples, 2001, pp. 270-283, http://dx.doi.org/10.1007/978-3-642-24270-0-20.
  17. J. Strug, “Classification of mutation operators applied to design models,” Key Engineering Materials, vol. 572, 2014, pp. 539-542, http://dx.doi.org/10.4028/www.scientific.net/KEM.572.539.
  18. J. Strug, “Mutation testing approach to negative testing,” Journal of Engineering, vol. 2016, 2016, http://dx.http://dx.doi.org/10.1155/2016/6589140.
  19. J. Strug, “Mutation testing approach to evaluation of design mod- els,” Key Engineering Materials, vol. 572, 2014, pp. 543-546, http://dx.doi.org/10.4028/www.scientific.net/KEM.572.543.
  20. J. Strug J and B. Strug, “Machine learning approach in mutation testing,” Testing Software and Systems, vol. 7641 of LNCS, Springer, 2012, pp. 200-214, http://dx.doi.org/10.1007/978-3-642-34691-0-15.
  21. J. Strug J and B. Strug, “Using structural similarity to classify tests in mutation testing,” Applied Mechanics and Materials, vol. 378, 2013, pp. 546-551, http://dx.doi.org/10.4028/www.scientific.net/AMM.378.546 .
  22. J. Strug and B. Strug, “Classifying mutants with decomposition kernel,” LNCS, Springer Berlin Heidelberg, vol. 9692, 2016, pp. 644-654, http://dx.doi.org/10.1007/978-3-319-39378-0-55
  23. M. Trakhtenbrot, “New mutations for evaluation of specification and implementation levels of adequacy in testing of statecharts models,” in Testing: Academic and Industrial Conference Practice and Research Techniques, Windsor, 2007, pp. 151-160, http://dx.doi.org/10.1109/TAIC.PART.2007.23.
  24. S. Weissleder and B. H. Schlingloff, “Quality of automatically generated test cases based on ocl expressions,” in 1st International Conference on Software Testing, Verification, and Validation, Los Alamitos, 2008, pp. 517-520.
  25. M-F. Wendland, “Abstractions on Test Design Techniques," in Federated Conference on Computer Science and Information Systems, Warsaw, 2014, pp. 1575-1584, http://dx.doi.org/10.15439/2014F316.