Citation: Proceedings of the 2017 Federated Conference on Computer Science and Information Systems, M. Ganzha, L. Maciaszek, M. Paprzycki (eds). ACSIS, Vol. 11, pages 1315–1324 (2017)
Abstract. Mutation testing of object-oriented programs differs from that of standard (traditional) mutation operators in accordance to the number of generated mutants and ability of tests to kill mutants. Therefore, outcomes of cost reduction analysis cannot be directly transferred from a standard mutation to an object-oriented one. Mutant sampling is one of reduction methods of the number of generated and tested mutants. We proposed different mutant sampling criteria based on equivalence partitioning in respect to object-oriented program features. The criteria were experimentally evaluated for object-oriented and standard mutation operators applied in C# programs. We compared results using a quality metric, which combines mutation score accuracy with mutation cost factors. In result, class random sampling and operator random sampling are recommended for OO and standard mutation testing, accordingly. With a reasonable decline of result accuracy, the mutant sampling technique is easily applicable in comparison to other cost reduction techniques.
- Y. Jia and M. Harman, “An analysis and survey of the development of mutation testing,” IEEE Transactions on Software Engineering, vol. 37, no.5, pp. 649—678, Sept-Oct. 2011, https://dx.doi.org/10.1109/TSE.2010.62
- A. Derezińska and A. Szustek, “Object-Oriented testing capabilities and performance evaluation of the C# mutation system,” in Proc. CEE-SET 2009, Szmuc, T., Szpyrka, M., Zendulka, J. Eds., LNCS, vol. 7054, 2012, pp. 229–242, https://dx.doi.org/10.1007/978-3-642-28038-2_18
- M. P. Usaola and P. R. Mateo, “Mutation testing cost reduction techniques: a survey,” IEEE Software, vol. 27, no. 3, pp. 80–86, May-June 2010, https://dx.doi.org/10.1109/MS.2010.79
- A. Derezińska and M. Rudnik, “Quality evaluation of Object-Oriented and standard mutation operators applied to C# programs,” in Proc. TOOLS Europe 2012, C.A. Furia, S. Nanz Eds., LNCS, vol. 7304, Springer Berlin Heildelberg, 2012, pp. 42–57, https://dx.doi.org/10.1007/978-3-642-30561-0_5
- A. P. Mathur and W. E Wong, “Reducing the cost of mutation testing: an empirical study,” J. of Systems and Software, vol. 31, no. 3, pp. 185–196, Dec. 1995, http://dx.doi.org/10.1016/0164-1212(94)00098-0
- J. Offut, G. Rothermel, and C.Zapf, “An experimental evaluation of selective mutation,” in Proc. 15th International Conference on Software Engineering, IEEE Comp. Soc. Press, 1993, pp. 100–107, https://dx.doi.org/10.1109/ICSE.1993.346062
- A. Derezińska, “A quality estimation of mutation clustering in C# programs,” in New Results in Dependability and Computer Systems W. Zamojski et al. Eds., AISC vol. 224, Springer, Switzerland, 2013, pp.183-194, https://dx.doi.org/10.1007/978-3-319-00945-2_11
- A. T. Acree, “On Mutation,”, Ph.D. thesis, Georgia Institute of Technology, Atlanta, GA, 1980.
- T. A. Budd, “Mutation analysis of program test data,” Ph.D. thesis, Yale University, New Haven, CT, 1980.
- G. J. Myers, The Art of Software Testing, John Wiley & Sons, 1979, 3rd. ed 2011
- M. Kryszkiewicz, “Fast algorithm finding minima in monotonic Boolean functions,” Warsaw Univ. of Technology, ICS Res Rep. 42/93, 1993.
- A. Derezińska and A. Szustek, “Tool-supported mutation approach for verification of C# programs,” in Proc. International Conference on Dependability of Computer Systems, W. Zamojski, et al. Eds., pp. 261–268, 2008, https://dx.doi.org/10.1109/DepCoS-RELCOMEX.2008.51
- CREAM, http://galera.ii.pw.edu.pl/~adr/CREAM/
- E. F. Barbosa, J.C. Maldonado, and A.M.R. Vincenzi, “Toward the determination of sufficient mutant operators for C,” Softw. Test. Verif. and Reliab. vol. 11, pp. 113–136, June 2001, https://dx.doi.org/10.1002/stvr.226
- L Zhang, S-S., Hou, J-J. Hu, T., Xie, and H. Mei, “Is operator-based mutant selection superior to random mutant selection?” in Proc. 32nd International Conference on Software Engineering, ICSE 2010, 2010, pp. 435–444, https://dx.doi.org/10.1145/1806799.1806863
- J. Hu, N. Li, and J. Offutt, “An analysis of OO mutation operators,” in Proc. of 4th International Conference Software Testing Verification and Validation Workshops, 6th Workshop on Mutation Analysis, IEEE Comp. Soc., 2011, pp. 334–341, https://dx.doi.org/10.1109/ICSTW.2011.47
- K. Adamopoulos, M. Harman, and R. M. Hierons, “How to overcome the equivalent mutant problem and achieve tailored selective mutation using co-evolution,” GECCO’04, LNCS, vol. 3103, pp. 1338-1349. Springer, 2004, https://dx.doi.org/10.1007/978-3-540-24855-2_155
- M. Scholive, V. Beroulle, C. Robach, M. L. Flottes, and B. Rouzeyre, “Mutation sampling technique for the generation of structural test data,” in Proc. of the Conference on Design, Automation and Test in Europe, DATE’05, vol. 2, pp.1022 – 1023. IEEE Comp. Soc., 2005.
- C. Sun, F. Xue, H. Liu, and X. Zhang, “A path-aware approach to mutant reduction in mutation testing,” Information and Software Technology, vol. 81, pp. 65-81, Jun. 2017, https://dx.doi.org/10.1016/j.infsof.2016.02.006
- S. Segura, R. M. Hierons, D. Benavides, and A. Ruiz-Cortes, “Mutation testing on an object-oriented framework: An experience report,” Information and Software Technology, 53(10), pp. 1124—1136, Oct. 2011, https://dx.doi.org/10.1016/j.infsof.2011.03.006
- L. Zhang, M. Gligoric, D. Marinov, and S. Khurshid, “Operator-based and random mutant selection: better together,” in 28th IEEE/ACM Conference on Automated Software Engineering, Palo Alto, USA, 2013, pp. 92-102, https://dx.doi.org/10.1109/ASE.2013.6693070
- I. Bluemke and K. Kulesza, “Reduction of computational cost in mutation testing by sampling mutants,” in New Results in Dependability and Computer System, W. Zamojski at al. Eds., Springer, 2013, pp. 41-51, https://dx.doi.org/10.1007/978-3-319-07013-1_9
- Y.-S. Ma and S.-W. Kim, „Mutation testing cost reduction by clustering overlapped mutants,” J. of Systems and Software, vol. 115, pp. 18-30, May 2016, http://dx.doi.org/10.1016/j.jss.2016.01.007
- P. Delgado-Perez, S. Segura, and S. Media-Bulo, „Assessment of C++ object-oriented mutation operators: A selective mutation approach,” Softw TestVerif Reliab., 2017, https://dx.doi.org/10.1002/stvr.1630
- A. Estero-Botaro, F. Palomo-Lozano, I. Medina-Bulo, J. J. Dominguez-Jimenez, and A. Garcia-Dominguez, “Quality metrics for mutation testing with application to WS-BPL compositions,” Softw Test Verif Reliab, vol. 25, no. 5-7, pp. 536-571, Aug.-Nov. 2015, https://dx.doi.org/10.1002/stvr.1528
- M. Moghadam and S. Babamir, “Mutation score evaluation in terms of object-oriented metrics,” 4th International eConference on Computer and Knowledge Engineering (ICCKE), 2014, Mashhad, Iran 2014, pp. 775–780, https://dx.doi.org/10.1109/ICCKE.2014.6993419
- A. Derezińska and P. Trzpil, “Mutation testing process combined with Test-Driven Development in .NET Environment,” in Theory and Engineering of Complex Systems and Dependability, W. Zamojski et al. Eds., AISC vol. 365, Springer, pp. 131-140, 2015, https://dx.doi.org/10.1007/978-3-319-19216-1_13