Logo PTI Logo FedCSIS

Proceedings of the 16th Conference on Computer Science and Intelligence Systems

Annals of Computer Science and Information Systems, Volume 25

Combinatorial Testing of Context Aware Android Applications

, , , ,

DOI: http://dx.doi.org/10.15439/2021F003

Citation: Proceedings of the 16th Conference on Computer Science and Intelligence Systems, M. Ganzha, L. Maciaszek, M. Paprzycki, D. Ślęzak (eds). ACSIS, Vol. 25, pages 17–26 ()

Full text

Abstract. Mobile devices such as smart phones and smart watches utilize apps that run in context aware environments and must respond to context changes such as changes in network connectivity, battery level, screen orientation, and more. The large number of GUI events and context events often complicate the testing process. This work expands the AutoDroid tool to automatically generate tests that are guided by PairwiseInterleaved coverage of GUI event and context event sequences. We systematically weave context and GUI events into testing using the pairwise interleaved algorithm. The results show that the pairwise interleaved algorithm achieves up to five times higher code coverage compared to a technique that generates test suites in a single predefined context (without interleaving context and GUI events), a technique that changes the context at the beginning of each test case (without interleaving context and GUI events), and Monkey-Context-GUI (which randomly chooses context and GUI events). Future work will expand this strategy to include more context variables and test emerging technologies such as IoT and autonomous vehicles.

References

  1. Uber, “Uber- earn money by driving or get a ride now,” 2019, retrieved Feb 25, 2020 from https://www.uber.com.
  2. C.-W. You, M. Montes-de Oca, T. J. Bao, N. D. Lane, H. Lu, G. Cardone, L. Torresani, and A. T. Campbell, “Carsafe: a driver safety app that detects dangerous driving behavior using dual-cameras on smartphones,” in Proceedings of the 2012 ACM Conference on Ubiquitous Computing, 2012, pp. 671–672.
  3. D. Adamo, D. Nurmuradov, S. Piparia, and R. Bryce, “Combinatorial-based event sequence testing of android applications,” Information and Software Technology, vol. 99, pp. 98–117, 2018.
  4. D. Amalfitano, A. R. Fasolino, P. Tramontana, B. D. Ta, and A. M. Memon, “MobiGUITAR: Automated model-based testing of mobile apps,” IEEE Software, vol. 32, no. 5, pp. 53–59, 2015.
  5. A. Machiry, R. Tahiliani, and M. Naik, “Dynodroid: An input generation system for android apps,” in Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering. ACM, 2013, pp. 224–234.
  6. I. C. Morgado and A. C. Paiva, “The iMPAcT tool: Testing UI patterns on mobile applications,” in 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 2015, pp. 876–881.
  7. N. Mirzaei, J. Garcia, H. Bagheri, A. Sadeghi, and S. Malek, “Reducing combinatorics in GUI testing of android applications,” in Proceedings of the 38th International Conference on Software Engineering. ACM, 2016, pp. 559–570.
  8. D. Amalfitano, N. Amatucci, A. M. Memon, P. Tramontana, and A. R. Fasolino, “A general framework for comparing automatic testing techniques of android mobile apps,” Journal of Systems and Software, vol. 125, pp. 322–343, 2017.
  9. K. Mao, M. Harman, and Y. Jia, “Sapienz: Multi-objective automated testing for android applications,” in Proceedings of the 25th International Symposium on Software Testing and Analysis. ACM, 2016, pp. 94–105.
  10. R. Michaels, D. Adamo, and R. Bryce, “Combinatorial-based event sequences for reduction of android test suites,” in 2020 10th Annual Computing and Communication Workshop and Conference (CCWC), 2020, pp. 0598–0605.
  11. R. Michaels, M. K. Khan, and R. Bryce, “Mobile test suite generation via combinatorial sequences,” in ITNG 2021 18th International Conference on Information Technology-New Generations, S. Latifi, Ed. Cham: Springer International Publishing, 2021, pp. 273–279.
  12. S. Piparia, M. K. Khan, and R. Bryce, “Discovery of real world context event patterns for smartphone devices using conditional random fields,” in ITNG 2021 18th International Conference on Information Technology-New Generations, S. Latifi, Ed. Cham: Springer International Publishing, 2021, pp. 221–227.
  13. D. Amalfitano, N. Amatucci, A. R. Fasolino, and P. Tramontana, “A Conceptual Framework for the Comparison of Fully Automated GUI Testing Techniques,” in 2015 30th IEEE/ACM International Conference on Automated Software Engineering Workshop (ASEW), 2015, pp. 50–57.
  14. D. Amalfitano, A. R. Fasolino, P. Tramontana, and N. Amatucci, “Considering context events in event-based testing of mobile applications,” in 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops (ICSTW). IEEE, 2013, pp. 126–133.
  15. T. Griebe and V. Gruhn, “A model-based approach to test automation for context-aware mobile applications,” in Proceedings of the 29th Annual ACM Symposium on Applied Computing. ACM, 2014, pp. 420–427.
  16. Z. Liu, X. Gao, and X. Long, “Adaptive random testing of mobile application,” in 2010 2nd International Conference on Computer Engineering and Technology (ICCET), vol. 2. IEEE, 2010, pp. V2–297.
  17. K. Song, A. R. Han, S. Jeong, and S. Cha, “Generating various contexts from permissions for testing android applications,” in 27th International Conference on Software Engineering and Knowledge Engineering (SEKE), 2015, pp. 87–92.
  18. C. Q. Adamsen, G. Mezzetti, and A. Møller, “Systematic execution of android test suites in adverse conditions,” in Proceedings of the 2015 International Symposium on Software Testing and Analysis. ACM, 2015, pp. 83–93.
  19. T. A. Majchrzak and M. Schulte, “Context-dependent testing of applications for mobile devices,” Open Journal of Web Technologies (OJWT), vol. 2, no. 1, pp. 27–39, 2015.
  20. T. Griebe, M. Hesenius, and V. Gruhn, “Towards automated UI-tests for sensor-based mobile applications,” in International Conference on Intelligent Software Methodologies, Tools, and Techniques. Springer, 2015, pp. 3–17.
  21. Uber, “Calabash-android,” 2019, retrieved Feb 25, 2020 from https://github.com/calabash.
  22. K. Moran, M. Linares-Vásquez, C. Bernal-Cárdenas, C. Vendome, and D. Poshyvanyk, “Automatically discovering, reporting and reproducing android application crashes,” in 2016 IEEE International Conference on Software Testing, Verification and Validation (ICST). IEEE, 2016, pp. 33–44.
  23. C.-J. M. Liang, N. D. Lane, N. Brouwers, L. Zhang, B. F. Karlsson, H. Liu, Y. Liu, J. Tang, X. Shan, R. Chandra, and F. Zhao, “Caiipa: Automated large-scale mobile app testing through contextual fuzzing,” in Proceedings of the 20th Annual International Conference on Mobile Computing and Networking, ser. MobiCom ’14. New York, NY, USA: ACM, 2014, pp. 519–530. [Online]. Available: http://doi.acm.org/10.1145/2639108.2639131
  24. G. Hu, X. Yuan, Y. Tang, and J. Yang, “Efficiently, effectively detecting mobile app bugs with appdoctor,” in Proceedings of the Ninth European Conference on Computer Systems, 2014, pp. 1–15.
  25. M. Gómez, R. Rouvoy, B. Adams, and L. Seinturier, “Reproducing context-sensitive crashes of mobile apps using crowdsourced monitoring,” in Proceedings of the International Conference on Mobile Software Engineering and Systems, ser. MOBILESoft ’16. New York, NY, USA: ACM, 2016, pp. 88–99. [Online]. Available: http://doi.acm.org/10.1145/2897073.2897088
  26. A. S. Ami, M. M. Hasan, M. R. Rahman, and K. Sakib, “Mobicomonkey - context testing of android apps,” in 2018 IEEE/ACM 5th International Conference on Mobile Software Engineering and Systems (MOBILE-Soft), May 2018, pp. 76–79.
  27. R. C. Bryce and C. J. Colbourn, “Prioritized interaction testing for pair-wise coverage with seeding and constraints,” Information and Software Technology, vol. 48, no. 10, pp. 960–970, 2006.
  28. H. Tsuyoshi. (2021) Pairwise combinatosila package. https://github.com/thombashi/allpairspy. (Accessed: 25-05-2021).
  29. F-Droid, “F-droid: Free and open source android app repository,” http: //f-droid.org, 2017, (Accessed: 26-02-2021).
  30. Mountainminds GmbH, “EclEmma: JaCoCo java code coverage library,” http://www.eclemma.org/jacoco/, 2017, (Accessed: 26-02-2021).
  31. Google, “UI/application exerciser monkey,” 2017, retrieved May 10, 2021 from https://developer.android.com/studio/test/monkey.html.
  32. A. M. Memon, “Developing testing techniques for event-driven pervasive computing applications,” in Proceedings of The OOPSLA 2004 workshop on Building Software for Pervasive Computing (BSPC 2004), 2004.
  33. A. Memon, “An event-flow model of gui-based applications for testing,” Software testing, verification and reliability, vol. 17, no. 3, pp. 137–157, 2007.