Logo PTI Logo FedCSIS

Proceedings of the 21st Conference on Computer Science and Intelligence Systems (FedCSIS)

Annals of Computer Science and Information Systems, Volume 47

Regression Exception Rules

, ,

DOI: http://dx.doi.org/10.15439/2026F8053

Citation: Dawid Macha, ,

Full text

Abstract. This paper introduces a novel framework for discovering exception rules in regression problems. While exception rules have been widely studied in classification and association analysis, their application to continuous outcomes has remained largely unexplored. To address this gap, we propose a formal definition of regression exception rule. We also present a rule induction framework based on sequential covering for discovering such patterns. Experiments conducted on 48 benchmark datasets show that the proposed approach is able to identify rare and interpretable exceptions while maintaining predictive performance comparable to other interpretable regression models. The results demonstrate that regression exception rules can effectively support knowledge discovery by revealing non-obvious relationships in continuous data.

References

  1. Q. An, S. Rahman, J. Zhou, J.J. Kang. A Comprehensive Review on Machine Learning in Healthcare Industry: Classification, Restrictions, Opportunities and Challenges. Sensors, 23(9):4178, 2023. https://doi. org/10.3390/s23094178
  2. R. Pugliese, S. Regondi, R. Marini. Machine learning-based approach: global trends, research directions, and regulatory standpoints. Data Science and Management, 4:19–29, 2021. https://doi.org/10.1016/j.dsm. 2021.12.002
  3. Maszczyk, C., Kozielski, M., Sikora, M. (2022). Rule-Based Approximation of Black-Box Classifiers for Tabular Data to Generate Global and Local Explanations. Annals of Computer Science and Information Systems, vol. 30, pp. 89–92. https://doi.org/10.15439/2022F258
  4. J. Fürnkranz, D. Gamberger, N. Lavrač. Foundations of Rule Learning. 2012. 10.1007/978-3-540-75197-7.
  5. B. Letham, C. Rudin, T. H. McCormick, and D. Madigan. Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model. Annals of Applied Statistics, 9:1350–1371, 2015.
  6. B. Duval, A. Salleb, C. Vrain. On the Discovery of Exception Rules: A Survey. In Guillet, F.J., Hamilton, H.J. (eds) Quality Measures in Data Mining. Studies in Computational Intelligence, vol 43. Springer, Berlin, Heidelberg, 2007. https://doi.org/10.1007/978-3-540-44918-8_4
  7. F. Hussain, H. Liu, E. Suzuki, H. Lu. Exception Rule Mining with a Relative Interestingness Measure. In Proceedings of the 4th Pacific-Asia Conference on Knowledge Discovery and Data Mining, pages 86–97, 2000. 10.1007/3-540-45571-X_11.
  8. M. Calvo-Flores, M. Ruiz, D. Sánchez. New Approaches for Discovering Exception and Anomalous Rules. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 19:361–399, 2011. https://dx.doi.org/10.1142/S0218488511007039
  9. Zielosko, B., Stańczyk, U., Jabłoński, K. (2023). Filtering Decision Rules Driven by Sequential Forward and Backward Selection of Attributes: An Illustrative Example in Stylometric Domain. Annals of Computer Science and Information Systems, vol. 35, pp. 833–842. https://doi.org/10.15439/2023F7295
  10. Sikora, M., Skowron, A., Wróbel, Ł. (2012). Rule Quality MeasureBased Induction of Unordered Sets of Regression Rules. In: Ramsay, A., Agre, G. (eds) Artificial Intelligence: Methodology, Systems, and Applications. AIMSA 2012. Lecture Notes in Computer Science(), vol 7557. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-64233185-5_18
  11. A. Silberschatz and A. Tuzhilin. On subjective measure of interestingness in knowledge discovery. In: Proceedings of the First International Conference on Knowledge Discovery and Data Mining, Montreal, August 20–21, 1995, pp. 275–281.
  12. B. Padmanabhan and A. Tuzhilin. A belief-driven method for discovering unexpected patterns. In: Proceedings of the Fourth International Conference on Knowledge Discovery and Data Mining, New York, August 27–31, 1998, pp. 94–100.
  13. E. Suzuki and M. Shimura. Exceptional knowledge discovery in databases based on information theory. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (KDD’96), AAAI Press, pages 275–278, 1996.
  14. E. Suzuki. Discovering unexpected exceptions: A stochastic approach. In: Proceedings of the Fourth International Workshop on Rough Sets, Fuzzy Sets, and Machine Discovery (RSFD), 1996.
  15. J. Vashishtha, D. Kumar, S. Ratnoo. An evolutionary approach to discover intra-and inter-class exceptions in databases. International Journal of Intelligent Systems Technologies and Applications, 12:283300, 2013. 10.1504/IJISTA.2013.056535.
  16. S. Ratnoo, A. Pathak, J. Ahuja, J. Vashishtha. Exception Discovery Using Ant Colony Optimization. International Journal of Computational Systems Engineering, 4, 2018. 10.1504/IJCSYSE.2018.10011809.
  17. F. Janssen, J. F¨urnkranz, Heuristic rule-based regression via dynamic reduction to classification, in: IJCAI’11, AAAI Press, 2011, pp. 1330–1335.
  18. H. B. Mann and D. R. Whitney. On a test of whether one of two random variables is stochastically larger than the other. The Annals of Mathematical Statistics, 18(1):50–60, 1947.
  19. A. Gudyś, M. Sikora, Ł. Wróbel. RuleKit: A comprehensive suite for rule-based learning. Knowledge-Based Systems, 194, 2020, 105480, ISSN 0950-7051. https://doi.org/10.1016/j.knosys.2020.105480.
  20. Ł. Wróbel, M. Sikora, M. Michalak. Rule Quality Measures Settings in Classification, Regression and Survival Rule Induction — an Empirical Approach. Fundamenta Informaticae, 149(4):419–449, 2016. https://dx.doi.org/10.3233/FI-2016-1455
  21. Yeh, I. (1998). Concrete Compressive Strength [Dataset]. UCI Machine Learning Repository. https://doi.org/10.24432/C5PK67.
  22. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. VanderPlas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, E. Duchesnay. Scikitlearn: Machine Learning in Python. Journal of Machine Learning Research, 12, 2011, 2825–2830. https://jmlr.org/papers/v12/pedregosa11a. html.
  23. P. Nemenyi. Distribution-Free Multiple Comparisons. PhD Thesis, Princeton University, Princeton, NJ, 1963.
  24. J. Demšar. Statistical Comparisons of Classifiers over Multiple Data Sets. Journal of Machine Learning Research, 7, 2006, 1–30. https://jmlr.org/papers/v7/demsar06a.html.