Communication Papers of the 20th Conference on Computer Science and Intelligence Systems (FedCSIS)

DOI: http://dx.doi.org/10.15439/2025F6104

Citation: Communication Papers of the 20th Conference on Computer Science and Intelligence Systems (FedCSIS), M. Bolanowski, M. Ganzha, L. Maciaszek, M. Paprzycki, D. Ślęzak (eds). ACSIS, Vol. 45, pages 95–101 (2025)

Full text

Abstract. With growing privacy regulations, removing user-related information from machine learning models has become essential. Machine unlearning addresses this by enabling selective removal of learned information, but most existing methods rely on deep learning models, which are computationally expensive and lack interpretability. To overcome these limitations, we propose a novel machine unlearning framework using selective knowledge distillation into a Soft Decision Tree (SDT). A convolutional neural network (ConvNet) is first trained to generate soft labels and intermediate features, which are transferred to the SDT. During distillation, an unlearning algorithm adjusts specific leaf node distributions and routing weights using soft redistribution and path pruning. This enables class-specific forgetting without retraining and preserves accuracy on non-target classes. Experiments on MNIST and CIFAR-10 demonstrate that our framework effectively removes class-specific knowledge while maintaining overall model performance. The interpretable SDT structure also allows for clear visualization of model changes before and after unlearning.

References

1. EU. Regulation (eu) 2016/679. https://eur-lex.europa.eu/eli/reg/2016/679/oj, 2016. [Online; accessed 16-April-2020].
2. California. California consumer privacy act. https://leginfo.legislature.ca.gov/ faces/billTextClient.xhtml?bill_id= 201720180AB375, 2018. [Online; accessed 16-April 2020].
3. L. Bourtoule, V. Chandrasekaran, C. A. Choquette-Choo, H. Jia, A. Travers, B. Zhang, ... & N. Papernot, “Machine unlearning,” In 2021 IEEE symposium on security and privacy (SP), pp.141-159, May 2021.
4. L. Wang, T. Chen, W. Yuan, X. Zeng, K. F. Wong, & H. Yin, “KGA: A general machine unlearning framework based on knowledge gap alignment,” arXiv preprint https://arxiv.org/abs/2305.06545.
5. A. Sekhari, J. Acharya, G. Kamath, & A. T. Suresh, “Remember what you want to forget: Algorithms for machine unlearning,” Advances in Neural Information Processing Systems, vol. 34, pp. 18075-18086, 2021.
6. J. Li, Y. Li, X. Xiang, S. T. Xia, S. Dong & Y. Cai, “TNT: An interpretable tree-network-tree learning framework using knowledge distillation,” Entropy, vol. 22, pp. 1203, 2020.
7. H. Lin, J. W. Chung, Y. Lao, & W. Zhao, “Machine unlearning in gradient boosting decision trees,” In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.1374-1383, August 2023.
8. Z. Wu, J. Zhu, Q. Li, & B. He, “Deltaboost: Gradient boosting decision trees with efficient machine unlearning,” Proceeding of the ACM on Management of Data, Vol. 1, pp. 1-26, 2023.
9. J. Brophy, & D. Lowd, “Machine unlearning for random forests,” In International Conference on Machine Learning, PMLR, pp.1092-1104, July 2021.
10. T. Surve, & R. Pradhan, “Example-based Explanations for Random Forests using Machine Unlearning,” arXiv preprint https://arxiv.org/abs/2402.05007.
11. S. Wang, Z. Shen, X. Qiao, T. Zhang, & M. Zhang, “DynFrs: An Efficient Framework for Machine Unlearning in Random Forest,” arXiv preprint https://arxiv.org/abs/ 2410.01588.
12. Z. Zuo, Z. Tang, B. Wang, K. Li, & A. Datta, “Ecil-mu: Embedding based class incremental learning and machine unlearning,” IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6275-6279, 2024.
13. J. Xu, Z. Wu, C. Wang, & X. Jia, “Machine unlearning: Solutions and challenges,” IEEE Transactions on Emerging Topics in Computational Intelligence.
14. J. Gou, B. Yu, S. J. Maybank & D. Tao, “Knowledge distillation: A survey,” International Journal of Computer Vision, vol. 129, pp. 1789-1819, 2021.
15. G. Hinton, O. Vinyals, & J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint https://arxiv.org/abs/1503.02531, 2015.
16. L. Deng, “The MNIST database of handwritten digit images for machine learning research [best of the web],” IEEE signal processing magazine, vol. 29, pp.141-142, 2012.
17. A. Krizhevsky, & G. Hinton, “Learning multiple layers of features from tiny images,” 2009.
18. N. Frosst, & G. Hinton, “Distilling a neural network into a soft decision tree,” arXiv preprint https://arxiv.org/abs/1711.09784, 2017.
19. T. Akiba, S. Sano, T. Yanase, T. Ohta, & M. Koyama, “Optuna: A next-generation hyperparameter optimization framework,” In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp.2623-2631, July 2019.