Logo PTI Logo FedCSIS

Proceedings of the 21st Conference on Computer Science and Intelligence Systems (FedCSIS)

Annals of Computer Science and Information Systems, Volume 47

Searching for Accuracy vs. Efficiency Trade-off Solution in Deep Differentiable Logic Gate Networks

,

DOI: http://dx.doi.org/10.15439/2026F6629

Citation: Chan Duong Nguy,

Full text

Abstract. Recent research on Deep Differentiable Logic Gate Networks (DLNs) aim to shift machine learning from traditional weight based models to weightless networks that learn the best logical operators using gradient-based optimization. While DLNs can provide much faster inference and require less memory, they still face issues such as vanishing gradients in deeper networks and high computational cost during training. This paper compares several DLN configurations to find the best trade-off between accuracy and efficiency. We evaluate baseline DLNs against versions that use Residual Initialization (RI) at identity gates (denoted as DLN$\_{\alpha**}$ where $\alpha \in \{\text{Z}, \text{M}, \text{\L}\}$, representing Zadeh (Z), Menger (M), and \L{}ukasiewicz (\L{})) and a new mirror gate architecture (denoted as DLN$\_{\alpha**}^{m}$). Our study focuses on three main T-norm relaxations: Menger, Zadeh, and \L{}ukasiewicz. Results on benchmark datasets such as MONK, Adult, Breast Cancer, and MNIST show that DLN$\_{\alpha}^{**}$ models greatly reduce vanishing gradients and improve accuracy in deeper networks. In addition, the DLN$\_{\alpha}^{m**}$ variant reduces training memory use and computational cost by about 50\\% while not significantly reducing the expressive power in some dataset.

References

  1. B. Becker and R. Kohavi, “Adult," UCI Machine Learning Repository, 1996. Available: https://doi.org/10.24432/C5XW20
  2. S. Bosse, “IoT and Edge Computing using virtualized low-resource integer Machine Learning with support for CNN, ANN, and Decision Trees,” Proc. 18th Conf. Comput. Sci. Intell. Syst. (FedCSIS), vol. 35, pp. 367–376, 2023, M. Ganzha, L. Maciaszek, M. Paprzycki, and D. Śl˛ ezak, Eds., IEEE. Available: http://dx.doi.org/10.15439/2023F7745
  3. L. Breiman, “Random Forests,” Machine Learning, vol. 45, no. 1, pp. 5–32, Oct. 2001, https://doi.org/10.1023/A:1010933404324.
  4. T. Chen and C. Guestrin, “XGBoost: a Scalable Tree Boosting System,” Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD ’16, vol. 1, no. 1, pp. 785–794, Aug. 2016, https://doi.org/10.1145/2939672.2939785.
  5. L. da Cruz, C. Sierra-Franco, G. Silva-Calpa, and A. Raposo, “Enabling Autonomous Medical Image Data Annotation: A human-in-the-loop Reinforcement Learning Approach,” in *Proc. 16th Conf. on Computer Science and Intelligence Systems (FedCSIS)*, vol. 25, M. Ganzha, L. Maciaszek, M. Paprzycki, and D. Śl˛ezak, Eds., IEEE, 2021, pp. 271– 279. Available: http://dx.doi.org/10.15439/2021F86
  6. D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization," 2014. Available: https://arxiv.org/abs/1412.6980
  7. E. van Krieken, E. Acar, and F. van Harmelen, “Analyzing Differentiable Fuzzy Logic Operators," Artificial Intelligence, vol. 302, 2022, pp. 103602. Available: https://www.sciencedirect.com/science/article/pii/ S0004370221001533
  8. A. Krizhevsky, “Learning Multiple Layers of Features from Tiny Images," Univ. of Toronto, 2012.
  9. Y. LeCun, C. Cortes, and C. J. C. Burges, “The MNIST database of handwritten digits," 1998. Available: http://yann.lecun.com/exdb/mnist/
  10. K. Menger, “Statistical Metrics," Proc. Nat. Acad. Sci. U.S.A., vol. 28, no. 12, Dec. 1942, pp. 535–537. Available: https://doi.org/10.1073/pnas. 28.12.535
  11. D. B. de Oliveira, D. Casini and T. Cucinotta, "Operating System Noise in the Linux Kernel," in IEEE Transactions on Computers, vol. 72, no. 1, pp. 196-207, 1 Jan. 2023, https://doi.org/10.1109/TC.2022.3187351.
  12. A. Paszke et al., “PyTorch: An Imperative Style, High-Performance Deep Learning Library," 2019. Available: https://arxiv.org/abs/1912.01703
  13. F. Petersen, C. Borgelt, H. Kuehne, and O. Deussen, “Deep Differentiable Logic Gate Networks," in Advances in Neural Information Processing Systems, vol. 35, 2022, pp. 2006–2018. Available: https://proceedings.neurips.cc/paper_files/paper/2022/file/ 0d3496dd0cec77a999c98d35003203ca-Paper-Conference.pdf
  14. F. Petersen, H. Kuehne, C. Borgelt, J. Welzel, and S. Ermon, “Convolutional Differentiable Logic Gate Networks," in Advances in Neural Information Processing Systems, vol. 37, 2024, pp. 121185– 121203. Available: https://proceedings.neurips.cc/paper_files/paper/ 2024/file/db988b089d8d97d0f159c15ed0be6a71-Paper-Conference.pdf
  15. J. R. Quinlan, “Induction of decision trees,” Machine Learning, vol. 1, no. 1, pp. 81–106, Mar. 1986, https://doi.org/10.1007/bf00116251.
  16. C. D. Nguy and P. Wasilewski, “Deep Differentiable Logic Gate Networks Based on Fuzzy Łukasiewicz T-norm," 2025 20th Conference on Computer Science and Intelligence Systems (FedCSIS), Kraków, Poland, 2025, pp. 219-230, https://doi.org/10.15439/2025F1666
  17. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors," Nature, vol. 323, no. 6088, Oct. 1986, pp. 533–536. Available: https://doi.org/10.1038/323533a0
  18. S. Thrun et al.,“The Monk’s Problem’s: A Performance Comparison of Different Learning Methods," 1991. Available: https://api. semanticscholar.org/CorpusID:59810521
  19. P. Wasilewski and C. D. Nguy, “Deep Differentiable Logic Gate Networks Based on Fuzzy Zadeh’s T-Norm," Lecture Notes in Networks and Systems, pp. 57–70, 2026, https://doi.org/10.1007/ 978-3-032-04197-5_5.
  20. P. Wasilewski, and Ch. D. Nguy, “Deep Differentiable Logic Gate Networks: Learning with Mirror Gates," Proceedings of 6th Polish Conference on Artificial Intelligence (PP-RAI 2026), Krakow, Poland, Apr. 20-22, 2026. to appear in Lecture Notes in Networks and Systems, Springer.
  21. L. A. Zadeh, “Fuzzy sets," Information and Control, vol. 8, no. 3, 1965, pp. 338–353. Available: https://www.sciencedirect.com/science/article/ pii/S001999586590241X
  22. M. Zwitter and M. Soklic, “Breast Cancer," UCI Machine Learning Repository, 1988. Available: https://doi.org/10.24432/C51P4M