Proceedings of the 20th Conference on Computer Science and Intelligence Systems (FedCSIS)

DOI: http://dx.doi.org/10.15439/2025F5937

Citation: Proceedings of the 20th Conference on Computer Science and Intelligence Systems (FedCSIS), M. Bolanowski, M. Ganzha, L. Maciaszek, M. Paprzycki, D. Ślęzak (eds). ACSIS, Vol. 43, pages 849–854 (2025)

Full text

Abstract. We summarize the results of the FedCSIS 2025 machine learning competition organized on the knowledgepit.ai platform. We recall the competition's goals corresponding to estimations of the chess puzzle difficulty levels, we refer to the winning solutions, and we also compare the scope of this year's competition (and particularly the data available to competition participants) with its previous edition associated with the IEEE BigData 2024 conference. Finally, we discuss the new functionality of the knowledgepit.ai platform, which enables competition participants to submit additional ``uncertainty masks'' reflecting their assessment of test cases that are mostly problematic for their machine learning models.

References

1. J. Zyśko, M. Świechowski, S. Stawicki, K. Jagieła, A. Janusz, and D. Ślęzak, “IEEE Big Data Cup 2024 Report: Predicting Chess Puzzle Difficulty at KnowledgePit.ai,” in IEEE International Conference on Big Data, BigData 2024, Washington, DC, USA, December 15-18, 2024, W. Ding, C. Lu, F. Wang, L. Di, K. Wu, J. Huan, R. Nambiar, J. Li, F. Ilievski, R. Baeza-Yates, and X. Hu, Eds. IEEE, 2024, pp. 8423–8429. [Online]. Available: https://doi.org/10.1109/BigData62323.2024.10825289
2. Z. Tang, D. Jiao, R. McIlroy-Young, J. M. Kleinberg, S. Sen, and A. Anderson, “Maia-2: A Unified Model for Human-AI Alignment in Chess,” in Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10-15, 2024, A. Globersons, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. M. Tomczak, and C. Zhang, Eds., 2024. [Online]. Available: http://papers.nips.cc/paper_files/paper/2024/hash/250190819ff1dda47cd23cecc0c5a69b-Abstract-Conference.html
3. T. Woodruff, O. Filatov, and M. Cognetta, “The bread emoji Team’s Submission to the IEEE BigData 2024 Cup: Predicting Chess Puzzle Difficulty Challenge,” in IEEE International Conference on Big Data, BigData 2024, Washington, DC, USA, December 15-18, 2024, W. Ding, C. Lu, F. Wang, L. Di, K. Wu, J. Huan, R. Nambiar, J. Li, F. Ilievski, R. Baeza-Yates, and X. Hu, Eds. IEEE, 2024, pp. 8415–8422. [Online]. Available: https://doi.org/10.1109/BigData62323.2024.10826037
4. A. Schütt, T. Huber, and E. André, “Estimating Chess Puzzle Difficulty Without Past Game Records Using a Human Problem-Solving Inspired Neural Network Architecture,” in IEEE International Conference on Big Data, BigData 2024, Washington, DC, USA, December 15-18, 2024, W. Ding, C. Lu, F. Wang, L. Di, K. Wu, J. Huan, R. Nambiar, J. Li, F. Ilievski, R. Baeza-Yates, and X. Hu, Eds. IEEE, 2024, pp. 8396–8402. [Online]. Available: https://doi.org/10.1109/BigData62323.2024.10826087
5. S. Björkqvist, “Estimating the Puzzlingness of Chess Puzzles,” in IEEE International Conference on Big Data, BigData 2024, Washington, DC, USA, December 15-18, 2024, W. Ding, C. Lu, F. Wang, L. Di, K. Wu, J. Huan, R. Nambiar, J. Li, F. Ilievski, R. Baeza-Yates, and X. Hu, Eds. IEEE, 2024, pp. 8370–8376. [Online]. Available: https://doi.org/10.1109/BigData62323.2024.10825991
6. A. Janusz, M. Przyborowski, P. Biczyk, and D. Ślęzak, “Network Device Workload Prediction: A Data Mining Challenge at Knowledge Pit,” in Proceedings of the 2020 Federated Conference on Computer Science and Information Systems, FedCSIS 2020, Sofia, Bulgaria, September 6-9, 2020, ser. Annals of Computer Science and Information Systems, M. Ganzha, L. Maciaszek, and M. Paprzycki, Eds., vol. 21, 2020, pp. 77–80. [Online]. Available: https://doi.org/10.15439/2020F159
7. A. Janusz and D. Ślęzak, “KnowledgePit Meets BrightBox: A Step Toward Insightful Investigation of the Results of Data Science Competitions,” in Proceedings of the 17th Conference on Computer Science and Intelligence Systems, FedCSIS 2022, Sofia, Bulgaria, September 4-7, 2022, ser. Annals of Computer Science and Information Systems, vol. 30, 2022, pp. 393–398. [Online]. Available: https://doi.org/10.15439/2022F309
8. M. Wnuk, J. Dziuba, A. Janusz, and D. Ślęzak, “IEEE BigData Cup 2023 Report: Object Recognition with Muon Tomography Using Cosmic Rays,” in IEEE International Conference on Big Data, BigData 2023, Sorrento, Italy, December 15-18, 2023, J. He, T. Palpanas, X. Hu, A. Cuzzocrea, D. Dou, D. Ślęzak, W. Wang, A. Gruca, J. C. Lin, and R. Agrawal, Eds. IEEE, 2023, pp. 6084–6091. [Online]. Available: https://doi.org/10.1109/BigData59044.2023.10386564
9. A. Janusz and D. Ślęzak, “Predicting Frags in Tactic Games at knowledgepit.ai: ICME 2023 Grand Challenge Report,” in IEEE International Conference on Multimedia and Expo Workshops, ICMEW Workshops 2023, Brisbane, Australia, July 10-14, 2023. IEEE, 2023, pp. 1–5. [Online]. Available: https://doi.org/10.1109/ICMEW59549.2023.00006
10. A. M. Rakicevic, P. D. Milosevic, I. T. Dragovic, A. M. Poledica, M. M. Zukanovic, A. Janusz, and D. Ślęzak, “Predicting Stock Trends Using Common Financial Indicators: A Summary of FedCSIS 2024 Data Science Challenge Held on knowledgepit.ai Platform,” in Proceedings of the 19th Conference on Computer Science and Intelligence Systems, FedCSIS 2024, Belgrade, Serbia, September 8-11, 2024, ser. Annals of Computer Science and Information Systems, M. Bolanowski, M. Ganzha, L. Maciaszek, M. Paprzycki, and D. Ślęzak, Eds., vol. 39, 2024, pp. 731–737. [Online]. Available: https://doi.org/10.15439/2024F7912
11. D. Ślęzak, A. Janusz, M. Świechowski, A. Chadzyńska-Krasowska, and J. Kamiński, “Do Data Scientists Dream About Their Skills’ Assessment? – Transforming a Competition Platform Into an Assessment Platform,” in IEEE International Conference on Big Data, BigData 2024, Washington, DC, USA, December 15-18, 2024, W. Ding, C. Lu, F. Wang, L. Di, K. Wu, J. Huan, R. Nambiar, J. Li, F. Ilievski, R. Baeza-Yates, and X. Hu, Eds. IEEE, 2024, pp. 8403–8414. [Online]. Available: https://doi.org/10.1109/BigData62323.2024.10825378
12. E. Hüllermeier and W. Waegeman, “Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods,” Machine Learning, vol. 110, no. 3, pp. 457–506, 2021. [Online]. Available: https://doi.org/10.1007/s10994-021-05946-3
13. D. Kałuża, A. Janusz, and D. Ślęzak, “Evidence-theoretical Modeling of Uncertainty Induced by Posterior Probability Distributions,” International Journal of Applied Mathematics and Computer Science, vol. 35, no. 1, 2025. [Online]. Available: https://doi.org/10.61822/amcs-2025-0003
14. A. Janusz, A. Zalewska, Ł. Wawrowski, P. Biczyk, J. Ludziejewski, M. Sikora, and D. Ślęzak, “BrightBox – A Rough Set based Technology for Diagnosing Mistakes of Machine Learning Models,” Applied Soft Computing, vol. 141, p. 110285, 2023. [Online]. Available: https://doi.org/10.1016/j.asoc.2023.110285
15. A. Janusz, D. Kałuża, D. Ślęzak, and S. Stawicki, “Automatic Generation of Attributes based on Semantic Categorization of Large Datasets in Artificial Intelligence Models and Applications,” US Patent Application US-20250005436-A1, 2025.
16. R. McIlroy-Young, S. Sen, J. M. Kleinberg, and A. Anderson, “Aligning Superhuman AI with Human Behavior: Chess as a Model System,” in KDD ’20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020, R. Gupta, Y. Liu, J. Tang, and B. A. Prakash, Eds. ACM, 2020, pp. 1677–1687. [Online]. Available: https://doi.org/10.1145/3394486.3403219
17. S. Björkqvist, “Estimating the Difficulty of Chess Puzzles by Combining Fine-Tuned Maia-2 with Hand-Crafted and Engine Features,” in Proceedings of the 20th Conference on Computer Science and Intelligence Systems, ser. Annals of Computer Science and Information Systems, M. Bolanowski, M. Ganzha, L. Maciaszek, M. Paprzycki, and D. Ślęzak, Eds., vol. 43. Polish Information Processing Society, 2025. [Online]. Available: http://dx.doi.org/10.15439/2025F6497
18. T. Woodruff, L. Imbing, and M. Cognetta, “The bread emoji Team’s Submission to the 2025 FedCSIS Predicting Chess Puzzle Difficulty Challenge,” in Proceedings of the 20th Conference on Computer Science and Intelligence Systems, ser. Annals of Computer Science and Information Systems, M. Bolanowski, M. Ganzha, L. Maciaszek, M. Paprzycki, and D. Ślęzak, Eds., vol. 43. Polish Information Processing Society, 2025. [Online]. Available: http://dx.doi.org/10.15439/2025F6771
19. S. Miłosz, “Pretraining Transformers for Chess Puzzle Difficulty Prediction,” in Proceedings of the 20th Conference on Computer Science and Intelligence Systems, ser. Annals of Computer Science and Information Systems, M. Bolanowski, M. Ganzha, L. Maciaszek, M. Paprzycki, and D. Ślęzak, Eds., vol. 43. Polish Information Processing Society, 2025. [Online]. Available: http://dx.doi.org/10.15439/2025F7603
20. H. Xiao, D. Yu, X. Wen, L. Chen, and K. Fu, “Multi-Source Feature Fusion and Neural Embedding for Predicting Chess Puzzle Difficulty,” in Proceedings of the 20th Conference on Computer Science and Intelligence Systems, ser. Annals of Computer Science and Information Systems, M. Bolanowski, M. Ganzha, L. Maciaszek, M. Paprzycki, and D. Ślęzak, Eds., vol. 43. Polish Information Processing Society, 2025. [Online]. Available: http://dx.doi.org/10.15439/2025F2456
21. L. Cen, J. Cen, and M. Song, “A Multi-Stage Framework for Chess Puzzle Difficulty Prediction,” in Proceedings of the 20th Conference on Computer Science and Intelligence Systems, ser. Annals of Computer Science and Information Systems, M. Bolanowski, M. Ganzha, L. Maciaszek, M. Paprzycki, and D. Ślęzak, Eds., vol. 43. Polish Information Processing Society, 2025. [Online]. Available: http://dx.doi.org/10.15439/2025F4532
22. A. Liang, C. Liu, K. Wang, and E. Liu, “A Stacking-Based Ensemble Approach for Predicting Chess Puzzle Difficulty,” in Proceedings of the 20th Conference on Computer Science and Intelligence Systems, ser. Annals of Computer Science and Information Systems, M. Bolanowski, M. Ganzha, L. Maciaszek, M. Paprzycki, and D. Ślęzak, Eds., vol. 43. Polish Information Processing Society, 2025. [Online]. Available: http://dx.doi.org/10.15439/2025F1698
23. M. Liu, J. Wang, Y. Hu, and D. Lin, “Hybrid Boosting and Multi-Modal Fusion for Chess Puzzle Difficulty Prediction,” in Proceedings of the 20th Conference on Computer Science and Intelligence Systems, ser. Annals of Computer Science and Information Systems, M. Bolanowski, M. Ganzha, L. Maciaszek, M. Paprzycki, and D. Ślęzak, Eds., vol. 43. Polish Information Processing Society, 2025. [Online]. Available: http://dx.doi.org/10.15439/2025F3675
24. J. Chen, C. Liu, and Y. Gao, “Multi-Modal Deep Learning with Residual and Structure-Guided Refinement for Chess Puzzle Difficulty Prediction,” in Proceedings of the 20th Conference on Computer Science and Intelligence Systems, ser. Annals of Computer Science and Information Systems, M. Bolanowski, M. Ganzha, L. Maciaszek, M. Paprzycki, and D. Ślęzak, Eds., vol. 43. Polish Information Processing Society, 2025. [Online]. Available: http://dx.doi.org/10.15439/2025F3227