Logo PTI Logo FedCSIS

Proceedings of the 20th Conference on Computer Science and Intelligence Systems (FedCSIS)

Annals of Computer Science and Information Systems, Volume 43

A Multi-Stage Framework for Chess Puzzle Difficulty Prediction

, , ,

DOI: http://dx.doi.org/10.15439/2025F4532

Citation: Proceedings of the 20th Conference on Computer Science and Intelligence Systems (FedCSIS), M. Bolanowski, M. Ganzha, L. Maciaszek, M. Paprzycki, D. Ślęzak (eds). ACSIS, Vol. 43, pages 807–812 ()

Full text

Abstract. Accurately estimating the difficulty of the chess puzzle is important for adaptive training systems, personalized recommendations, and large-scale content curation. Unlike engine evaluations optimized for perfect play, this task involves modeling human-perceived solving difficulty, typically expressed by Glicko-2 ratings. We present a multi-stage framework developed for the FedCSIS 2025 Challenge. The method trains four rating-banded neural regression models in different Elo ranges to capture localized difficulty patterns and reduce bias from unbalanced data. Their predictions are combined with statistical attributes, including success probabilities, failure distributions, and solution length, through a feature-based regression stage to improve cross-range generalization. A final calibration step adjusts the output to statistically plausible rating levels, mitigating systematic prediction biases without adding computational complexity. An additional mask selection procedure was explored as part of the competition extension to identify 10\% of the puzzles that are most likely to benefit from the refined evaluation. The proposed solution ranked $5 on the public leaderboard and $6 in the final standings. These results demonstrate that a lightweight and interpretable regression pipeline can achieve competitive precision in modeling human-perceived chess puzzle difficulty.

References

  1. J. Zyśko, M. Ślęzak, D. Ślęzak, and M. Świechowski, “FedCSIS 2025 knowledgepit.ai Competition: Predicting Chess Puzzle Difficulty Part 2 & A Step Toward Uncertainty Contests,” in Proc. 20th Conf. Comput. Sci. Intell. Syst. (FedCSIS), vol. 43, Polish Inf. Process. Soc., 2025. http://dx.doi.org/10.15439/2025F5937.
  2. S. Björkqvist, "Estimating the Puzzlingness of Chess Puzzles," 2024 IEEE International Conference on Big Data (BigData), Washington, DC, USA, 2024, pp. 8370-8376, https://dx.doi.org/10.1109/BigData62323.2024.10825991.
  3. A. Rafaralahy, "Pairwise Learning to Rank for Chess Puzzle Difficulty Prediction," 2024 IEEE International Conference on Big Data (BigData), Washington, DC, USA, 2024, pp. 8385-8389, https://dx.doi.org/10.1109/BigData62323.2024.10825356.
  4. D. Ruta, M. Liu and L. Cen, "Moves Based Prediction of Chess Puzzle Difficulty with Convolutional Neural Networks," 2024 IEEE International Conference on Big Data (BigData), Washington, DC, USA, 2024, pp. 8390-8395, https://dx.doi.org/10.1109/BigData62323.2024.10825595.
  5. S. Milosz and P. Kapusta, "Predicting Chess Puzzle Difficulty with Transformers," in 2024 IEEE International Conference on Big Data (BigData), Washington, DC, USA, 2024, pp. 8377-8384, https://dx.doi.org/10.1109/BigData62323.2024.10825919.
  6. M. Omori and P. Tadepalli, “Estimating Player Ratings and Puzzle Difficulty with CNN-LSTM Models,” arXiv preprint https://arxiv.org/abs/2409.11506, 2024.
  7. J. Zyśko, M. Świechowski, S. Stawicki, K. Jagieła, A. Janusz and D. Śl˛ ezak, "IEEE Big Data Cup 2024 Report: Predicting Chess Puzzle Difficulty at KnowledgePit.ai," 2024 IEEE International Conference on Big Data (BigData), Washington, DC, USA, 2024, pp. 8423-8429, https://dx.doi.org/10.1109/BigData62323.2024.10825289.
  8. T. Woodruff, O. Filatov and M. Cognetta, "The bread emoji Team’s Submission to the IEEE BigData 2024 Cup: Predicting Chess Puzzle Difficulty Challenge," 2024 IEEE International Conference on Big Data (BigData), Washington, DC, USA, 2024, pp. 8415-8422, https://dx.doi.org/10.1109/BigData62323.2024.10826037.
  9. A. Schütt, T. Huber and E. André, "Estimating Chess Puzzle Difficulty Without Past Game Records Using a Human Problem-Solving Inspired Neural Network Architecture," 2024 IEEE International Conference on Big Data (BigData), Washington, DC, USA, 2024, pp. 8396-8402, https://dx.doi.org/10.1109/BigData62323.2024.10826087.
  10. A. V. Dorogush, V. Ershov, and A. Gulin, “CatBoost: gradient boosting with categorical features support,” in Proc. Workshop on ML Systems (MLSys), 2018.
  11. G. Ke, Q. Meng, T. Finley, et al., “LightGBM: A highly efficient gradient boosting decision tree,” in Advances in Neural Information Processing Systems (NeurIPS), 2017.
  12. T. Chen and C. Guestrin, “XGBoost: A scalable tree boosting system,” in Proc. ACM SIGKDD, 2016, pp. 785–794.
  13. M. Liu, L. Cen and D. Ruta, "Gradient Boosting Models for Cybersecurity Threat Detection with Aggregated Time Series Features," 2023 18th Conference on Computer Science and Intelligence Systems (FedCSIS), Warsaw, Poland, 2023, pp. 1311-1315, https://dx.doi.org/10.15439/2023F4457.