Logo PTI Logo FedCSIS

Proceedings of the 20th Conference on Computer Science and Intelligence Systems (FedCSIS)

Annals of Computer Science and Information Systems, Volume 43

Estimating the Difficulty of Chess Puzzles by Combining Fine-Tuned Maia-2 with Hand-Crafted and Engine Features

DOI: http://dx.doi.org/10.15439/2025F6497

Citation: Proceedings of the 20th Conference on Computer Science and Intelligence Systems (FedCSIS), M. Bolanowski, M. Ganzha, L. Maciaszek, M. Paprzycki, D. Ślęzak (eds). ACSIS, Vol. 43, pages 801–806 ()

Full text

Abstract. A common way for chess players to practice tactical awareness is to solve chess puzzles, consisting of an initial position and a sequence of moves to achieve a winning position. This practice is more effective when puzzles are matched to the player's skill level. In this work, we present an approach for estimating the difficulty of a chess puzzle using only the initial position and the sequence of correct moves. Our approach uses a fine-tuned modification of the Maia-2 model combined with a set of hand-crafted features and features extracted from chess engines such as Leela Chess Zero and Stockfish. All of these features are then used as input to a gradient boosted decision tree model that predicts the final rating of the puzzle. We applied our approach to the FedCSIS 2025 Challenge on Predicting Chess Puzzle Difficulty Part 2, where it achieved first place.

References

  1. Lichess.org, “Lichess training,” https://lichess.org/training, 2025, accessed: 2025-08-16.
  2. Chess.com, “Chess.com puzzles,” https://www.chess.com/puzzles, 2025, accessed: 2025-08-16.
  3. ChessTempo, “Chesstempo,” https://chesstempo.com/, 2025, accessed: 2025-08-16.
  4. T. Duplessis, “lichess puzzler,” https://github.com/ornicar/lichess-puzzler. [Online]. Available: https://github.com/ornicar/lichess-puzzler
  5. A. E. Elo, The Rating of Chessplayers, Past and Present. New York: Arco Pub., 1978. ISBN 0668047216 9780668047210. [Online]. Available: http://www.amazon.com/Rating-Chess-Players-Past-Present/dp/0668047216
  6. M. E. Glickman, “Example of the glicko-2 system,” http://www.glicko.net/glicko/glicko2.pdf, 2022, published: 2022-03-22, accessed: 2025-08-16.
  7. J. Zyśko, M. Ślęzak, D. Ślęzak, and M. Świechowski, “FedCSIS 2025 knowledgepit.ai Competition: Predicting Chess Puzzle Difficulty Part 2 & A Step Toward Uncertainty Contests,” in Proceedings of the 20th Conference on Computer Science and Intelligence Systems, ser. Annals of Computer Science and Information Systems, M. Bolanowski, M. Ganzha, L. Maciaszek, M. Paprzycki, and D. Ślęzak, Eds., vol. 43. Polish Information Processing Society, 2025. https://dx.doi.org/10.15439/2025F5937. [Online]. Available: http://dx.doi.org/10.15439/2025F5937
  8. J. Zyśko, M. Świechowski, S. Stawicki, K. Jagieła, A. Janusz, and D. Ślęzak, “Ieee big data cup 2024 report: Predicting chess puzzle difficulty at knowledgepit.ai,” in IEEE International Conference on Big Data, Big Data 2024, Washington DC, USA, December 15-18, 2024. IEEE, 2024.
  9. Z. Tang, D. Jiao, R. McIlroy-Young, J. Kleinberg, S. Sen, and A. Anderson, “Maia-2: A unified model for human-ai alignment in chess,” Advances in Neural Information Processing Systems, vol. 37, pp. 20 919–20 944, 2024.
  10. T. Woodruff, O. Filatov, and M. Cognetta, “The bread emoji team’s submission to the ieee bigdata 2024 cup: Predicting chess puzzle difficulty challenge,” in 2024 IEEE International Conference on Big Data (BigData). IEEE, 2024, pp. 8415–8422.
  11. A. Schütt, T. Huber, and E. André, “Estimating chess puzzle difficulty without past game records using a human problem-solving inspired neural network architecture,” in 2024 IEEE International Conference on Big Data (BigData). IEEE, 2024, pp. 8396–8402.
  12. S. Björkqvist, “Estimating the puzzlingness of chess puzzles,” in 2024 IEEE International Conference on Big Data (BigData). IEEE, 2024, pp. 8370–8376.
  13. S. Stoiljkovikj, I. Bratko, M. Guid, and F. UNI, “A computational model for estimating the difficulty of chess problems,” in Proceedings of the third annual conference on advances in cognitive systems ACS, 2015, p. 7.
  14. D. Hristova, M. Guid, and I. Bratko, “Assessing the difficulty of chess tactical problems,” International journal on advances in intelligent systems, vol. 7, no. 3, pp. 728–738, 2014.
  15. R. McIlroy-Young, S. Sen, J. Kleinberg, and A. Anderson, “Aligning superhuman ai with human behavior: Chess as a model system,” in Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, ser. KDD ’20. New York, NY, USA: Association for Computing Machinery, 2020. https://dx.doi.org/10.1145/3394486.3403219. ISBN 9781450379984 p. 1677–1687. [Online]. Available: https://doi.org/10.1145/3394486.3403219
  16. The LCZero Authors, “Leela chess zero.” [Online]. Available: https://lczero.org/
  17. G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T.-Y. Liu, “Lightgbm: a highly efficient gradient boosting decision tree,” in Proceedings of the 31st International Conference on Neural Information Processing Systems, ser. NIPS’17. Red Hook, NY, USA: Curran Associates Inc., 2017. ISBN 9781510860964 p. 3149–3157.
  18. A. Rafaralahy, “Pairwise learning to rank for chess puzzle difficulty prediction,” in 2024 IEEE International Conference on Big Data (BigData). IEEE, 2024, pp. 8385–8389.
  19. S. Miłosz and P. Kapusta, “Predicting chess puzzle difficulty with transformers,” in 2024 IEEE International Conference on Big Data (BigData). IEEE, 2024, pp. 8377–8384.
  20. D. Ruta, M. Liu, and L. Cen, “Moves based prediction of chess puzzle difficulty with convolutional neural networks,” in 2024 IEEE International Conference on Big Data (BigData). IEEE, 2024, pp. 8390–8395.
  21. M. Omori and P. Tadepalli, “Chess rating estimation from moves and clock times using a cnn-lstm,” in International Conference on Computers and Games. Springer, 2024, pp. 3–13.
  22. Lichess.org, “Lichess open database: Chess puzzles,” https://database.lichess.org/#puzzles, 2025, accessed: 2025-08-16.
  23. Wikipedia contributors, “Forsyth–edwards notation — Wikipedia,” https://en.wikipedia.org/wiki/Forsyth%E2%80%93Edwards_Notation, 2025, accessed: 2025-08-16.
  24. Wikipedia contributors, “Algebraic Notation (Chess) — Wikipedia,” https://en.wikipedia.org/wiki/Algebraic_notation_(chess), 2025, accessed: 2025-08-16.
  25. A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Köpf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, PyTorch: an imperative style, high-performance deep learning library. Red Hook, NY, USA: Curran Associates Inc., 2019.
  26. A. Defazio, X. Yang, A. Khaled, K. Mishchenko, H. Mehta, and A. Cutkosky, “The road less scheduled,” Advances in Neural Information Processing Systems, vol. 37, pp. 9974–10 007, 2024.
  27. A. V. Dorogush, V. Ershov, and A. Gulin, “Catboost: gradient boosting with categorical features support,” arXiv preprint https://arxiv.org/abs/1810.11363, 2018. [Online]. Available: https://arxiv.org/abs/1810.11363
  28. Leela Chess Zero contributors, “Best networks for lc0,” https://lczero.org/dev/wiki/best-nets-for-lc0/, 2024, accessed: 2025-07-25. [Online]. Available: https://lczero.org/dev/wiki/best-nets-for-lc0/
  29. dkappe, “Leela chess weights: Bad gyal,” https://github.com/dkappe/leela-chess-weights/wiki/Bad-Gyal, 2020, GitHub release, accessed 2025-07-25. [Online]. Available: https://github.com/dkappe/leela-chess-weights/wiki/Bad-Gyal
  30. Leela Chess Zero contributors, “Contributed networks for lc0,” https://storage.lczero.org/files/networks-contrib/, 2024, accessed: 2025-07-26. [Online]. Available: https://storage.lczero.org/files/networks-contrib/
  31. The Stockfish developers, T. Romstad, M. Costalba, J. Kiiski, G. Linscott, Y. Nasu, M. Isozaki, and H. Noda, “Stockfish.” [Online]. Available: https://stockfishchess.org/
  32. N. Fiekas, “python-chess: a chess library for python,” https://github.com/niklasf/python-chess, 2025. [Online]. Available: https://github.com/niklasf/python-chess
  33. W. McKinney, “Data structures for statistical computing in python,” in Proceedings of the 9th Python in Science Conference, S. van der Walt and J. Millman, Eds., 2010, pp. 51 – 56.