Logo PTI Logo FedCSIS

Position Papers of the 21st Conference on Computer Science and Intelligence Systems

Annals of Computer Science and Information Systems, Volume 48

Challenges in Modeling Historical Language Style for Diachronic Style Recognition

,

DOI: http://dx.doi.org/10.15439/2026F9713

Citation: Adrian Niedziółka-Domański,

Full text

Abstract. This paper addresses the issue of modeling and recognizing historical linguistic styles. Particular attention is paid to the challenges associated with processing diachronic data, such as stylistic differences between eras, limited availability of historical data, and issues arising from the digitization of texts. As part of this work, a process was developed for constructing and cleaning a corpus consisting of texts in Early Modern English and contemporary English. Next, a SentencePiece BPE tokenizer adapted to historical language variants was developed. A discriminator model based on a Transformer Encoder with a FiLM conditioning mechanism was also proposed, designed to determine whether a text belongs to a specific linguistic period. The experiments conducted demonstrated high classification accuracy and a negligible impact of increasing the model size on the final prediction quality.

References

  1. R. W. Murray, “Historical linguistics: the study of language change,” Contemporary linguistics. An introduction, pp. 313– 371, 1996. [Online]. Available: http://www.pearsoncanada.ca/media/ highered-showcase/multi-product-showcase/ogrady-ch08.pdf
  2. S. Kirby, M. Dowman, and T. L. Griffiths, “Innateness and culture in the evolution of language,” Proceedings of the National Academy of Sciences, vol. 104, no. 12, pp. 5241–5245, 2007. https://dx.doi.org/10.1073/pnas.0608222104 _eprint: https://www.pnas.org/doi/pdf/10.1073/pnas.0608222104. [Online]. Available: https://www.pnas.org/doi/abs/10.1073/pnas. 0608222104
  3. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is All you Need,” in Advances in Neural Information Processing Systems, vol. 30. Curran Associates, Inc., 2017. https://dx.doi.org/10.48550/arXiv.1706.03762. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/ 2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html
  4. H. Face, “Tokenizers: Fast state-of-the-art tokenizers optimized for research and production,” 2020. [Online]. Available: https://github.com/huggingface/tokenizers
  5. T. Kudo and J. Richardson, “SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing.” arXiv, Aug. 2018. https://dx.doi.org/10.48550/arXiv.1808.06226 ArXiv:1808.06226 [cs.CL]. [Online]. Available: http://arxiv.org/abs/ 1808.06226
  6. E. Perez, F. Strub, H. De Vries, V. Dumoulin, and A. Courville, “FiLM: Visual Reasoning with a General Conditioning Layer,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, no. 1, Apr. 2018. https://dx.doi.org/10.1609/aaai.v32i1.11671. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/view/11671
  7. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), J. Burstein, C. Doran, and T. Solorio, Eds. Minneapolis, Minnesota: Association for Computational Linguistics, Jun. 2019. https://dx.doi.org/10.18653/v1/N19-1423 pp. 4171–4186. [Online]. Available: https://aclanthology.org/N19-1423/
  8. G. Salton and C. Buckley, “Term-weighting approaches in automatic text retrieval,” Information Processing & Management, vol. 24, no. 5, pp. 513–523, Jan. 1988. https://dx.doi.org/10.1016/0306-4573(88)90021-0. [Online]. Available: https://www.sciencedirect.com/science/article/pii/0306457388900210
  9. C. Cortes and V. Vapnik, “Support-vector networks,” Machine Learning, vol. 20, no. 3, pp. 273–297, Sep. 1995. https://dx.doi.org/10.1007/BF00994018. [Online]. Available: https://doi.org/10.1007/BF00994018
  10. A. Niedziółka-Domański and J. Bylina, “Adapting CycleGAN architecture for Unpaired Diachronic Text Style Transfer,” Kraków, Poland, Oct. 2025. https://dx.doi.org/10.15439/2025F4661 pp. 81–86. [Online]. Available: https://annals-csis.org/Volume_44/drp/4661.html