Logo PTI Logo FedCSIS

Proceedings of the 18th Conference on Computer Science and Intelligence Systems

Annals of Computer Science and Information Systems, Volume 35

Controllability for English-Ukrainian Machine Translation by Using Style Transfer Techniques

, , , , ,

DOI: http://dx.doi.org/10.15439/2023F895

Citation: Proceedings of the 18th Conference on Computer Science and Intelligence Systems, M. Ganzha, L. Maciaszek, M. Paprzycki, D. Ślęzak (eds). ACSIS, Vol. 35, pages 1059–1068 ()

Full text

Abstract. While straightforward machine translation got significant improvements in the last 10 years with the arrival of encoder-decoder neural networks and transformers architecture, controllable machine translation still remains a difficult task, which requires lots of research. Existing methods like tagging provide very limited control over model results or they require to support multiple models at once, like domain fine-tuning approach. In this paper, we propose a method to control translation results style by transferring features from a set of texts with target structure and wording. Our solution consists of new modifications for the encoder-decoder networks, where we can add feature descriptors to each token embedding to decode input text into the translation with the proposed domain. In conducted experiments with English-Ukrainian translation and a set of 4 domains our proposed model gives more options to influence the result than some existing approaches to solve the controllability model.

References

  1. A. Gillioz, J. Casas, E. Mugellini and O. A. Khaled, "Overview of the Transformer-based Models for NLP Tasks," 2020 15th Conference on Computer Science and Information Systems (FedCSIS), Sofia, Bulgaria, 2020, pp. 179-183, http://dx.doi.org/10.15439/2020F20.
  2. M. Lewis, ‘BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension’. arXiv, 2019.
  3. S. M. Lundberg and S.-I. Lee, ‘A Unified Approach to Interpreting Model Predictions’, Advances in Neural Information Processing Systems, 2017, p. 30.
  4. Neural Natural Language Generation: A Survey on Multilinguality, Multimodality, Controllability and Learning Erdem, Kuyu, Yagcioglu, Frank, Parcalabescu, Plank, Babii, Turuta et al. Journal of Artificial Intelligence Research 73 (2022) 1131-1207. https://doi.org/10.1613/jair.1.12918
  5. T. B. Brown et al., ‘Language Models are Few-Shot Learners’, arXiv [cs.CL]. 2020.
  6. Saichyshyna N., Maksymenko D., Turuta O., Yerokhin A., Babii A. and Turuta O. 2023. Extension Multi30K: Multimodal Dataset for Integrated Vision and Language Research in Ukrainian. In Proceedings of the Second Ukrainian Natural Language Processing Workshop (UNLP), pages 54–61, Dubrovnik, Croatia. Association for Computational Linguistics.
  7. https://zakon.rada.gov.ua/rada/main/en/llenglaws
  8. R. Hanslo, "Deep Learning Transformer Architecture for Named-Entity Recognition on Low-Resourced Languages: State of the art results," 2022 17th Conference on Computer Science and Intelligence Systems (FedCSIS), Sofia, Bulgaria, 2022, pp. 53-60, http://dx.doi.org/10.15439/2022F53.
  9. J. Tiedemann, S. Thottingal, ‘OPUS-MT -- Building open translation services for the World’, Proceedings of the 22nd Annual Conference of the European Association for Machine Translation, 2020, pp. 479–480.
  10. A. Conneau *, K. Khandelwal *, N. Goyal, V. Chaudhary, G. Wenzek, F. Guzman, E. Grave, M. Ott, L. Zettlemoyer, V. Stoyanov Unsupervised Cross-lingual Representation Learning at Scale
  11. N. Reimers and I. Gurevych, ‘Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks’, in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, 11 2019.
  12. M. Post, ‘A Call for Clarity in Reporting BLEU Scores’, Proceedings of the Third Conference on Machine Translation: Research Papers, 2018, pp. 186–191.
  13. S. Banerjee, A. Lavie, ‘METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments’, Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, 2005, pp. 65–72.
  14. T. Zhang, V. Kishore, F. Wu, K. Q. Weinberger, and Y. Artzi, ‘BERTScore: Evaluating Text Generation with BERT’. ArXiv, 2019.
  15. D.Maksymenko, N.Saichyshyna, O.Turuta, O.Turuta, A.Yerokhin, and A. Babii. 2022. Improving the machine translation model in specific domains for the ukrainian language. In 2022 IEEE 17th International Conference on Computer Sciences and Information Technologies (CSIT), pages 123–129.
  16. M. Junczys-Dowmunt, ‘Marian: Fast Neural Machine Translation in C++’. arXiv, 2018.
  17. https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2