Logo PTI Logo FedCSIS

Proceedings of the 20th Conference on Computer Science and Intelligence Systems (FedCSIS)

Annals of Computer Science and Information Systems, Volume 43

Assigning scientific texts to existing ontologies

,

DOI: http://dx.doi.org/10.15439/2025F1850

Citation: Proceedings of the 20th Conference on Computer Science and Intelligence Systems (FedCSIS), M. Bolanowski, M. Ganzha, L. Maciaszek, M. Paprzycki, D. Ślęzak (eds). ACSIS, Vol. 43, pages 185–193 ()

Full text

Abstract. Humans try to help computers understand the properties of the real world, and ontologies can be used for this task. Scientists publish their research in papers, and their results should be used to improve existing ontologies to be up-to-date. Manual enhancement of ontologies is highly time-consuming for domain experts. This paper proposes a solution to match a scientific text to the most relevant ontology using artificial neural networks. Our approach selects a paragraph or a sentence, uses representation learning to embed it into a vector space by some embedder, and measures its relevance to embedded textual properties from the selected ontology by a modified version of a Siamese neural network. A modification is based on the extension of one branch of the Siamese network to aggregate inputs from a group of embeddings. We have considered different embedders, in particular two variants of BERT, InferSent, GloVe with TF-IDF weighted mean, Doc2Vec in the distributed memory variant, and the Llama 3.1 with LLM2vec framework. Their quality has been evaluated on a use case with available ontologies from several application domains. The best results were achieved with InferSent and SentenceBERT.

References

  1. W. N. Borst, “Construction of engineering ontologies for knowledge sharing and reuse,” Enschede Centre for Telematics and Information Technology (CTIT), Tech. Rep., 1997.
  2. F. Al-Aswadi, H. Chan, and K. Gan, “Extracting semantic concepts and relations from scientific publications by using deep learning,” in Proceedings of IRICT 2020, 2021, pp. 374–383.
  3. N. Gupta, S. Podder, K. Annervaz, and S. Sengupta, “Domain ontology induction using word embeddings,” in ICMLA, 2016, pp. 115–119.
  4. A. Katyshev, A. Anikin, M. Denisov, and T. Petrova, “Intelligent approaches for the automated domain ontology extraction,” in International Congress on Information and Communication Technology, 2021, pp. 410–417.
  5. S. Althubaiti, S. Kafkas, M. Abdelhakim, and R. Hoehndorf, “Combining lexical and context features for automatic ontology extension,” Journal of Biomedical Semantics, vol. 11, p. article no. 1, 2020.
  6. L. Espinoza-Anke, F. Ronzano, and H. Saggion, “Hypernym extraction: Combining machine-learning and dependency grammar,” in CICLing, 2015, pp. 372–383.
  7. F. Martel and A. Zouaq, “Taxonomy extraction using knowledge graph embeddings and hierarchical clustering,” in SAC’21, 2021, pp. 836–844.
  8. N.-A. R., J. R., and J. Castro, “Automated ontology extraction from unstructured texts using deep learning,” in Intuitionistic and Type-2 Fuzzy Logic Enhancements in Neural and Optimization Algorithms: Theory and Applications. Springer, 2020, pp. 727–755.
  9. A. Bento, A. Zouaq, and M. Gagnon, “Ontology matching using convolutional neural networks,” in LREC, 2020, pp. 5648–5653.
  10. J. Chakraborty, B. Yaman, L. Virgili, K. Konar, and S. Bansal, “Onto-Connect: Results for OAEI 2020,” in OM ISWC, 2020, pp. 204–210.
  11. L. Hao, C. Lei, V. Efthymiou, A. Quamar, F. Özcan et al., “MEDTO: Medical data to ontology matching using hybrid graph neural networks,” in KDD’21, 2021, pp. 2946–2954.
  12. J. Wu, J. Lv, H. Guo, and S. Ma, “Daeom: A deep attentional embedding approach for biomedical ontology matching,” Applied Sciences, vol. 10, p. article no. 7909, 2020.
  13. A. Benarab, G. Yin, and F. Rafique, “An embedding-based approach to the ontology matching problem,” in 2023 IEEE 4th International Conference on Pattern Recognition and Machine Learning (PRML), 2023. https://dx.doi.org/10.1109/PRML59573.2023.10348247 pp. 642–649.
  14. S. Kolozali, G. Fazekas, M. Barthet, and M. Sandler, “A framework for automatic ontology generation based on semantic audio analysis,” in Audio Engineering Society International Conference, 2014, pp. 87–96.
  15. M. Hourali and G. Montazer, “Using ART2 neural network and bayesian network for automating the ontology constructing process,” Procedia Engineering, vol. 29, pp. 3914–3923, 2012.
  16. C. Mercier, H. Chateau-Laurent, F. Alexandre, and T. ViĂ©ville, “Ontology as neuronal-space manifold: Towards symbolic and numerical artificial embedding,” in Workshop on Knowledge Representation for Hybrid and Compositional AI, 2021, pp. 1–11.
  17. G. Li, “CNN based ontology learning algorithm and applied in PE data,” IAENG International Journal of Computer Science, vol. 48, pp. 1–8, 2021.
  18. R. Mueller and S. Abdullaev, “Deep cause: Hypothesis extraction from information systems papers with deep learning for theory ontology learning,” in Annual Hawaii International Conference on System Sciences, 2019, pp. 6250–6259.
  19. G. Petrucci, M. Rospocher, and C. Ghindini, “Expressive ontology learning as neural machine translation,” Journal of Web Semantics, vol. 52–53, pp. 66–82, 2018.
  20. J. Potoniec, “Learning OWL 2 property characteristics as an explanation for an RNN,” Bulletin of the Polish Academy of Sciences, Technical Sciences, vol. 68, pp. 1481–1490, 2020.
  21. A. Memariani, M. Glauer, F. Neuhaus, T. Mossakowski, and J. Hatings, “Automated and explainable ontology extension based on deep learning: A case study in the chemical domain,” in 3rd International Workshop on Data Meets Applied Ontologies, 2021, pp. 1–16.
  22. A. Oba, I. Paik, and A. Kuwana, “Automatic classification for ontology generation by pretrained language model,” in International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, 2021, pp. 210–221.
  23. N. Teslya and S. Savosin, “Matching ontologies with Word2Vec-based neural network,” in ICCSA, 2019, pp. 745–756.
  24. P. Ristoski and H. Paulheim, “Rdf2vec: Rdf graph embeddings for data mining,” in International Semantic Web Conference, 2016, pp. 498–514.
  25. A. Ritchie, J. Chen, L. Castro, D. Rebholz-Schuhmann, and E. JimĂ©nez-Ruiz, “Ontology clustering with OWL2Vec,” in DeepOntoNLP, 2021, pp. 54–61.
  26. S. Teymurova, E. Jiménez-Ruiz, T. Weyde, and J. Chen, “Owl2vec4oa: Tailoring knowledge graph embeddings for ontology alignment,” in Sixth International Knowledge Graph and Semantic Web Conference. Springer International Publishing, October 2024. ISSN 0302-9743. [Online]. Available: https://openaccess.city.ac.uk/id/eprint/34052/
  27. U. Yorsh, A. S. Behr, N. Kockmann, and M. Holeňa, “Text-to-ontology mapping via natural language processing models,” in Conference on Theory and Practice of Information Technologies, 2022. [Online]. Available: https://api.semanticscholar.org/CorpusID:253161694
  28. G. Petrucci, C. Ghindini, and M. Rospocher, “Ontology learning in the deep,” in EKAW, 2016, pp. 480–495.
  29. M. A. Musen, “The protégé project: a look back and a look forward,” AI Matters, vol. 1, no. 4, pp. 4–12, 2015. https://dx.doi.org/10.1145/2757001.2757003. [Online]. Available: https://doi.org/10.1145/2757001.2757003
  30. J.-B. Lamy, “Owlready: Ontology-oriented programming in python with automatic classification and high level constructs for biomedical ontologies,” Artificial Intelligence In Medicine, vol. 80, pp. 11–28, 2017.
  31. J. Pennington, R. Socher, and C. Manning, “GloVe: Global vectors for word representation,” in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), A. Moschitti, B. Pang, and W. Daelemans, Eds. Doha, Qatar: Association for Computational Linguistics, Oct. 2014. https://dx.doi.org/10.3115/v1/D14-1162 pp. 1532–1543. [Online]. Available: https://aclanthology.org/D14-1162
  32. A. Conneau, D. Kiela, H. Schwenk, L. Barrault, and A. Bordes, “Supervised learning of universal sentence representations from natural language inference data,” in Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, M. Palmer, R. Hwa, and S. Riedel, Eds. Copenhagen, Denmark: Association for Computational Linguistics, Sep. 2017. https://dx.doi.org/10.18653/v1/D17-1070 pp. 670–680. [Online]. Available: https://aclanthology.org/D17-1070
  33. A. M. Dai, C. Olah, and Q. V. Le, “Document embedding with paragraph vectors,” ArXiv, vol. abs/1507.07998, 2015. [Online]. Available: https://api.semanticscholar.org/CorpusID:7364264
  34. T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, “Distributed representations of words and phrases and their compositionality,” in Advances in Neural Information Processing Systems, C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Weinberger, Eds., vol. 26. Curran Associates, Inc., 2013. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2013/file\/9aa42b31882ec039965f3c4923ce901b-Paper.pdf
  35. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pretraining of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), J. Burstein, C. Doran, and T. Solorio, Eds. Minneapolis, Minnesota: Association for Computational Linguistics, Jun. 2019. https://dx.doi.org/10.18653/v1/N19-1423 pp. 4171–4186. [Online]. Available: https://aclanthology.org/N19-1423
  36. Meta, “Llama-models/models/llama3_1 at meta-llama/llama-models,” Jul 2024. [Online]. Available: https://github.com/meta-llama/llama-models/tree/main/models/llama3_1
  37. P. BehnamGhader, V. Adlakha, M. Mosbach, D. Bahdanau, N. Chapados, and S. Reddy, “LLM2Vec: Large language models are secretly powerful text encoders,” in First Conference on Language Modeling, 2024. [Online]. Available: https://openreview.net/forum?id=IW1PR7vEBf
  38. J. Bromley, J. Bentz, L. Bottou, I. Guyon, Y. Lecun, C. Moore, E. Sackinger, and R. Shah, “Signature verification using a "siamese" time delay neural network,” International Journal of Pattern Recognition and Artificial Intelligence, vol. 7, p. 25, 08 1993. https://dx.doi.org/10.1142/S0218001493000339
  39. J. Whang and A. Matsukawa, “Exploring batch normalization in recurrent neural networks,” Stanford Center for Professional Development, 2019, accessed on 29 February 2024. [Online]. Available: https://jaywhang.com/assets/batchnorm_rnn.pdf
  40. L. Korel, A. S. Behr, M. Holeňa, and N. Kockmann, “Using artificial neural networks to determine ontologies most relevant to scientific texts,” in Proceedings of the 22nd Conference Information Technologies – Applications and Theory (ITAT 2022), ser. CEUR Workshop Proceedings, vol. 3226. CEUR-WS.org, 2022. ISSN 1613-0073 pp. 44–54. [Online]. Available: https://ceur-ws.org/Vol-3226/paper5.pdf
  41. R. A. Company, “Bert for chemical industry,” https://huggingface.co/recobo/chemical-bert-uncased, 2022.
  42. A. Benavoli, G. Corani, and F. Mangili, “Should we really use post-hoc tests based on mean-ranks?” Journal of Machine Learning Research, vol. 17, pp. 1–10, 2016.