Logo PTI Logo FedCSIS

Position Papers of the 17th Conference on Computer Science and Intelligence Systems

Annals of Computer Science and Information Systems, Volume 31

Evaluation without Ground Truth: a Comparative Study on Preference Mining Techniques in Twitter Social Network

DOI: http://dx.doi.org/10.15439/2022F280

Citation: Position Papers of the 17th Conference on Computer Science and Intelligence Systems, M. Ganzha, L. Maciaszek, M. Paprzycki, D. Ślęzak (eds). ACSIS, Vol. 31, pages 65–68 ()

Full text

Abstract. In social media research the lack of ground truth for evaluation is a recurrent problem. We study the preference mining task in Twitter network which suffers from this lack of ground truth problem. We implement three different methods from literature, considering a common preference domain of news and carry a comparative study among them. Our preliminary findings show that is possible to combine methods in order to avoid unfeasible user surveying baselines and enable the evaluation of techniques. In the future, our target is to completely eliminate ground truth sets and evaluate based on correlation and causality techniques.

References

  1. R. Zafarani and H. Liu, “Evaluation without ground truth in social media research,” Com. ACM, vol. 58, no. 6, pp. 54–60, 2015.
  2. J. Furnkranz and E. Hullermeier, Preference Learning. Springer, New York, 2010.
  3. D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent dirichlet allocation,” J. Mach. Learn. Res., vol. 3, pp. 993–1022, Mar. 2003.
  4. F. S. F. Pereira, S. de Amo, and J. Gama, “Detecting events in evolving social networks through node centrality analysis,” Large-scale Learning from Data Streams in Evolving Environments with ECML/PKDD, 2016.
  5. F. S. F. Pereira, S. de Amo, and J. Gama, “On using temporal networks to analyze user preferences dynamics,” in Discovery Science: 19th International Conference, DS 2016, Bari, Italy, 2016., 2016.
  6. X. Liu, “Modeling users’ dynamic preference for personalized recommendation,” in Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI’15), 2015, pp. 1785–1791.
  7. F. S. F. Pereira and S. de Amo, “Mineracao de preferencias do usuario em textos de redes sociais usando sentencas comparativas,” in Symposium on Knowledge Discovery, Mining and Learning (KDMiLe), 2015, pp. 94–97.
  8. M. A. Abbasi, J. Tang, and H. Liu, “Scalable learning of users’ preferences using networked data,” in Proceedings of the 25th ACM Conference on Hypertext and Social Media, ser. HT ’14. New York, NY, USA: ACM, 2014, pp. 4–12.
  9. H. Al-Jarrah, M. Al-Asa’d, S. A. Al-Zboon, S. K. Tawalbeh, M. M. Hammad, and M. AL-Smadi, “Resolving conflict of interests and recommending expert reviewers for academic publications using linked open data,” in 2019 Sixth International Conference on Social Networks Analysis, Management and Security (SNAMS), 2019, pp. 91–98.
  10. T. Elsaleh, S. Enshaeifar, R. Rezvani, S. T. Acton, V. Janeiko, and M. Bermudez-Edo, “Iot-stream: A lightweight ontology for internet of things data streams and its use with data analytics and event detection services,” Sensors, vol. 20, no. 4, 2020. [Online]. Available: https://www.mdpi.com/1424-8220/20/4/953
  11. T. R. Tangherlini, S. Shahsavari, B. Shahbazi, E. Ebrahimzadeh, and V. Roychowdhury, “An automated pipeline for the discovery of conspiracy and conspiracy theory narrative frameworks: Bridgegate, pizzagate and storytelling on the web,” PLOS ONE, vol. 15, no. 6, pp. 1–39, 06 2020. [Online]. Available: https://doi.org/10.1371/journal.pone.0233879