Logo PTI
Polish Information Processing Society
Logo FedCSIS

Annals of Computer Science and Information Systems, Volume 21

Proceedings of the 2020 Federated Conference on Computer Science and Information Systems

Age-related Spike Timing Dependent Plasticity of Brain-inspired Model of Visual Information Processing with Reinforcement Learning

,

DOI: http://dx.doi.org/10.15439/2020F141

Citation: Proceedings of the 2020 Federated Conference on Computer Science and Information Systems, M. Ganzha, L. Maciaszek, M. Paprzycki (eds). ACSIS, Vol. 21, pages 93100 ()

Full text

Abstract. The paper summarizes our efforts to develop a spike timing neural network model of dynamic visual information processing and decision making inspired by the available knowledge about how the human brain performs this complicated task. It consists of multiple layers with functionality corresponding to the main visual information processing structures starting from the early level of the visual system up to the areas responsible for decision making based on accumulated sensory evidence as well as the basal ganglia modulation due to the feedback from the environment. In the present work, we investigated age-related changes in the spike timing dependent plastic synapses of the model as a result of reinforcement learning.

References

  1. A. G. Barto, R. S. Sutton and C. W. Anderson, C.W., "Neuronlike adaptive elements that can solve difficult learning control problems," IEEE Trans. on Systems, Man, and Cybernetics, vol. 13 (5), 1983, pp. 834-846. http://dx.doi.org/10.1109/TSMC.1983.6313077
  2. M. N. Shadlen and W. T. Newsome, "Motion perception: seeing and deciding," Proc. Natl. Acad. Sci. USA, vol. 93 (2), pp. 628-633, 1996. http://dx.doi.org/10.1073/pnas.93.2.628
  3. D. M. Herz, B. A. Zavala, R. Bogacz and P. Brown, "Neural correlates of decision thresholds in the human subthalamic nucleus," Current Biology, vol. 26 (7), pp. 916-920, 2016. http://dx.doi.org/10.1016/j.cub.2016.01.051
  4. K. Dunovan, B. Lynch, T. Molesworth and T. Verstynen, T., "Competing basal-ganglia pathways determine the difference between stopping and deciding not to go," eLife, vol. 4, Article number e08723, 2015. DOI: 10.7554/eLife.08723
  5. A. G. Barto, "Adaptive critics and the basal ganglia," in J. C. Houk, J. L. Davis and D. G. Beiser, Editors, Models of Information Processing in the Basal Ganglia, MIT Press, Cambridge, MA; 1995, pp. 215-232.
  6. D. Joel, Y. Niv and E. Ruppin, "Actor-critic models of the basal ganglia: new anatomical and computational perspectives," Neural Networks, vol. 15, pp. 535-547, 2002. http://dx.doi.org/10.1016/S0893-6080(02)00047-3
  7. M. J. Frank, L. C. Seeberger and R. C. O’Reilly, "By carrot or by stick: cognitive reinforcement learning in Parkinsonism," Science, vol. 306 (5703), pp. 1940-1943, 2004. http://dx.doi.org/10.1126/science.1102941
  8. R. Bogacz and T. Larsen, T., "Integration of reinforcement learning and optimal decision-making theories of the basal ganglia,", Neural Computation, vol. 23 (4), pp. 817-851, 2011. http://dx.doi.org/10.1162/NECO_a_00103
  9. K. Dunovan and T. Verstynen, "Believer-Skeptic meets actor-critic: Rethinking the role of basal ganglia pathways during decision-making and reinforcement learning,", Frontiers in Neuroscience, vol. 10, Article number 106, 2016. http://dx.doi.org/10.3389/fnins.2016.00106
  10. P. Koprinkova-Hristova and N. Bocheva, "Spike timing neural model of eye movement motor response with reinforcement learning," Lecture Notes in Computer Science, in press.
  11. J. Igarashi, O. Shounob, T. Fukai and H. Tsujino, "Real-time simulation of a spiking neural network model of the basal ganglia circuitry using general purpose computing on graphics processing units," Neural Networks, vol. 24, pp. 950-960, 2011. http://dx.doi.org/10.1016/j.neunet.2011.06.008
  12. R. Krishnan, S. Ratnadurai, D. Subramanian, V. S. Chakravarthy and M. Rengaswamyd, "Modeling the role of basal ganglia in saccade generation: Is the indirect pathway the explorer?," Neural Networks, vol. 24, pp. 801-813, 2011. http://dx.doi.org/10.1016/j.neunet.2011.06.002
  13. S. Kunkel et al.,"NEST 2.12.0," Zenodo, 2017. http://dx.doi.org/10.5281/zenodo.259534
  14. P. Koprinkova-Hristova, N. Bocheva, S. Nedelcheva, M. Stefanova, B. Genova, R. Kraleva and V. Kralev, "STDP plasticity in TRN within hierarchical spike timing model of visual information processing," IFIP Advances in Information and Communication Technology, vol. 583 IFIP, pp. 279-290, 2020. http://dx.doi.org/10.1007/978-3-030-49161-1_24
  15. P. Koprinkova-Hristova, N. Bocheva, S. Nedelcheva and M. Stefanova, "Spike timing neural model of motion perception and decision making," Frontiers in Computational Neuroscience, vol. 13, Article number 20, 2019. http://dx.doi.org/10.3389/fncom.2019.00020
  16. P. Koprinkova-Hristova, N. Bocheva and S. Nedelcheva, "Investigation of feedback connections effect of a spike timing neural network model of early visual system, " in Innovations in Intelligent Systems and Applications (INISTA), Thessaloniki, Greece, 2018, http://dx.doi.org/10.1109/INISTA.2018.8466292
  17. S. Nedelcheva and P. Koprinkova-Hristova, "Orientation selectivity tuning of a spike timing neural network model of the first layer of the human visual cortex," Studies in Computational Intelligence, vol. 793, pp. 291-303, 2019. http://dx.doi.org/10.1007/978-3-319-97277-0_24
  18. T. W. Troyer, A. E. Krukowski, N. J. Priebe and K. D. Miller, "Contrast invariant orientation tuning in cat visual cortex: thalamocortical input tuning and correlation-based intracortical connectivity," J. Neurosci., vol. 18, pp. 5908-5927, 1998. http://dx.doi.org/10.1523/jneurosci.18-15-05908.1998
  19. J. Kremkow, L. U. Perrinet, C. Monier, J.-M. Alonso, A. Aertsen, Y. Fregnac and G. S. Masson, "Push-pull receptive field organization and synaptic depression: Mechanisms for reliably encoding naturalistic stimuli in V1," Frontiers in Neural Circuits, vol. 10, Article number 37, 2016. http://dx.doi.org/10.3389/fncir.2016.00037
  20. A. Casti, F. Hayot, Y. Xiao and E. Kaplan, "A simple model of retina-LGN transmission," J. Computational Neuroscience, vol. 24, pp. 235-252, 2008. http://dx.doi.org/10.1007/s10827-007-0053-7
  21. M. Ghodratia, S.-M. Khaligh-Razavic and S. R. Lehky, "Towards building a more complex view of the lateral geniculate nucleus: Recent advances in understanding its role," Progress in Neurobiology, vol. 156, pp. 214-255, 2017. http://dx.doi.org/10.1016/j.pneurobio.2017.06.002
  22. P. Gleeson, R. Martinez and A. Davison, "Network models of V1," Open Source Brain, http://www.opensourcebrain.org/projects/111.
  23. S. Sadeh and S. Rotter, "Statistics and geometry of orientation selectivity in primary visual cortex," Biol. Cybern., vol. 108, pp. 631-653, 2014. http://dx.doi.org/10.1007/s00422-013-0576-0
  24. M.-J. Escobar, G. S. Masson, T. Vieville and P. Kornprobst, "Action recognition using a bio-inspired feedforward spiking network," Int. J. Comput. Vis., vol. 82, pp. 284-301, 2009. http://dx.doi.org/10.1007/s11263-008-0201-1
  25. O. W. Layton and B. R. Fajen, "Possible role for recurrent interactions between expansion and contraction cells in MSTd during self-motion perception in dynamic environments," Journal of Vision, vol. 17 (5), Article number 5, 2017. http://dx.doi.org/10.1167/17.5.5
  26. W. Potjans, A. Morrison and M. Diesmann, "Enabling functional neural circuit simulations with distributed computing of neuromodulated plasticity," Front. in Comp. Neuroscience, vol. 4, 2010. http://dx.doi.org/10.3389/fncom.2010.00141
  27. J. L. Plotkin and L. A. Goldberg, "Thinking outside the box (and arrow): Current themes in striatal dysfunction in movement disorders," The Neuroscientist, vol. 25 (4), pp. 359-379, 2019. http://dx.doi.org/10.1177/1073858418807887
  28. W. Wei, J. E. Rubin and X.-J. Wang, "Role of the indirect pathway of the basal ganglia in perceptual decision making," The Journal of Neuroscience, vol. 35 (9), pp. 4052-4064, 2015. http://dx.doi.org/10.1523/JNEUROSCI.3611-14.2015
  29. H. Yan and J. Wang, "Quantification of motor network dynamics in Parkinson’s disease by means of landscape and flux theory," PLoS ONE, vol. 12 (3), Article number e0174364, 2017. http://dx.doi.org/10.1371/journal.pone.0174364
  30. M. Tsodyks, A. Uziel and H. Markram, "Synchrony generation in recurrent networks with frequency-dependent synapses," The Journal of Neuroscience, vol. 20 (1), pp. RC50, 2000. http://dx.doi.org/10.1523/jneurosci.20-01-j0003.2000
  31. N. Bocheva, B. Genova and M. Stefanova, "Drift diffusion modeling of response time in heading estimation based on motion and form cues," Int. J. of Biology and Biomedical Engineering, vol. 12, pp. 75-83, 2018.