admin

Issue 1 (195), article 1

DOI:https://doi.org/10.15407/kvt195.01.005

Kibern. vyčisl. teh., 2019, Issue 1 (195), pp.

Grytsenko V.I., Corresponding Member of NAS of Ukraine,
Director of International Research and Training
Center for Information Technologies and Systems
of the National Academy of Sciences of Ukraine
and Ministry of Education and Science of Ukraine
e-mail: vig@irtc.org.ua

Volkov A.E., Acting Head of Department,
Intelligent Control Department,
e-mail: alexvolk@ukr.net

Komar N.N., Researcher,
Intelligent Control Department,
e-mail: nickkomar08@gmail.com

Shepetukha Yu.M., PhD (Engineering)
Leading Researcher,
Intelligent Control Department,
e-mail: dep185@irtc.org

Voloshenyuk D.A., Researcher,
Intelligent Control Department,
e-mail: dep185@irtc.org

International Research and Training Center for Information Technologies
and Systems of the National Academy of Sciences of Ukraine
and Ministry of Education and Science of Ukraine,
Acad. Glushkov av., 40, Kiev, 03187, Ukraine

INTEGRAL-ADAPTIVE AUTOPILOT AS A MEANS OF INTELLECTUALIZING A MODERN UNMANNED AERIAL VEHICLE

Introduction. At present unmanned aerial vehicles (UAVs) are successfully used in various industries in performing scientific and engineering, economical, military and a number of other missions. Effectiveness of their functioning is mainly determined by an onboard suit of hardware and software of a UAV’s control system. The process of the existing autopilot systems enhancement is intended to broaden the range of UAV’s tasks without direct human involvement and introduce additional smart functions into autopilot operation.

Purpose. The aim of research is to study the modern algorithms used in autopilots of unmanned aerial vehicles and formulation of the problem of development and usage of new intellectual methods for automatic control systems.

Results. The approach considered in the article is based on the theory of high-precision remote control of dynamic objects and on the complex interaction of methods of theory of invariance, adaptive control and intellectualization of processes of UAV control.

One of the features of the proposed method of intellectual control for unmanned aerial vehicle autopilot is the procedure of transforming a multi-dimensional system into an aggregate of virtual autonomous processes, for each of which the control algorithm is easily generated by an autonomous subsystem. Coming up next is the procedure of coordination of actions of all the autonomous systems into single functioning complex. This provides an opportunity to improved precision and sustainability of control.

Conclusion. Using the method described in the article allows creating integral and adaptive autopilots to perform complicated spatial maneuvering an unmanned aerial vehicle being based on usage of full non-linear models without simplifications and linearization.

Keywords: unmanned aerial vehicle, control system, virtual control, adaptation invariance.

Download full text!

REFERENCES

1 Fahlstrom P., Gleason T. Introduction to UAV systems. Hoboken: Wiley, 2012. 4th ed. 308 p. https://doi.org/10.1002/9781118396780

2 Moiseyev V.S. Applied theory of unmanned aerial vehicles control. Kazan: GBU Republican centre for monitoring education quality, 2013.p. 768. (in Russian).

3 Beard R.W., McLain T.W. Small Unmanned Aircraft: Theory and Practice. Princeton: Princeton Univ. Press, 2012. 320 p. https://doi.org/10.1515/9781400840601

4 Chao H., Cao Y., and Chen Y.Q. Autopilots for small unmanned aerial vehicles: a survey. International Journal of Control, Automation, and Systems, 2010, vol. 8, N 1. P. 36-44. https://doi.org/10.1007/s12555-010-0105-z

5 Feng G. A survey on analysis and design of model-based fuzzy control systems. IEEE Transactions on Fuzzy Systems, 2006, vol. 14, N 5. P. 676 – 697. https://doi.org/10.1109/TFUZZ.2006.883415

6 Shilov K.Ye. Development of unmanned air vehicle automatic control system of a rotorcraft. Works of MFTI, 2014. N 4 . P. 139-152. (in Russian).

7 Calise A., Rysdyk R. Nonlinear Adaptive Flight Control Using Neural Networks. Control Systems Magazine, 1998, vol. 18, N. 6. P. 14-25. https://doi.org/10.1109/37.736008

8 Johnson E.N., Kannan S.K. Adaptive Flight Control for an Autonomous Unmanned Helicopter. AIAA Guidance, Navigation, and Control Conference and Exhibit. Monterey, California, August, 2002. https://doi.org/10.2514/6.2002-4439

9 Lopez J., Dormido R., Gomez J.P., Dormido S., Diaz J.M. Comparison of H-infinity with QFT applied to an Altitude Command Tracker for an UAV. Proc. of the European Control Conference (2 – 5th of July, 2007, Kos, Greece) Kos, Greece, 2007. P. 46-54. https://doi.org/10.23919/ECC.2007.7068482

10 Lopez J., Dormido R., Dormido S., Gomez J.P. A Robust Controller for an UAV Flight Control System. The Scientific World Journal. 2015. vol. 2015. P. 15-26. https://doi.org/10.1155/2015/403236

11 Ross T.J. Fuzzy Logic with Engineering Applications, 2nd Edition. NY: Wiley, 2004. 228 p.

12 Kumon M., Udo Y., Michihira H., Nagata M., Mizumoto I., Iwai Z. Autopilot system for kiteplane. IEEE/ASME Transactions on Mechatronics. 2006. vol. 11. N 5. P. 615-624. https://doi.org/10.1109/TMECH.2006.882994

13 Albus J.S. On intelligence and its dimensions. Technical report of the ISIS (Interdisciplinary studies of intelligent systems) group N. ISIS 94-001. University of Notre Dame, 1994. P. 11-13.

14 Antsaklis P.J. On autonomy and intelligence in control. Technical report of the ISIS (Interdisciplinary studies of intelligent systems) group N. ISIS 94-001. University of Notre Dame, 1994. P. 14-18.

15 Grytsenko V.I., Volkov O.E., Komar M.M., Bogachuk Yu.P. Intellectualization of the modern automatic control systems for unmanned aerial vehicles. Kibernetika i vycislitelnaa tehnika. 2018. N 1 (191). P. 45-59. (in Ukrainian) https://doi.org/10.15407/kvt191.01.045

16 Pavlov V.V., Pavlova S.V. Intellectual control of complex non-linear dynamic systems. Kiev: Naukova dumka. 2015. 216 p. (in Russian).

17 Kharchenko V.P., Chepizhenko V.I., Tounik A.A., Pavlova S.V. Unmanned aerial vehicles avionics. Kiev: TOV Abris-print, 2012. 464 p. (in Ukrainian).

18 Bodner V.A. Air vehicle control systems. Moscow: Mashinostroyeniye, 1973. 501 p. (in Russian).

Received 19.11.2018

admin

Issue 4 (194), article 1

DOI:https://doi.org/10.15407/kvt194.04.007

Kibern. vyčisl. teh., 2018, Issue 4 (194), pp.

Grytsenko V.I., Corresponding Member of NAS of Ukraine,
Director of International Research and Training
Center for Information Technologies and Systems
of the National Academy of Sciences of Ukraine
and Ministry of Education and Science of Ukraine
e-mail: vig@irtc.org.ua

Rachkovskij D.A., DSc (Engineering), Leading Researcher
Dept. of Neural Information Processing Technologies
e-mail: dar@infrm.kiev.ua

Revunova E.G., PhD (Engineering), Senior Researcher
Dept. of Neural Information Processing Technologies
e-mail: egrevunova@gmail.com

International Research and Training Center for Information Technologies
and Systems of the National Academy of Sciences of Ukraine
and Ministry of Education and Science of Ukraine,
Acad. Glushkov av., 40, Kiev, 03187, Ukraine

NEURAL DISTRIBUTED REPRESENTATIONS OF VECTOR DATA IN INTELLIGENT INFORMATION TECHNOLOGIES

Introduction. Distributed representation (DR) of data is a form of a vector representation, where each object is represented by a set of vector components, and each vector component can belong to representations of many objects. In ordinary vector representations, the meaning of each component is defined, which cannot be said about DR. However, the similarity of RP vectors reflects the similarity of the objects they represent.
DR is a neural network approach based on modeling the representation of information in the brain, resulted from ideas about a “distributed” or “holographic” representations. DRs have a large information capacity, allow the use of a rich arsenal of methods developed for vector data, scale well for processing large amounts of data, and have a number of other advantages. Methods for data transformation to DRs have been developed for data of vari-ous types – from scalar and vector to graphs.

The purpose of the article is to provide an overview of part of the work of the Department of Neural Information Processing Technologies (International Center) in the field of neural network distributed representations. The approach is a development of the ideas of Nikolai Amosov and his scientific school of modeling the structure and functions of the brain.

Scope. The formation of distributed representations from the original vector representations of objects using random projection is considered. With the help of the DR, it is possible to efficiently estimate the similarity of the original objects represented by numerical vectors. The use of DR allows developing regularization methods for obtaining a stable solution of discrete ill-posed inverse problems, increasing the computational efficiency and accuracy of their solution, analyzing analytically the accuracy of the solution. Thus DRs allow for in-creasing the efficiency of information technologies applying them.

Conclusions. DRs of various data types can be used to improve the efficiency and intelligence level of information technologies. DRs have been developed for both weakly structured data, such as vectors, and for complex structured representations of objects, such as sequences, graphs of knowledge-base situations (episodes), etc. Transformation of different types of data into the DR vector format allows unifying the basic information technologies of their processing and achieving good scalability with an increase in the amount of data processed.
In future, distributed representations will naturally combine information on structure and semantics to create computationally efficient and qualitatively new information technologies in which the processing of relational structures from knowledge bases is performed by the similarity of their DRs. The neurobiological relevance of distributed representations opens up the possibility of creating intelligent information technologies based on them that func-tion similarly to the human brain.

Keywords: distributed data representation, random projection, vector similarity estimation, discrete ill-posed problem, regularization.

Download full text!

REFERENCES

1. Amosov N. M. Modelling of thinking and the mind. New York: Spartan Books, 1967. 192 p. https://doi.org/10.1007/978-1-349-00640-3

2. Amosov N.M., Baidyk T.N., Goltsev A.D., Kasatkin A.M., Kasatkina L.M., Rachkovskij D.A. Neurocomputers and Intelligent Robots. Kyiv: Nauk. Dumka. 1991. 269 p.(in Russian)

3. Gritsenko V.I., Rachkovskij D.A., Goltsev A.D., Lukovych V.V., Misuno I.S., Revunova E.G., Slipchenko S.V., Sokolov A.M., Talayev S.A. Neural distributed representation for intelligent information technologies and modeling of thinking. Kibernetika i vycislitelnaa tehnika. 2013. Vol. 173. P. 7–24. (in Russian)

4. Goltsev A.D., Gritsenko V.I. Neural network technologies in the problem of handwriting recognition. Control Systems and Machines. 2018. N 4. P. 3–20. (in Russian).

5. Kussul E.M. Associative neuron-like structures. Kyiv: Nauk. Dumka. 1992. 144 p. (in Russian)

6. Kussul E.M., Rachkovskij D.A., Baidyk T.N. Associative-Projective Neural Networks: Architecture, Implementation, Applications. Proc. Neuro-Nimes’91. (Nimes, 25–29th of Oct. 25–29, 1991). Nimes, 1991. P. 463–476.

7. Gayler R. Multiplicative binding, representation operators, and analogy. Advances in Analogy Research: Integration of Theory and Data from the Cognitive, Computational, and Neural Sciences. Edited by K. Holyoak, D. Gentner, and B. Kokinov. Sofia, Bulgaria: New Bulgarian University, 1998. P. 405.

8. Kanerva P. Hyperdimensional computing: An introduction to computing in distributed representation with high-dimensional random vectors. Cognitive Computation. 2009. Vol. 1, N 2. P. 139–159. https://doi.org/10.1007/s12559-009-9009-8

9. Goltsev A., Husek D. Some properties of the assembly neural networks. Neural Network World. 2002. Vol. 12, N. 1. P. 15–32.

10. Goltsev A.D. Neural networks with assembly organization. Kyiv: Nauk. Dumka. 2005. 200 p. (in Russian)

11. Goltsev A., Gritsenko V. Modular neural networks with radial neural columnar architecture. Biologically Inspired Cognitive Architectures. 2015. Vol. 13. P. 63–74. https://doi.org/10.1016/j.bica.2015.06.001

12. Frolov A.A., Rachkovskij D.A., Husek D. On information characteristics of Willshaw-like auto-associative memory. Neural Network World. 2002. Vol. 12, No 2. P. 141–158.

13. Frolov A.A., Husek D., Rachkovskij D.A. Time of searching for similar binary vectors in associative memory. Cybernetics and Systems Analysis. 2006. Vol. 42, N 5. P. 615–623. https://doi.org/10.1007/s10559-006-0098-z

14. Gritsenko V.I., Rachkovskij D.A., Frolov A.A., Gayler R., Kleyko D., Osipov E. Neural distributed autoassociative memories: A survey. Kibernetika i vycislitel`naa tehnika. 2017. N 2 (188). P. 5–35.

15. Li P., Hastie T.J., Church K.W. Very sparse random projections. Proc. KDD’06. (Philadelphia, 20 – 23th of Aug.). Philadelphia, 2006. P. 287–296. https://doi.org/10.1145/1150402.1150436

16. Rachkovskij D.A. Vector data transformation using random binary matrices. Cybernetics and Systems Analysis. 2014. Vol. 50, N 6. P. 960–968. https://doi.org/10.1007/s10559-014-9687-4

17. Rachkovskij D.A. Formation of similarity-reflecting binary vectors with random binary projections. Cybernetics and Systems Analysis. 2015. Vol. 51, N 2. P. 313–323. https://doi.org/10.1007/s10559-015-9723-z

18. Rachkovskij D.A. Estimation of vectors similarity by their randomized binary projections. Cybernetics and Systems Analysis. 2015. Vol. 51, N 5. P. 808–818. https://doi.org/10.1007/s10559-015-9774-1

19. Revunova E.G., Rachkovskij D.A. Using randomized algorithms for solving discrete ill-posed problems. Intern. Journal Information Theories and Applications. 2009. Vol. 16, N 2. P. 176–192.

20. Durrant R.J., Kaban A. Random projections as regularizers: learning a linear discriminant from fewer observations than dimensions. Machine Learning. 2015. Vol. 99, N 2. P. 257–286. https://doi.org/10.1007/s10994-014-5466-8

21. Xiang H., Zou J. Randomized algorithms for large-scale inverse problems with general Tikhonov regularizations. Inverse Problems. 2015. Vol. 31, N 8: 085008. P. 1–24.

22. Revunova E.G. Study of error components for solution of the inverse problem using random projections. Mathematical Machines and Systems. 2010. N 4. P. 33–42 (in Russian).

23. Rachkovskij D.A., Revunova E.G. Randomized method for solving discrete ill-posed problems. Cybernetics and Systems Analysis. 2012. Vol. 48, N. 4. P. 621–635. https://doi.org/10.1007/s10559-012-9443-6

24. Revunova E.G. Randomization approach to the reconstruction of signals resulted from indirect measurements. Proc. ICIM’13 (Kyiv 16–20th of Sept., 2013). Kyiv, 2013. P. 203–208.

25. Revunova E.G. Analytical study of the error components for the solution of discreteill-posed problems using random projections. Cybernetics and Systems Analysis. 2015. Vol. 51, N. 6. P. 978–991. https://doi.org/10.1007/s10559-015-9791-0

26. Revunova E.G. Model selection criteria for a linear model to solve discrete ill-posed problems on the basis of singular decomposition and random projection. Cybernetics and Systems Analysis. 2016. Vol. 52, N.4. P. 647–664. https://doi.org/10.1007/s10559-016-9868-4

27. Revunova E.G. Averaging over matrices in solving discrete ill-posed problems on the basis of random projection. Proc. CSIT’17 (Lviv 05–08th of Sept., 2017). Lviv, 2017. Vol. 1. P. 473–478. https://doi.org/10.1109/STC-CSIT.2017.8098831

28. Revunova E.G. Solution of the discrete ill-posed problem on the basis of singular value decomposition and random projection. Advances in Intelligent Systems and Computing II. Cham: Springer. 2018. P. 434–449.

29. Hansen P. Rank-deficient and discrete ill-posed problems. Numerical aspects of linear inversion. Philadelphia: SIAM, 1998. 247 p. https://doi.org/10.1137/1.9780898719697

30. Nowicki D., Verga P., Siegelmann H. Modeling reconsolidation in kernel associative memory. PLoS ONE. 2013. Vol. 8(8): e68189. doi:10.1371/journal.pone.0068189. https://doi.org/10.1371/journal.pone.0068189

31. Nowicki D, Siegelmann H. Flexible kernel memory. PLoS ONE. 2010. Vol. 5(6): e10955. doi:10.1371/journal.pone.0010955. https://doi.org/10.1371/journal.pone.0010955

32. Revunova E.G., Tyshchuk A.V. A model selection criterion for solution of discrete ill-posed problems based on the singular value decomposition. Proc. IWIM’2015 (20–24th of July, 2015, Kyiv-Zhukin). Kyiv-Zhukin, 2015. P.43–47.

33. Revunova E.G. Improving the accuracy of the solution of discrete ill-posed problem by random projection. Cybernetics and Systems Analysis. 2018. Vol. 54, N 5. P. 842–852. https://doi.org/10.1007/s10559-018-0086-0

34. Marzetta T., Tucci G., Simon S. A random matrix-theoretic approach to handling singular covariance estimates. IEEE Trans. Information Theory. 2011. Vol. 57, N 9. P. 6256–6271. https://doi.org/10.1109/TIT.2011.2162175

35. Stepashko V. Theoretical aspects of GMDH as a method of inductive modeling. Control systems and machines. 2003. N 2. P. 31–38. (in Russian)

36. Stepashko V. Method of critical variances as analytical tool of theory of inductive modeling. Journal of Automation and Information Sciences. 2008. Vol. 40, N 3. P. 4–22. https://doi.org/10.1615/JAutomatInfScien.v40.i3.20

37. Kussul E.M., Baidyk T.N., Lukovich V.V., Rachkovskij D.A. Adaptive neural network classifier with multifloat input coding. Proc. Neuro-Nimes’93 (25–29th of Oct., 1993, Nimes). Nimes, France, 1993 P. 209–216.

38. Lukovich V.V., Goltsev A.D., Rachkovskij D.A. Neural network classifiers for micromechanical equipment diagnostics and micromechanical product quality inspection. Proc. EUFIT’97 (8–11th of Sept, 1997, Aachen). Aachen, Germany, 1997. P. 534–536.

39. Kussul E.M., Kasatkina L.M., Rachkovskij D.A., Wunsch D.C. Application of random threshold neural networks for diagnostics of micro machine tool condition. Proc. IJCNN’01 (4–9th of May, 1998, Anchorage). Anchorage, Alaska, USA, 1998 P. 241–244. https://doi.org/10.1109/IJCNN.1998.682270

40. Gol’tsev A.D. Structured neural networks with learning for texture segmentation in images. Cybernetics and Systems Analysis. 1991. Vol. 27, N 6. P. 927–936. https://doi.org/10.1007/BF01246527

41. Rachkovskij D.A., Revunova E.G. Intelligent gamma-ray data processing for environmental monitoring. In: Intelligent Data Processing in Global Monitoring for Environment and Security. Kyiv-Sofia: ITHEA. 2011. P. 136–157.

42. Revunova E.G., Rachkovskij D.A. Random projection and truncated SVD for estimating direction of arrival in antenna array. Kibernetika i vycislitel`naa tehnika. 2018. N 3(193). P. 5–26.

43. Ferdowsi S., Voloshynovskiy S., Kostadinov D., Holotyak T. Fast content identification in highdimensional feature spaces using sparse ternary codes. Proc. WIFS’16 (4–7th of Dec., 2016, Abu Dhabi) Abu Dhabi, UAE, 2016. P. 1–6.

44. Dasgupta S., Stevens C.F., Navlakha S. A neural algorithm for a fundamental computing problem. Science. 2017. Vol. 358(6364). P. 793–796. https://doi.org/10.1126/science.aam9868

45. Iclanzan D., Szilagyi S.M., Szilagyi L.. Evolving computationally efficient hashing for similarity search. Proc. ICONIP’18. 2. (Siem Reap, 15-18th of Dec., 2018). Siem Reap, Cambodia, 2018. 2018. https://doi.org/10.1007/978-3-030-04179-3_49

46. Rachkovskij D.A., Slipchenko S.V., Kussul E.M., Baidyk T.N. Properties of numeric codes for the scheme of random subspaces RSC. Cybernetics and Systems Analysis. 2005. Vol. 41, N. 4. P. 509–520. https://doi.org/10.1007/s10559-005-0086-8

47. Rachkovskij D.A., Slipchenko S.V., Kussul E.M., Baidyk T.N. Sparse binary distributed encoding of scalars. 2005. Journal of Automation and Information Sciences. Vol. 37, N 6. P. 12–23. https://doi.org/10.1615/J
Automat Inf Scien.v37.i6.20

48. Rachkovskij D.A., Slipchenko S.V., Misuno I.S., Kussul E.M., Baidyk T. N. Sparse binary distributed encoding of numeric vectors. Journal of Automation and Information Sciences. 2005. Vol. 37, N 11. P. 47–61. https://doi.org/10.1615/J
Automat Inf Scien.v37.i11.60

49. Kleyko D., Osipov E., Rachkovskij D.A. Modification of holographic graph neuron using sparse distributed representations. Procedia Computer Science. 2016. Vol. 88. P. 39–45. https://doi.org/10.1016/j.procs.2016.07.404

50. Kleyko D., Rahimi A., Rachkovskij D., Osipov E., Rabaey J. Classification and recall with binary hyperdimensional computing: Tradeoffs in choice of density and mapping characteristics. IEEE Trans. Neural Netw. Learn. Syst. 2018.

51. Kussul E., Baidyk T., Kasatkina L. Lukovich V. Rosenblatt perceptrons for handwritten digit recognition. Proc. IJCNN’01. (Washington, 15-19 July, 2001). Washington, USA. 2001. P. 1516–1521. https://doi.org/10.1109/IJCNN.2001.939589

52. Baidyk T, Kussul E., Makeyev O., Vega A., Limited receptive area neural classifier based image recognition in micromechanics and agriculture. International Journal of Applied Mathematics and Informatics. 2008.Vol. 2, N 3. P. 96–103.

53. Baydyk T., Kussul E., Hernandez Acosta M. LIRA neural network application for microcomponent measurement. International Journal of Applied Mathematics and Informatics. Vol.6, N 4. 2012. P.173–180.

54. Goltsev A.D., Gritsenko V.I. Algorithm of sequential finding the textural features characterizing homogeneous texture segments for the image segmentation task. Kibernetika i vycislitel`naa tehnika. 2013. N 173. P. 25–34 (in Russian).

55. Goltsev A., Gritsenko V., Kussul E., Baidyk T. Finding the texture features characterizing the most homogeneous texture segment in the image. Proc. IWANN’15. (Palma de Mallorca, Spain, June 10-12, 2015). Palma de Mallorca, 2015. 2015. P. 287–300. https://doi.org/10.1007/978-3-319-19258-1_25

56. Goltsev A., Gritsenko V., Husek D. Extraction of homogeneous fine-grained texture segments in visual images. Neural Network World. 2017. Vol. 27, N 5. P. 447– 477. https://doi.org/10.14311/NNW.2017.27.024

57. Kussul N.N., Sokolov B.V., Zyelyk Y.I., Zelentsov V.A., Skakun S.V., Shelestov A.Y. Disaster risk assessment based on heterogeneous geospatial information. J. of Automation and Information Sci. 2010. Vol. 42, N 12. P. 32–45. https://doi.org/10.1615/JAutomatInfScien.v42.i12.40

58. Kussul N., Lemoine G., Gallego F. J., Skakun S. V, Lavreniuk M., Shelestov A. Y. Parcel-based crop classification in Ukraine using Landsat-8 data and Sentinel-1A data. IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. 2016. Vol. 9, N 6. P. 2500–2508. https://doi.org/10.1109/JSTARS.2016.2560141

59. Kussul N., Lavreniuk M., Shelestov A., Skakun S. Crop inventory at regional scale in Ukraine: developing in season and end of season crop maps with multi-temporal optical and SAR satellite imagery. European Journal of Remote Sensing. 2018. Vol. 51, N 1. P. 627–636. https://doi.org/10.1080/22797254.2018.1454265

60. Sokolov A., Rachkovskij D. Approaches to sequence similarity representation. Information Theories and Applications. 2005. Vol.13, N 3. P. 272–278.

61. Recchia G., Sahlgren M., Kanerva P., Jones M. Encoding sequential information in semantic space models: Comparing holographic reduced representation and random permutation. Comput. Intell. Neurosci. 2015. Vol. 2015. Art. 986574. P. 1–18.

62. Rasanen O.J., Saarinen J.P. Sequence prediction with sparse distributed hyperdimensional coding applied to the analysis of mobile phone use patterns. IEEE Trans. Neural Netw. Learn. Syst. 2016. Vol. 27, N 9. P. 1878–1889. https://doi.org/10.1109/TNNLS.2015.2462721

63. Gallant S.I., Culliton P. Positional binding with distributed representations. Proc. ICIVC’16. (Portsmouth, UK 3–5 Aug., 2016). Portsmouth, 2016. 2016. P. 108–113. https://doi.org/10.1109/ICIVC.2016.7571282

64. Frady E. P., Kleyko D., Sommer F. T. A theory of sequence indexing and working memory in recurrent neural networks. Neural Comput. 2018. Vol. 30, N. 6. P. 1449–1513. https://doi.org/10.1162/neco_a_01084

65. Rachkovskij D.A. Some approaches to analogical mapping with structure sensitive distributed representations. Journal of Experimental and Theoretical Artificial Intelligence. 2004. Vol. 16, N 3. P. 125–145. https://doi.org/10.1080/09528130410001712862

66. Slipchenko S.V., Rachkovskij D.A. Analogical mapping using similarity of binary distributed representations. Int. J. Information Theories and Applications. 2009. Vol. 16, N 3. P. 269–290.

Received 22.08.2018

admin

Issue 2 (192), article 2

DOI:https://doi.org/10.15407/kvt192.02.023

Kibern. vyčisl. teh., 2018, Issue 2 (192), pp.

Grytsenko V.I.1,
Corresponding Member of NAS of Ukraine,
Director of International Research and Training
Center for Information Technologies and Systems
of the National Academy of Sciences of Ukraine
and Ministry of Education and Science of Ukraine
e-mail: vig@irtc.org.ua
Gladun A.Y.1,
PhD (Engineering), Senior Researcher of the Department of Complex Research
of Information Technologies and Systems
e-mail: glanat@yahoo.com
Rogushina Y.V.2,
PhD (Phyz&Math), Senior Researcher of the Department of Automated Information Systems
e-mail: ladanandraka2010@gmail.com
1International Research and Training Center for Information
Technologies and Systems of the National Academy
of Sciences of Ukraine and Ministry of Education and Science of Ukraine,
40, Acad. Glushkov av., 03187, Kiev, Ukraine
2Institute of Program Systems of the National Academy of Sciences of Ukraine,
40, Acad. Glushkov av., 03187, Kiev, Ukraine

MODELS AND METHODS OF THE SEMANTIC WIKI RESOURCES USE AS KNOWLEDGE SOURCES FOR RENEWAL OF FORMAL DOMEN ONTOLOGIES

Introduction. The construction and implementation of intelligent systems based on the formalization and reuse of knowledge is a promising direction for the practical application of artificial intelligence methods. The basis of such systems is formalized representations of knowledge about the subject area, for example, in the form of ontology. There remains an open question of the choice of the formal apparatus tools for the construction of ontology.
The purpose of the paper is to develop models of structured representation of knowledge in Wiki-resources on the basis of ontologies and methods of their application for improving and replenishing ontologies of the subject area. The offered approach will allow integrating the current information on changes in the subject area and creating actual ontologies for various applied information technologies using ontologies.
Results. The expediency of using ontologies for presentation of knowledge in systems of artificial intelligence oriented to functioning in the open environment of the Web is considered. The researches connected with the construction of formal ontologies of subject areas and the means of their formalization are analyzed. A formal model of ontology, which specifies the existing approaches, describing in more detail the properties and characteristics of the relations between the main elements of ontology is proposed. An example of using the proposed method in the task of transforming the natural text into a sign language in the system of information support of people with speech and hearing impairments is given.
Conclusions. The paper describes a method for renewal the ontology of a subject area based on the proposed model and the use of semantically-tagged Wiki-resources as a source of knowledge. This provides a dynamic replenishment of the knowledge base of applied intelligent systems. The proposed method of renewal formal ontologies of the subject domain from semantic Wiki-resources provides the expansion of the vocabulary and the construction of its specialized versions for various professional fields or subject areas using external databases. The automatic addition of new words from subject areas is particularly important for developing industries, especially for the IT sector, which has a large number of people with speech and hearing impairments. The proposed approach will improve the quality of life for many people, expanding the boundaries of their communication.

Keywords: formal ontology, ontological languages, formal model of ontology, interpretation of ontologies, semantic Wiki-resources, information system.

Download full text!

REFERENCES

1 Gruber, T.R. A translation approach to portable ontology specifications. Knowledge Acquisition. 1993. Vol 5. P. 199–220. https://doi.org/10.1006/knac.1993.1008

2 Poli R. Decriptive, Formal and Formalized Ontology. In: Husserl’s Logical Investigations reconsidered, by Denis Fisette (ed.). Dordrecht: Kluwer Academic Publishers, 2003, pp. 183–210. 24p. https://www.ontology.co/essays/descriptive-ontologies.pdf

3 Degen W., Heller B., Herre H., Smith B. GOL: A General Ontological Language. 2002, 63 p. https://www.researchgate.net/publication/2498727_GOL_A_General_Ontological_Language

4 Genesereth V.R., Fikes R.E. Knowledge Interchange Format, Version 3.0, Reference Manual. Logic Group Report Logic-92-1, Computer Science Department, Stanford University. 1992. 68p. https://pdfs.semanticscholar.org/b20c/866025b85f165557235a68143c42d53fa70f.pdf

5 Guarino N. Formal Ontology and Information Systems. In: Formal Ontology in Information Systems. Proceedings of FOIS’98, by N. Guarino (ed.). Trento. Italy, Amsterdam, IOS-Press. 1998. P. 3–15. https://klevas.mif.vu.lt/~donatas/Vadovavimas/Temos/OntologiskaiTeisingasKoncepcinisModeliavi-mas/papildoma/Guarino98-Formal%20Ontology%20and%20Information%20Systems.pdf

6 Vasyukov V.L. Formal ontology and artificial intelligence (monograph). Moscow: IF RAS, 2006. 140 p. (in Russian)

7 Rogushina Yu.V., Gladun A.Ya. Mereological aspects of the ontological analysis of intelligent Web-services. Proceedings of the VII International Conference “Intellectual Analysis of Information” IAI-2007, 12-14 Mai 2007, Kyiv. — P. 312–321. (in Russian)

8 Kotarbi!ǹ!sky T. Elements of the theory of cognition, formal logic and methodology of sciences. Lviv, 1929. 232p.

9 9. Wille R., Ganter B. Formale Begriffsanalyse. Berlin-Heidelberg: Springer-Verlag, 1996, 192 p.

10 Ganter B., Stumme G., Wille R., eds. Formal Concept Analysis: Foundation and Application. Lecture Notes in Artificial Intelligence. 2005, No. 3626. 349 p. https://link.springer.com/content/pdf/bfm%3A978-3-540-31881-1%2F1.pdf

11 Lammari N., Metais E. Building and maintaining ontologies: a set of algorithms. Data Knowledge Engineering, 2004, No. 48 (2), P 155–176.

12 Guarino N. Formal Ontology and Information Systems. In: Ontology in Information Systems. Proceedings of FOIS-08, Trento. Italy, by N. Guarino (ed.), Amsterdam, IOS-Press, 2009. 340 p. http://mba.eci.ufmg.br/downloads/recol/FormalOntologyinInforma-tionSystems2008.pdf

13 Rogushina Yu.V., Priyma S.M, Strokan O.V. Creating and using semantic Wiki-resources: tutorial. Melitopol, FOP Odnorog T.V., 2017. 169 p. (in Ukrainian).

14 Anisimov A.V., Lyman K.S., Marchenko A.A. Methods for computing measures of semantic proximity of natural language words. Artificial Intelligence, 2009, No 3. P. 612–617. (in Russian)

15 15. Lozynska O., Davydov M. Information technology for Ukrainian Sign Language translation based on ontologies. An International quarterly journal ECONTECHMOD, 2015, Vol. 04, No. 2, P. 13–18.

16 16. Gladun A.Ya. Khala K.O. Standard DSTU ISO/IEC 2382:2018 Information Technologies. Dictionary. Kyiv: UkrNDNC, 2018. 526 p. (in Ukrainian).

17 Anisimov A., Marchenko O., Taranukha V., Vozniuk T. Semantic and Syntactic Model of Natural Language Based on Non-negative Matrix and Tensor Factorization. Proceedings of the International Conference on Natural Language Processing, 2014, Springer, Cham. P. 177–184.

18 Gladun A.Ya., Rogushina Yu.V. Ontology Repository as a Tool for Reusing Knowledge for Recognizing Information Objects. Ontology of Design, No 1, 2013. P. 35–49. (in Russian).

19 Gladun A.Ya., Rogushina Yu.V. Semantic Technologies: Principles and Practices. Kyiv: Universarium, 2016. 387 p. (in Ukrainian).

20 Gladun A.Ya., Rogushina Yu.V. Bases of Methodology of Formation of Thesauruses with Use Ontologic and the Mereologic Analysis. Artificial Intelligence, 2008, No 5. P. 112–124. (in Ukrainian).

Received 02.04.2018

admin

Issue 1 (191), article 3

DOI:https://doi.org/10.15407/kvt191.01.045

Kibern. vyčisl. teh., 2018, Issue 1 (191), pp.

Grytsenko V.I., Corresponding Member of NAS of Ukraine,
Director of International Research and Training
Center for Information Technologies and Systems
of the National Academy of Sciences of Ukraine
and Ministry of Education and Science of Ukraine
e-mail: vig@irtc.org.ua
Volkov О.Y., Senior Researcher,
Intellectual Control Department
e-mail: alexvolk@ukr.net
Komar M.M., Researcher,
Intellectual Control Department
e-mail: nickkomar08@gmail.com
Bogachuk Y.P., PhD (Engineering), Senior Researcher,
Intellectual Control Department
e-mail: dep185@irtc.org.ua
International Research and Training Center for Information
Technologies and Systems of the National Academy
of Sciences of Ukraine and Ministry of Education and Science of Ukraine,
40, Acad. Glushkov av., 03187, Kiev, Ukraine

INTELLECTUALIZATION OF MODERN SYSTEMS OF AUTOMATIC CONTROL OF UNMANNED AERIAL VEHICLES

Introduction. The article discusses the actual questions of the need of creation of modern systems of automatic control of unmanned aerial vehicle (UAV) and describes new methods of its intellectualization. Today’s development of information technology requires accelerated development of the theory of intellectual control and the theory of systemic information technology. New technologies of intellectual control are extremely important for solving the problems of modern unmanned aviation.
The purpose of the article is to solve the issues of the development of the control system of UAV and to provide a number of measures aimed to ensuring its intellectualization. The approach considered in the article is based on the theory of high-precision remote control of dynamic objects and on the complex interaction of methods of theory of invariance, adaptive control and intellectualization of processes of UAV control.
Results. The development and implementation of control algorithms using functional program modules written in modern high-level programming languages in the computer environment based on microprocessors with a computing power sufficient to implement these algorithms in the form of a unified hardware and software complex of the integrated avionics.
The expansion of the range of functional capabilities of UAV control system that is offered to supplement the developed channels and algorithms of autopilot by the methods of intellectualization.
Conclusions. It is shown that combining the developed control laws for UAV autopilot into a unified hardware and software complex of integrated avionics and supplementing them with the proposed components of intellectualization will create a synergy effect and ensure the effectiveness and sustainability of the process of controlling the motion of the UAV.

Keywords: unmanned aerial vehicle, control system, invariance, intellectualization,

Download full text (ua)!

REFERENCES

  1. Krasil’shchikovM.N., SerebryakovG.G.Modern information technologies in the tasks of navigation and guidance of unmanned maneuverable aircrafts. Moscow: FIZMATLIT, 2009. 556 p. (in Russian).
  2. Kharchenko V.P., Chepizhenko V.I., Tunik A.А., Pavlova S.V. Avionic-sofunmannedaerialvehicles. Kyiv: Abris-Print, 2012. 464 p. (in Ukrainian)
  3. Fedosov E.A., Bobronnikov V.T., Kukhtenko V.I. Dynamic design of control systems for automatic maneuverable aircrafts. Moscow: Mashinostroyeniye, 1997. 336 p. (In Russian).
  4. Pavlova S., Komar M. The Invariant Adaptation of the Aircraft Control System in Emergency Situation During the Flight. ProceedingoftheNationalAviationUniversity. 2016. № 4(69). P. 28–33.
  5. Fahlstrom P., Gleason T. Introduction to UAV systems. Hoboken: Wiley, 2012. 4th ed. 308 p.
  6. Kortunov V.I., Mazurenko A.V., AliHusseinVaticMohammedControlsofminiandmicro-UAVs. Radiotelectronicandcomputersystems. 2016. № 1. P. 45–55 (In Russian).
  7. Austin R. Unmannedaircraftsystems. UAVsdesign, developmentanddeployment. JohnWiley&Sons, 2010. 372 p.
  8. Randal W. Beard, Timothy W. McLaine Small unmanned aerial vehicles: theory and practice. Moscow: TEKHNOSFERA, 2015. 312 c.
  9. AlyoshinB.S.Orientationandnavigationofmobileobjects: moderninformationtechnologies. Moscow: FIZMATLIT, 2006. 424 p. (In Russian).
  10. Volkov A.E., Pavlova S.V. Modelingoftheinvariantmethodforresolvingthedynamic-conflictsofaircraft. Cyberneticsandsystemsanalysis. 2017. № 53 (4). P. 105–112 (In Russian).
  11. Voloshenyuk D.A., Pavlova S.V. Managementofaircraftlandinginconditionsofincreasing-airtraffic. Controlsystemsandmachines. 2017 № 5. P. 62–74 (In Russian).

Received 27.12.2017

admin

Issue 4 (190), article 1

DOI:https://doi.org/10.15407/kvt190.04.005

Kibern. vyčisl. teh., 2017, Issue 4 (190), pp.

Grytsenko V.I., Corresponding Member of NASU of Ukraine,
Director of International research and training
center for Information technologies and systems
of the NASU and MESU
e-mail: vig@irtc.org.ua
Onyshchenko I.M., PhD (Economics),
Senior Researcher of the Department of Economic and Social
Systems and Information Technologies
e-mail: standardscoring@gmail.com
International research and training center for Information
technologies and systems of the NASU and MESU,
40, Ave Glushkov, 03680, Kiev, Ukraine

DETERMINING THE INFORMATIVITY OF PARAMETERS IN A PROGNOSTIC MODEL FOR EVALUATING THE PROBABILITY OF PRODUCT SELECTION IN THE CONDITIONS OF “BIG DATA”

Introduction. Fast growth of collected and stored data due to IT bumming caused a problem called “Big Data Problem”. Most of the new data are unstructured and this is the core reason why traditional relational data warehouse are so inefficient to deal with “Big Data”. Predicting and modeling based on “Big Data” also can be problematic because of high volume and velocity. To avoid some problems online learning algorithms can be successful for high-load systems.
The purpose of the article is to develop an approach to feature selection and modeling in case of “Big Data” with using online learning algorithm.
Method. Online learning algorithm for FTRL (Follow-The-Regularized-Leader) model with L1 and L2 regularization to select only important features was used.
Results. The approaches of modeling in cases of using batch and online learning algorithms are described on the example of online auction system. The online learning algorithm has very strong preferences in case of high load and high velocity. Mathematical background for modification of linear discriminator of FTL (Follow-The-Leader) model with adding regularization was described. L1 and L2 regularization allows us to select important features in real time. If the feature becomes useless, the regularization will set the corresponding coefficient equal to 0. But it does not remove the feature from training process and the coefficient can be restored with some value in case of its importance for model. The full process is prepared as a program in Python and can be used in practice.
The results may be applied for modeling and predicting in projects with high volume or velocity of data for example — social networks, online auctions, online gaming, recommendation systems and others.
The results may be applied for modeling and forcasting in projects with high volume or velocity of data, for example — social networks, online auctions, online gaming, recommendation systems and others .
Conclusions. FTRL model to work as online learning algorithm that allows to predict binary outcomes in high load “Big Data” systems was modified.
Getting into account that number of predictors can be enormous it takes much computing resources, time and make the process difficult. This feature selection problem was solved with using L1 regularization. The selection procedure was added to modified online learning FTRL model. L1 regularization to score the importance of predictors in real time was used.
A program that runs described mathematical algorithm was developed. Note that the algorithm effectively works with sparse matrices by analyzing incoming data and updating weights only for predictors that are presented. The algorithm has L1 and L2 regularization features that may be used for feature selection and avoid overfitting.
Keywords: information technologies in economics, economical and mathematical modeling, online learning algorithms, regularization, Big Data.

Download full text (ua)!

REFERENCES

1 Maier-Shenberher Vyktor. Bolshye dannye. Revoliutsyia, kotoraia yzmenyt to, kak my zhyvem, rabotaem i myslym/Vyktor Maier-Shenberher, Kennet Kuker; per. s anhl. Ynna Haidiuk. — Moskow: Mann, Yvanovy Ferber, 2014. — 240 p. (in Russian).

2 M. Regelson and D. Fain. Predicting click-through rate using keyword clusters. In Proceedings of the Second Workshop on Sponsored Search Auctions, volume 9623. Citeseer, 2006.

3 M. Richardson, E. Dominowska, and R. Ragno. Predicting clicks: estimating the click-through rate for new ads. In Proceedings of the 16th international conference on World Wide Web, pages 521–530. ACM, 2007.
https://doi.org/10.1145/1242572.1242643

4 Shalev-Shwartz, Shai. “Online Learning and Online Convex Optimization”. Foundations and Trends in Machine Learning. 2011. pp. 107–194.
https://doi.org/10.1561/2200000018

5 Gilles Gasso. Batch and online learning algorithms for nonconvex Neyman-Pearson classification / Gilles Gasso, Aristidis Pappaioannou, Marina Spivak, Leon Bottou / ACM Transaction on Intelligent System and Technologies, 2(3), 2011.
https://doi.org/10.1145/1961189.1961200

6 H Brendan McMahan. Follow-the-regularized-leader and mirror descent: Equivalence theorems and l1 regularization. International Conference on Artificial Intelligence and Statistics, pages 525–533, 2011.

7 Byll Franks. Ukroshchenye bolshykh dannykh: kak yzvlekat znanyia yz massyvov ynformatsyy s pomoshchiu hlubokoi analytyky / Byll Franks; per. s anhl. Andreia Baranova. — M. : Mann, Yvanov y Ferber, 2014. — 352 p. (in Russian).

8 N.B. Shakhovska. Model Velykykh Danykh “Sutnist — kharakterystyka”. / N.B. Shakhovska, Yu.Ia. Boliubash / 2015 r. [Elektronnyi resurs] — Rezhym dostupu: http://www.academia.edu/19609620/%D0%9C%D0%9E%D0%94%D0%95%D0%9B%D0%AC_%D0%92%D0%95%D0%9B%D0%98%D0%9A%D0%98%D0%A5_%D0%94%D0%90%D0%9D%D0%98%D0%A5_%D0%A1%D0%A3%D0%A2%D0%9D%D0%86%D0%A1%D0%A2%D0%AC-%D0%A5%D0%90%D0%A0%D0%90%D0%A% D0%A2%D0%95%D0%A0%D0%98%D0%A1%D0%A2%D0%98%D0%9A%D0%90_ (in Ukrainian).

9 Cherniak Leonyd. Bolshye Dannye — novaia teoryia y praktyka. Otkrytye systemy. SUBD. — M.: Otkrytye systemy, 2011. — No 10. [Elektronnyi resurs] — Rezhym dostupu: http://www.osp.ru/os/2011/10/13010990/ (in Russian).

10 Uskenbaeva, R.K. Tasks of resources provision of distributed computer systems functionality / R.K. Uskenbayeva, A.A. Kuandykov, A.U. Kalizhanova. — Dubai, World Academy of Science, Engineering and Technology. — 2012. — Iss. 70. — P. 580–581.

11 R. Bekkerman, M. Bilenko, and J. Langford. Scaling up machine learning: Parallel and distributed approaches. 2011

12 H.B. McMahan. Follow-the-regularized-leader and mirror descent: Equivalence theorems and L1 regularization. In AISTATS, 2011.

13 H.B. McMahan and M. Streeter. Adaptive bound optimization for online convex optimization. In COLT, 2010.

14 Hrytsenko V.I. Zastosuvannia instrumentiv Big Data dlia pidvyshchennia efektyvnosti onlain reklamy. Ekonomiko-matematychne modeliuvannia sotsialno-ekonomichnykh system. Vypusk 21. — Kyiv, 2016. P 5–21 (in Ukrainian).

15 Big Data — Wikipedia. [Elektronnyi resurs] — Rezhym dostupu: https://en.wikipedia.org/wiki/Big_data

16 Chto takoe Real-Time Bidding. [Elektronnyi resurs] — rezhym dostupu: http://konverta.ru/how (in Russian).

17 Introduction to online machine learning: Simplified. [Elektronnyi resurs] — rezhym dostupu: http://www.analyticsvidhya.com/blog/2015/01/introduction-online-machine-learning-simplified-2/

18 Riedman J. H. Regularization paths for generalized linear models via coordinate descent / Riedman J. H., Hastie T., Tibshirani R. / Journal of Statistical Software. 2010. Vol. 33, no. 1. pp. 1–22

19 L1- y L2-rehuliaryzatsyia v mashynnom obuchenyy. [Elektronnyi resurs] — rezhym dostupu: https://msdn.microsoft.com/uk-ua/magazine/dn904675.aspx (in Russian).

20 L1-rehuliaryzatsyia lyneinoi rehressyy. Rehressyia naymenshykh uhlov (alhorytm LARS). [Elektronnyi resurs] — rezhym dostupu: chrome-extension: //ecnphlgnajanjnkcmbpancdjoidceilk/content/web/viewer.html?source=extension_pdfhandler &file=http%3A%2F%2Fwww.machinelearning.ru%2Fwiki%2Fimages%2F7%2F7e%2F VetrovSem11_LARS.pdf (in Russian).

Received 28.09.2017

admin

Issue 2 (188), article 6

Kibern. vyčisl. teh., 2017, Issue 2 (188), pp.

80TH ANNIVERSARY OF CORRESPONDING MEMBER OF NAS OF UKRAINE VLADIMIR ILYICH GRITSENKO

May 23, 2017 the 80th anniversary of Vladimir Ilyich Gritsenko, known scientist in computer science, information technologies and its applications in economics, industrial and technological field, biological and medical cybernetics, computer technology training, director of the International Scientific and Training Center for Information Technologies and Systems. He is an initiator of development of a new class of high technologies — intelligent information technologies. Gritsenko V.I. is a member of a number of leading international and state councils of Ukraine on informatics, Permanent Representative of Ukraine to the Council of UNESCO Intergovernmental Programme on the information and communications, heads the UNESCO Chair “New Information Technologies in Education for All”, the chief editor of the scientific journals “Control Systems and Machines” and “Cybernetics”.

Download full text (ua)!

admin

Issue 2 (188), article 1

DOI:https://doi.org/10.15407/kvt188.02.005

Kibern. vyčisl. teh., 2017, Issue 2 (188), pp.

Grytsenko V.I., Corresponding Member of NAS of Ukraine, Director
e-mail: vig@irtc.org.ua
International Research and Training Center for Information Technologies and Systems of the NAS of Ukraine and of Ministry of Education and Science of Ukraine,
av. Acad. Glushkova, 40, Kiev, 03680, Ukraine

Rachkovskij D.A., Doctor of Engineering, Leading Researcher,
Dept. of Neural Information Processing Technologies,
e-mail: dar@infrm.kiev.ua
International Research and Training Center for Information Technologies and Systems of the NAS of Ukraine and of Ministry of Education and Science of Ukraine,
av. Acad. Glushkova, 40, Kiev, 03680, Ukraine

Frolov A.A., Doctor of Biology, Professor,
Faculty of Electrical Engineering and Computer Science FEI,
e-mail: docfact@gmail.com
Technical University of Ostrava, 17 listopadu 15, 708 33 Ostrava-Poruba, Czech Republic

Gayler R., PhD,
Independent Researcher,
r.gayler@gmail.com
Melbourne, VIC, Australia

Kleyko D., PhD post graduated,
Department of Computer Science, Electrical and Space Engineering,
denis.kleyko@ltu.se
Lulea University of Technology, 971 87 Lulea, Sweden

Osipov E., PhD, Professor,
Department of Computer Science, Electrical and Space Engineering,
evgeny.osipov@ltu.se
Lulea University of Technology, 971 87 Lulea, Sweden

NEURAL DISTRIBUTED AUTOASSOCIATIVE MEMORIES: A SURVEY.

Introduction. Neural network models of autoassociative, distributed memory allow storage and retrieval of many items (vectors) where the number of stored items can exceed the vector dimension (the number of neurons in the network). This opens the possibility of a sublinear time search (in the number of stored items) for approximate nearest neighbors among vectors of high dimension.

The purpose of the paper is to review models of autoassociative, distributed memory that can be naturally implemented by neural networks (mainly with local learning rules and iterative dynamics based on information locally available to neurons).

Scope. The survey is focused mainly on the networks of Hopfield, Willshaw, and Potts, that have connections between pairs of neurons and operate on sparse binary vectors. We discuss not only autoassociative memory, but also the generalization properties of these networks. We also consider neural networks with higher-order connections, and networks with a bipartite graph structure for non-binary data with linear constraints.

Conclusions. In conclusion we discuss the relations to similarity search, advantages and drawbacks of these techniques, and topics for further research. An interesting and still not completely resolved question is whether neural autoassociative memories can search for approximate nearest neighbors faster than other index structures for similarity search, in particular for the case of very high dimensional vectors.

Keywords: distributed associative memory, sparse binary vector, Hopfield network, Willshaw memory, Potts model, nearest neighbor, similarity search

Download full text!

REFERENCE

1 Abbott L.F., Arian Y. Storage capacity of generalized networks. Physical Review A. 1987. Vol. 36, N 10. P. 5091–5094. https://doi.org/10.1103/PhysRevA.36.5091

2 Ahle T.D. Optimal las vegas locality sensitive data structures. arXiv:1704.02054. 6 Apr 2017.

3 Aliabadi B. K., Berrou C., Gripon V., Jiang X. Storing sparse messages in networks of neural cliques. IEEE Trans. NNLS. 2014. Vol. 25. P. 980–989. https://doi.org/10.1109/TNNLS.2013.2285253

4 Amari S. Characteristics of sparsely encoded associative memory. Neural Networks. 1989. Vol. 2, N 6. P. 451–457. https://doi.org/10.1016/0893-6080(89)90043-9

5 Amari S., Maginu K. Statistical neurodynamics of associative memory. Neural Networks. 1988. Vol. 1. P. 63–73. https://doi.org/10.1016/0893-6080(88)90022-6

6 Amit D.J. Modeling brain function: the world of attractor neural networks. Cambridge: Cambridge University Press, 1989. 554 p. https://doi.org/10.1017/CBO9780511623257

7 Amit D.J., Fusi S. Learning in neural networks with material synapses. Neural Computation. 1994. V. 6, N 5. P. 957–982. https://doi.org/10.1162/neco.1994.6.5.957

8 Amit D.J., Gutfreund H., Sompolinsky H. Statistical mechanics of neural networks near saturation. Annals of Physics. 1987. Vol. 173. P. 30–67. https://doi.org/10.1016/0003-4916(87)90092-3

9 Amosov N. M. Modelling of thinking and the mind. New York: Spartan Books. 1967. https://doi.org/10.1007/978-1-349-00640-3

10 Anderson J. A. A theory for the recognition of items from short memorized lists. Psychological Review. 1973. Vol. 80, N 6. P. 417–438. https://doi.org/10.1037/h0035486

11 Anderson J. A. Cognitive and psychological computation with neural models. IEEE trans. Systems, Man, and Cybernetics. 1983. Vol. 13, N 5. P. 799–814. https://doi.org/10.1109/TSMC.1983.6313074

12 Anderson J.A., Murphy G.L. Psychological concepts in a parallel system. Physica D. 1986. Vol. 22, N 1–3. P. 318–336. https://doi.org/10.1016/0167-2789(86)90302-2

13 Anderson J.A., Silverstein J.W., Ritz S.A., Jones R.S. Distinctive features, categorical perception and probability learning: Some applications of a neural model. Psychological Review. 1977. V. 84. P. 413–451. https://doi.org/10.1037/0033-295X.84.5.413

14 Andoni A., Laarhoven T., Razenshteyn I., Waingarten E. Optimal hashing-based time-space trade-offs for approximate near neighbors. Proc. SODA’17. 2017. P. 47–66. https://doi.org/10.1137/1.9781611974782.4

15 Baidyk T.N., Kussul E.M. Structure of neural assembly. Proc. RNNS/IEEE symposium on neuroinformatics and neurocomputers. 1992. P. 423–434.

16 Baidyk T.N., Kussul E.M., Rachkovskij D.A. Numerical-analytical method for neural network investigation. Proc. NEURONET’90. 1990. P. 217–219.

17 Baldi, P. and Venkatesh, S.S. Number of stable points for spin-glasses and neural networks of higher orders. Physical Review Letters. 1987. Vol. 58, N 9. P. 913–916. https://doi.org/10.1103/PhysRevLett.58.913

18 Becker A., Ducas L., Gama N., Laarhoven T. New directions in nearest neighbor searching with applications to lattice sieving. Proc. SODA’16. 2016. P. 10–24. https://doi.org/10.1137/1.9781611974331.ch2

19 Boguslawski B., Gripon V., Seguin F., Heitzmann F. Twin neurons for efficient real-world data distribution in networks of neural cliques: Applications in power management in electronic circuits. IEEE Trans. NNLS. 2016. Vol. 27, N 2. P. 375–387. https://doi.org/10.1109/TNNLS.2015.2480545

20 Bovier A. Sharp upper bounds on perfect retrieval in the Hopfield model. J. Appl. Probab. 1999. Vol. 36, N 3. P. 941–950. https://doi.org/10.1239/jap/1032374647
https://doi.org/10.1017/S0021900200017708

21 Braitenberg V. Cell assemblies in the cerebral cortex. In Theoretical approaches to complex systems. Berlin: Springer-Verlag. 1978. P. 171–188. https://doi.org/10.1007/978-3-642-93083-6_9

22 Broder A., Mitzenmacher M. Network applications of Bloom filters: A survey. Internet mathematics. 2004. Vol. 1, N 4. P. 485–509. https://doi.org/10.1080/15427951.2004.10129096

23 Brunel N., Carusi F., Fusi S. Slow stochastic Hebbian learning of classes of stimuli in a recurrent neural network. Network. 1998. Vol. 9. P. 123–152. https://doi.org/10.1088/0954-898X_9_1_007

24 Buckingham J., Willshaw D. On setting unit thresholds in an incompletely connected associative net. Network. 1993. Vol. 4. P. 441–459. https://doi.org/10.1088/0954-898X_4_4_003

25 Burshtein D. Non-direct convergence radius and number of iterations of the Hopfield associative memory. IEEE Trans. Inform. Theory. 1994. Vol. 40. P. 838–847. https://doi.org/10.1109/18.335894

26 Burshtein D. Long-term attraction in higher order neural networks. IEEE Trans. Neural Networks. 1998. Vol. 9, N 1. P. 42–50. https://doi.org/10.1109/72.655028

27 Christiani T., Pagh R. Set similarity search beyond MinHash. Proc. STOC’17. 2017. https://doi.org/10.1145/3055399.3055443

28 Cole R., Gottlieb L.-A., Lewenstein M. Dictionary matching and indexing with errors and don’t cares. Proc. STOC’04. 2004. P. 91–100. https://doi.org/10.1145/1007352.1007374

29 Dahlgaard S., Knudsen M.B.T., Thorup M. Fast similarity sketching. arXiv:1704.04370. 14 Apr 2017.

30 Demircigil M., Heusel J., Lowe M., Upgang S. Vermet F. On a model of associative memory with huge storage capacity. J. Stat. Phys. doi:10.1007/s10955-017-1806-y. 2017. https://doi.org/10.1007/s10955-017-1806-y

31 Donaldson R., Gupta A, Plan Y., Reimer T. Random mappings designed for commercial search engines. arXiv:1507.05929. 21 Jul 2015.

32 Feigelman M.V., Ioffe L.B. The augmented models of associative memory – asymmetric interaction and hierarchy of patterns. Int. Journal of Modern Physics B. 1987. Vol. 1, N 1, P. 51–68. https://doi.org/10.1142/S0217979287000050

33 Ferdowsi S., Voloshynovskiy S., Kostadinov D., Holotyak T. Fast content identification in highdimensional feature spaces using sparse ternary codes. Proc. WIFS’16. 2016. P. 1–6.

34 Frolov A.A., Husek D., Muraviev I.P. Information capacity and recall quality in sparsely encoded Hopfield-like neural network: Analytical approaches and computer simulation. Neural Networks. 1997. Vol. 10, N 5. P. 845–855. https://doi.org/10.1016/S0893-6080(96)00122-0

35 Frolov A.A., Husek D., Muraviev I.P. Informational efficiency of sparsely encoded Hopfield-like associative memory. Optical Memory & Neural Networks. 2003. Vol. 12, N 3. P. 177–197.

36 Frolov A.A., Husek D., Muraviev I.P., Polyakov P. Boolean factor analysis by attractor neural network. IEEE Trans. Neural Networks. 2007. Vol. 18, N 3. P. 698–707. https://doi.org/10.1109/TNN.2007.891664

37 Frolov A.A., Husek D., Polyakov P.Y. Recurrent neural-network-based boolean factor analysis and its application to word clustering. IEEE Trans. Neural Networks. 2009.Vol. 20, N 7. P. 1073–1086. https://doi.org/10.1109/TNN.2009.2016090

38 Frolov A. A., Husek D., Rachkovskij. Time of searching for similar binary vectors in associative memory. Cybernetics and Systems Analysis. 2006. Vol. 42, N 5. P. 615–623. https://doi.org/10.1007/s10559-006-0098-z

39 Frolov A.A., Muraviev I.P. Neural models of associative memory. Moscow: Nauka, 1987. 161 p.

40 Frolov A.A., Muraviev I.P. Information characteristics of neural networks. Moscow: Nauka, 1988. 160 p.

41 Frolov A.A., Muraviev I.P. Information characteristics of neural networks capable of associative learning based on Hebbian plasticity. Network. 1993. Vol. 4, N 4. P. 495–536. https://doi.org/10.1088/0954-898X_4_4_006

42 Frolov A., Kartashov A., Goltsev A., Folk R. Quality and efficiency of retrieval for Willshaw-like autoassociative networks. Correction. Network. 1995. Vol. 6. P. 513–534. https://doi.org/10.1088/0954-898X_6_4_001
https://doi.org/10.1088/0954-898X_6_4_002

43 Frolov A., Kartashov A., Goltsev A., Folk R. Quality and efficiency of retrieval for Willshaw-like autoassociative networks. Recognition. Network. 1995. Vol. 6. P. 535–549. https://doi.org/10.1088/0954-898X_6_4_001
https://doi.org/10.1088/0954-898X_6_4_002

44 Frolov A.A., Rachkovskij D.A., Husek D. On information characteristics of Willshaw-like auto-associative memory. Neural Network World. 2002. Vol. 12, N 2. P. 141–157.

45 Gallant S.I., Okaywe T.W. Representing objects, relations, and sequences. Neural Computation. 2013. Vol. 25, N 8. P. 2038–2078. https://doi.org/10.1162/NECO_a_00467

46 Gardner E. 1987. Multiconnected neural network models. Journal of Physics A. 1998. Vol. 20, N 11. P.3453–3464. https://doi.org/10.1088/0305-4470/20/11/046

47 Gardner E. The space of interactions in neural-network models. J. Phys. A. 1988. Vol. 21. P. 257–270. https://doi.org/10.1088/0305-4470/21/1/030

48 Gibson W. G., Robinson J. Statistical analysis of the dynamics of a sparse associative memory. Neural Networks. 1992. Vol. 5. P. 645–661. https://doi.org/10.1016/S0893-6080(05)80042-5

49 Golomb D., Rubin N., Sompolinsky H. Willshaw model: Associative memory with sparse coding and low firing rates 1990. Phys Rev A. Vol. 41, N 4. P. 1843–1854.

50 Goltsev A. An assembly neural network for texture segmentation. Neural Networks. 1996. Vol. 9, N 4. P. 643–653. https://doi.org/10.1016/0893-6080(95)00136-0

51 Goltsev A. Secondary learning in the assembly neural network. Neurocomputing. 2004. Vol. 62. P. 405–426. https://doi.org/10.1016/j.neucom.2004.06.001
https://doi.org/10.1016/S0925-2312(04)00305-4

52 Goltsev A., Husek D. Some properties of the assembly neural networks. Neural Network World. 2002. Vol. 12, N 1. P. 15–32.

53 Goltsev A., Wunsch D.C. Generalization of features in the assembly neural networks. International Journal of Neural Systems. 2004. Vol. 14, N 1. P. 1–18. https://doi.org/10.1142/S0129065704001838

54 Goltsev A.D. Neural networks with the assembly organization. Kiev: Naukova Dumka, 2005. 200 p.

55 Goltsev A., Gritsenko V. Modular neural networks with Hebbian learning rule. Neurocomputing. 2009. Vol. 72. P. 2477–2482. https://doi.org/10.1016/j.neucom.2008.11.011

56 Goltsev A., Gritsenko V. Modular neural networks with radial neural columnar architecture. Biologically Inspired Cognitive Architectures. 2015. Vol. 13, P. 63–74. https://doi.org/10.1016/j.bica.2015.06.001

57 Goswami M., Pagh R., Silvestri F., Sivertsen J. Distance sensitive bloom filters without false negatives. Proc. SODA’17. 2017. P. 257–269. https://doi.org/10.1137/1.9781611974782.17

58 Gripon V., Berrou C. Sparse neural networks with large learning diversity. IEEE Trans. on Neural Networks. 2011. Vol. 22, N 7. P. 1087–1096. https://doi.org/10.1109/TNN.2011.2146789

59 Gripon V., Heusel J., Lowe M., Vermet F. A comparative study of sparse associative memories. Journal of Statistical Physics. 2016. Vol. 164. P. 105–129. https://doi.org/10.1007/s10955-016-1530-z

60 Gripon V., Lowe M., Vermet F. Associative memories to accelerate approximate nearest neighbor search. ArXiv:1611.05898. 10 Nov 2016.

61 Gritsenko V.I., Rachkovskij D.A., Goltsev A.D., Lukovych V.V., Misuno I.S., Revunova E.G., Slipchenko S.V., Sokolov A.M., Talayev S.A. Neural distributed representation for intelligent information technologies and modeling of thinking. Cybernetics and Computer Engineering. 2013. Vol. 173. P. 7–24.

62 Guo J. K., Brackle D. V., Lofaso N., Hofmann M. O. Vector representation for sub-graph encoding to resolve entities. Procedia Computer Science. 2016. Vol. 95. P. 327–334. https://doi.org/10.1016/j.procs.2016.09.342

63 Gutfreund H. Neural networks with hierarchically correlated patterns. Physical Review A. 1988. Vol. 37, N 2. P. 570–577. https://doi.org/10.1103/PhysRevA.37.570

64 Hacene G. B., Gripon V., Farrugia N., Arzel M., Jezequel M. Finding all matches in a database using binary neural networks. Proc. COGNITIVE’17. 2017. P. 59–64.

65 Hebb D.O. The Organization of Behavior: A Neuropsychological Theory. New York: Wiley, 1949. 335 p.

66 Herrmann M., Ruppin E., Usher M. A neural model of the dynamic activation of memory. Biol. Cybern. 1993. Vol. 68. P. 455–463. https://doi.org/10.1007/BF00198778

67 Heusel J., Lowe M., Vermet F. On the capacity of an associative memory model based on neural cliques. Statist. Probab. Lett. 2015. Vol. 106. P. 256–261. https://doi.org/10.1016/j.spl.2015.07.026

68 Hopfield J.J. Neural networks and physical systems with emergent collective computational abilities // Proc. of the Nat. Acad. Sci. USA. 1982. Vol. 79, N 8. P. 2554–2558. https://doi.org/10.1073/pnas.79.8.2554

69 Hopfield J.J., Feinstein D.I., Palmer R.G. “Unlearning” has a stabilizing effect in collective memories. Nature. 1983. Vol. 304. P. 158–159. https://doi.org/10.1038/304158a0

70 Horn D., Usher M. Capacities of multiconnected memory models. Journal de Physique, 1988. Vol. 49, N 3. P. 389–395. https://doi.org/10.1051/jphys:01988004903038900

71 Horner H., Bormann D., Frick M., Kinzelbach H., Schmidt A. Transients and basins of attraction in neutral network models. Z. Physik B. 1989. Vol. 76. P. 381–398. https://doi.org/10.1007/BF01321917

72 Howard M. W., Kahana M. J. A distributed representation of temporal context. Journal of Mathematical Psychology. 2002. Vol. 46. P. 269–299. https://doi.org/10.1006/jmps.2001.1388

73 Iscen A., Furon T., Gripon V., Rabbat M., Jegou H. Memory vectors for similarity search in high-dimensional spaces. arXiv:1412.3328. 1 Mar 2017.

74 Kakeya H., Kindo T. Hierarchical concept formation in associative memory composed of neuro-window elements. Neural Networks. 1996. Vol. 9, N 7. P. 1095–1098. https://doi.org/10.1016/0893-6080(96)00030-5

75 Kanerva P. Sparse Distributed Memory. Cambridge: MIT Press, 1988. 155 p.

76 Kanerva P. Hyperdimensional computing: an introduction to computing in distributed representation with high-dimensional random vectors. Cognitive Computation. 2009. Vol. 1, N 2. P. 139–159. https://doi.org/10.1007/s12559-009-9009-8

77 Kanter I. Potts-glass models of neural networks. Physical Rev. A. 1988. V. 37, N 7. P. 2739–2742. https://doi.org/10.1103/PhysRevA.37.2739

78 Karbasi A., Salavati A. H., Shokrollahi A. Iterative learning and denoising in convolutional neural associative memories. Proc. ICML’13. 2013. P. 445–453.

79 Kartashov A., Frolov A., Goltsev A., Folk R. Quality and efficiency of retrieval for Willshaw-like autoassociative networks. Willshaw–Potts model. Network. 1997. Vol. 8, N 1. P. 71–86. https://doi.org/10.1088/0954-898X_8_1_007

80 Kinzel W. Learning and pattern recognition in spin glass models. Z. Physik B. 1985. Vol. 60. P. 205–213. https://doi.org/10.1007/BF01304440

81 Knoblauch A., Palm G., Sommer F. T. Memory capacities for synaptic and structural plasticity. Neural Computation. 2010. Vol. 22, N 2. P. 289–341. https://doi.org/10.1162/neco.2009.08-07-588

82 Kleyko D., Khan S., Osipov E., Yong S. P. Modality Classification of Medical Images with Distributed Representations based on Cellular Automata Reservoir Computing. Proc. ISBI’17. 2017. P. 1–4. https://doi.org/10.1109/ISBI.2017.7950697

83 Kleyko D., Lyamin N., Osipov E., Riliskis L. Dependable MAC layer architecture based on holographic data representation using hyperdimensional binary spatter codes. Proc. MACOM’12. 2012. P. 134–145

84 Kleyko D., Osipov E. Brain-like classifier of temporal patterns. Proc. ICCOINS’14. 2014. P. 1–6. https://doi.org/10.1109/ICCOINS.2014.6868349

85 Kleyko D., Osipov E. On bidirectional transitions between localist and distributed representations: The case of common substrings search using vector symbolic architecture. Procedia Computer Science. 2014. Vol. 41. P. 104–113. https://doi.org/10.1016/j.procs.2014.11.091

86 Kleyko D., Osipov E., Gayler R. W. Recognizing permuted words with Vector Symbolic Architectures: A Cambridge test for machines. Procedia Computer Science. 2016. Vol. 88. P. 169–175. https://doi.org/10.1016/j.procs.2016.07.421

87 Kleyko D., Osipov E., Gayler R. W., Khan A. I., Dyer A. G. Imitation of honey bees’ concept learning processes using vector symbolic architectures. Biologically Inspired Cognitive Architectures. 2015. Vol. 14. P. 55–72. https://doi.org/10.1016/j.bica.2015.09.002

88 Kleyko D., Osipov E., Papakonstantinou N., Vyatkin V., Mousavi A. Fault detection in the hyperspace: Towards intelligent automation systems. Proc. INDIN’15. 2015. P. 1219–1224. https://doi.org/10.1109/INDIN.2015.7281909

89 Kleyko D., Osipov E., Senior A., Khan A. I., Sekercioglu Y. A. Holographic Graph Neuron: a bio-inspired architecture for pattern processing. IEEE Trans. Neural Networks and Learning Systems. 2017. Vol. 28, N 6. P. 1250–1262. https://doi.org/10.1109/TNNLS.2016.2535338

90 Kleyko D., Osipov E., Rachkovskij D. Modification of holographic graph neuron using sparse distributed representations. Procedia Computer Science. 2016. Vol. 88. P. 39–45. https://doi.org/10.1016/j.procs.2016.07.404

91 Kleyko D., Rahimi A., Rachkovskij D.A., Osipov E., Rabaey J.M. Classification and recall with binary hyperdimensional computing: trade-offs in choice of density and mapping characteristics (2017, Submitted).

92 Kleyko D., Rahimi A, Osipov E. Autoscaling Bloom Filter: controlling trade-off between true and false. arXiv:1705.03934. 10 May 2017

93 Kohonen T. Content-Addressable Memories. Berlin: Springer, 1987. 388 p. https://doi.org/10.1007/978-3-642-83056-3

94 Kohring G.A. Neural networks with many-neuron interactions. Journal de Physique. 1990. Vol. 51, N 2. P. 145–155. https://doi.org/10.1051/jphys:01990005102014500

95 Krotov D., Hopfield J.J. Dense associative memory for pattern recognition. Proc. NIPS’16. 2016. P. 1172–1180.

96 Krotov D., Hopfield J.J. Dense associative memory is robust to adversarial inputs. arXiv:1701.00939. 4 Jan 2017

97 Kryzhanovsky B.V., Mikaelian A.L., Fonarev A.B. Vector neural net identifing many strongly distorted and correlated patterns. Proc. SPIE. 2005. Vol. 5642. 124–133. https://doi.org/10.1117/12.572334

98 Kussul E. M. Associative neuron-like structures. Kiev: Naukova Dumka, 1992.

99 Kussul E. M., Baidyk T. N. A modular structure of associative-projective neural networks. Preprint 93-6. Kiev, Ukraine: GIC, 1993.

100 Kussul E.M., Fedoseyeva T.V. On audio signals recognition in neural assembly structures. Preprint 87-28. Kiev: Inst. of Cybern. 1987. 21 pp.

101 Kussul E., Makeyev O., Baidyk T., Calderon Reyes D. Neural network with ensembles. Proc. IJCNN’10. 2010. P. 2955–2961. https://doi.org/10.1109/IJCNN.2010.5596574

102 Kussul E.M., Rachkovskij D.A. Multilevel assembly neural architecture and processing of sequences. In Neurocomputers and Attention, V. II: Connectionism and neurocomputers. Manchester and New York: Manchester University Press, 1991. P.577–590.

103 Kussul E.M., Rachkovskij D.A., Wunsch D.C. The random subspace coarse coding scheme for real-valued vectors. Proc. IJCNN’99. 1999. P. 450-455. https://doi.org/10.1109/IJCNN.1999.831537

104 Lansner A. Associative memory models: From the cell assembly theory to biophysically detailed cortex simulations. Trends in Neurosciences. 2009. Vol. 32, N 3. P. 178–186. https://doi.org/10.1016/j.tins.2008.12.002

105 Lansner A., Ekeberg O. Reliability and speed of recall in an associative network. IEEE Trans. on Pattern Analysis and Machine Intelligence. 1985. Vol. 7. P. 490–498. https://doi.org/10.1109/TPAMI.1985.4767688

106 Levy S. D., Gayler R. Vector Symbolic Architectures: A new building material for artificial general intelligence. Proc. AGI’08. 2008. P. 414–418.

107 Lowe M. On the storage capacity of Hopfield models with correlated patterns. The Annals of Applied Probability. 1998. Vol. 8, N 4. P. 1216–1250. https://doi.org/10.1214/aoap/1028903378

108 Lowe M., Vermet F. The storage capacity of the Hopfield model and moderate deviations. Statistics and Probability Letters. 2005. Vol. 75. P. 237–248. https://doi.org/10.1016/j.spl.2005.06.001

109 Lowe M., Vermet F. The capacity of q-state Potts neural networks with parallel retrieval dynamics. Statistics and Probability Letters. 2007. Vol. 77, N 4. P. 1505–1514. https://doi.org/10.1016/j.spl.2007.03.030

110 Mazumdar A., Rawat A.S. Associative memory via a sparse recovery model. Proc. NIPS’15. 2015. P. 2683–2691.

111 Mazumdar A., Rawat A.S. Associative memory using dictionary learning and expander decoding. Proc. AAAI’17. 2017.

112 McEliece R.J., Posner E.C., Rodemich E.R., Venkatesh S.S. The capacity of the Hopfield associative memory. IEEE Trans. Information Theory. 1987. Vol. 33, N 4. P. 461–482. https://doi.org/10.1109/TIT.1987.1057328

113 Misuno I.S., Rachkovskij D.A., Slipchenko S.V. Vector and distributed representations reflecting semantic relatedness of words. Math. machines and systems. 2005. N 3. P. 50–67.

114 Misuno I.S., Rachkovskij D.A., Slipchenko S.V., Sokolov A.M. Searching for text information with the help of vector representations. Probl. Progr. 2005. N 4. P. 50–59.

115 Nadal J.-P. Associative memory: on the (puzzling) sparse coding limit. J. Phys. A. 1991. Vol. 24. P. 1093–1101. https://doi.org/10.1088/0305-4470/24/5/023

116 Norouzi M., Punjani A., Fleet D. J. Fast exact search in Hamming space with multi-index hashing. IEEE Trans. PAMI. 2014. Vol. 36, N 6. P. 1107–1119. https://doi.org/10.1109/TPAMI.2013.231

117 Onizawa N., Jarollahi H., Hanyu T., Gross W.J. Hardware implementation of associative memories based on multiple-valued sparse clustered networks. IEEE Journal on Emerging and Selected Topics in Circuits and Systems. 2016. Vol. 6, N 1. P. 13–24. https://doi.org/10.1109/JETCAS.2016.2528721

118 Palm G. On associative memory. Biological Cybernetics. 1980. Vol. 36. P. 19–31. https://doi.org/10.1007/BF00337019

119 Palm G. Memory capacity of local rules for synaptic modification. Concepts in Neuroscience. 1991. Vol. 2, N 1. P. 97–128.

120 Palm G. Neural associative memories and sparse coding. Neural Networks. 2013. Vol. 37. P. 165–171. https://doi.org/10.1016/j.neunet.2012.08.013

121 Palm G., Knoblauch A., Hauser F., Schuz A. Cell assemblies in the cerebral cortex. Biol. Cybern. 2014. Vol. 108, N 5. P. 559–572 https://doi.org/10.1007/s00422-014-0596-4

122 Palm G., Sommer F. T. Information capacity in recurrent Mc.Culloch-Pitts networks with sparsely coded memory states. Network. 1992, Vol. 3, P. 177–186. https://doi.org/10.1088/0954-898X_3_2_006

123 Parga N., Virasoro M.A. The ultrametric organization of memories in a neural network. J. Mod. Phys. 1986. Vol. 47, N 11. P. 1857–1864. https://doi.org/10.1142/9789812799371_0047
https://doi.org/10.1051/jphys:0198600470110185700

124 Peretto P., Niez J.J. Long term memory storage capacity of multiconnected neural networks. Biol. Cybern. 1986. Vol. 54, N 1. P. 53–63. https://doi.org/10.1007/BF00337115

125 Personnaz L., Guyon I., Dreyfus G. Collective computational properties of neural networks: New learning mechanisms. Phys. Rev. A. 1986. Vol. 34, N 5. P. 4217–4228. https://doi.org/10.1103/PhysRevA.34.4217

126 Plate T. Holographic reduced representation: Distributed representation for cognitive structures. Stanford: CSLI Publications, 2003. 300 p.

127 Rachkovskij D.A. Representation and processing of structures with binary sparse distributed codes. IEEE Transactions on Knowledge and Data Engineering. 2001. Vol. 13, N 2. P. 261–276. https://doi.org/10.1109/69.917565

128 Rachkovskij D.A. Some approaches to analogical mapping with structure sensitive distributed representations. Journal of Experimental and Theoretical Artificial Intelligence. 2004. Vol. 16, N 3. P. 125–145. https://doi.org/10.1080/09528130410001712862

129 Rachkovskij D.A. Formation of similarity-reflecting binary vectors with random binary projections. Cybernetics and Systems Analysis. 2015. Vol. 51, N 2. P. 313–323. https://doi.org/10.1007/s10559-015-9723-z

130 Rachkovskij D.A. Estimation of vectors similarity by their randomized binary projections. Cybernetics and Systems Analysis. 2015. Vol. 51, N 5. P. 808–818. https://doi.org/10.1007/s10559-015-9774-1

131 Rachkovskij D.A. Binary vectors for fast distance and similarity estimation. Cybernetics and Systems Analysis. 2017. Vol. 53, N 1. P. 138–156. https://doi.org/10.1007/s10559-017-9914-x

132 Rachkovskij D.A. Distance-based index structures for fast similarity search. Cybernetics and Systems Analysis. 2017. Vol. 53, N 4. https://doi.org/10.1007/s10559-017-9966-y

133 Rachkovskij D.A. Index structures for fast similarity search of binary vectors. Cybernetics and Systems Analysis. 2017. Vol. 53, N 5. https://doi.org/10.1007/s10559-017-9983-x

134 Rachkovskij D.A., Kussul E.M., Baidyk T.N. Building a world model with structure-sensitive sparse binary distributed representations. BICA. 2013. Vol. 3. P. 64–86. https://doi.org/10.1016/j.bica.2012.09.004

135 Rachkovskij D.A., Misuno I.S., Slipchenko S.V. Randomized projective methods for construction of binary sparse vector representations. Cybernetics and Systems Analysis. 2012. Vol. 48, N 1. P. 146–156. https://doi.org/10.1007/s10559-012-9384-0

136 Rachkovskij D.A., Slipchenko S.V. Similarity-based retrieval with structure-sensitive sparse binary distributed representations. Computational Intelligence. 2012. Vol. 28, N 1. P. 106–129. https://doi.org/10.1111/j.1467-8640.2011.00423.x

137 Rachkovskij D.A., Slipchenko S.V., Kussul E.M., Baidyk T.N. Sparse binary distributed encoding of scalars. J. of Automation and Inf. Sci. 2005. Vol. 37, N 6. P. 12–23. https://doi.org/10.1615/J
Automat Inf Scien.v37.i6.20

138 Rachkovskij D.A., Slipchenko S.V., Kussul E.M., Baidyk T.N. Properties of numeric codes for the scheme of random subspaces RSC. Cybernetics and Systems Analysis. 2005. 41, N 4. P. 509–520. https://doi.org/10.1007/s10559-005-0086-8

139 Rachkovskij D.A., Slipchenko S.V., Misuno I.S., Kussul E.M., Baidyk T.N. Sparse binary distributed encoding of numeric vectors. J. of Automation and Inf. Sci. 2005. Vol. 37, N 11. P. 47–61. https://doi.org/10.1615/J
Automat Inf Scien.v37.i11.60

140 Reznik A.M., Sitchov A.S., Dekhtyarenko O.K., Nowicki D.W. Associative memories with killed neurons: the methods of recovery. Proc. IJCNN’03. 2003. P. 2579–2582. https://doi.org/10.1109/IJCNN.2003.1223972

141 Rizzuto D.S., Kahana M.J. An autoassociative neural network model of paired-associate learning. Neural Computation. 2001. Vol. 13. P. 2075–2092. https://doi.org/10.1162/089976601750399317

142 Rolls E. T. Advantages of dilution in the connectivity of attractor networks in the brain. Biologically Inspired Cognitive Architectures. 2012. Vol. 1, P. 44–54. https://doi.org/10.1016/j.bica.2012.03.003

143 Romani S., Pinkoviezky I., Rubin A., Tsodyks M. Scaling laws of associative memory retrieval. Neural Computation. 2013. Vol. 25, N 10. P. 2523–2544. https://doi.org/10.1162/NECO_a_00499

144 Rosenfeld R., Touretzky D.S. Coarse-coded symbol memories and their properties. Complex Systems. 1988. Vol. 2, N 4. P. 463–484.

145 Salavati A.H., Kumar K.R., Shokrollahi A. Nonbinary associative memory with exponential pattern retrieval capacity and iterative learning. IEEE Trans. Neural Networks and Learning Systems. 2014. Vol. 25, N 3. P. 557–570. https://doi.org/10.1109/TNNLS.2013.2277608

146 Schwenker F., Sommer F.T., Palm G. Iterative retrieval of sparsely coded associative memory patterns. Neural Networks. 1996. Vol. 9. P. 445-455. https://doi.org/10.1016/0893-6080(95)00112-3

147 Shrivastava A., Li P. In defense of minhash over simhash. Proc. AISTATS’14. 2014. P. 886–894.

148 Slipchenko S.V., Rachkovskij D.A. Analogical mapping using similarity of binary distributed representations. International Journal Information Theories and Applications. 2009. Vol. 16, N 3. P. 269–290.

149 KO5. Tarkoma S., Rothenberg C. E., Lagerspetz E. Theory and Practice of Bloom Filters for Distributed Systems. IEEE Communications Surveys and Tutorials. 2012. Vol. 14, N 1. P. 131–155. https://doi.org/10.1109/SURV.2011.031611.00024

150 Tsodyks M. Associative memory in asymmetric diluted network with low level of activity. Europhysics Letters. 1988. Vol. 7, N 3. 203–208. https://doi.org/10.1209/0295-5075/7/3/003

151 Tsodyks M. Associative memory in neural networks with the hebbian learning rule. Modern Physics Letters B. 1989. Vol. 3, N 7. P. 555–560. https://doi.org/10.1142/S021798498900087X

152 Tsodyks M.V. Associative memory in neural networks with binary synapses. Mod. Phys. Lett. 1990. Vol. B4. P. 713–716. https://doi.org/10.1142/S0217984990000891

153 Tsodyks M.V. Hierarchical associative memory in neural networks with low activity level. Modern Physics Letters B. 1990. Vol. 4, N 4. P. 259–265. https://doi.org/10.1142/S0217984990000325

154 Tsodyks M., Feigelman M. The enhanced storage capacity in neural networks with low activity level. Europhysics Letters. 1988. Vol. 6, N 2. P. 101–105. https://doi.org/10.1209/0295-5075/6/2/002

155 Vedenov A. A., Ezhov A.A., Knizhnikova L.A., Levchenko E.B. “Spurious memory” in model neural networks. Preprint IAE-4395/1. 1987. Moscow: KIAE.

156 Willshaw D. Holography, associative memory and inductive generalization. In Parallel Models of Associative Memory. Hillside: Lawrence Erlbaum Associates. 1981. P. 83–104.

157 Willshaw D. J., Buneman O. P., Longuet-Higgins H. C. Non-holographic associative memory. Nature. Vol. 1969. Vol. 222. P. 960–962. https://doi.org/10.1038/222960a0

158 Yang X., Vernitski A., Carrea L. An approximate dynamic programming approach for improving accuracy of lossy data compression by Bloom filters. European Journal of Operational Research. 2016. Vol. 252, N 3. P 985–994. https://doi.org/10.1016/j.ejor.2016.01.042

159 Yu C., Gripon V., Jiang X., Jegou H. Neural associative memories as accelerators for binary vector search. Proc. COGNITIVE’15. 2015. P. 85–89.

Recieved 15.04.2017

admin

Issue 1 (187), article 1

DOI: https://doi.org/10.15407/kvt187.01.005

Kibern. vyčisl. teh., 2017, Issue 1 (187), pp.5-11

Grytsenko V.I., Corresponding Member of NAS of Ukraine, Director of International
Research and Training Center for Information Technologies and Systems of National
Academy of Sciences of Ukraine and Ministry of Education and Science of Ukraine

e-mail: vig@irtc.org.ua

20 YEARS OF THE INTERNATIONAL RESEARCH AND TRAINING CENTER FOR INFORMATION TECHNOLOGIES AND SYSTEMS

May 5, 1997 the International Research and Training Center for Information Technologies and Systems NAS and MES of Ukraine was established by National Academy of Sciences of Ukraine.

During 20 years new scientific direction — Intelligent Information Technology (IIT), was formed. This methodology, the software and hardware became the basis for the deve-lopment of IIT of imaginative thinking, neural network technology, IIT for digital medicine, the E-education and intelligent control technologies.

The basic directions of fundamental and applied scientific research in the International Center are: creation of intelligent information technologies based on methods and means of imaginative thinking, comprehensive research of problems of intelligent management, intelligent robotics, digital medicine, e-learning, digital information space and technologies for the development of a secure information society.

By the main directions of the International Center, scientific schools in the field of information technologies and systems, technical cybernetics, biological and medical cybernetics, and mathematical analysis of comprehensive economic systems have been formed. An important contribution to the development of these scientific schools was made by outstanding Ukrainian scientists — academicians V.I. Skurikhin, A.G. Ivakhnenko,
N.M. Amosov and A.A. Bakaev. Their students and followers successfully develop these scientific directions in our country and abroad.

The International Center is the initiator of research and development of the concept of a new class of information technologies — intelligent information technologies. These are special, knowledge-intensive information technologies that differ from the known IT in the new quality — operating images of information objects. At the same time, an understanding of human speech, recognition of real and artificially created objects, active interaction with the environment, revealing the essence of the phenomenon, operating knowledge and the choice of strategy and tactics for achieving the set goal are achieved through the contours of intellectual IT.

Technical Committee for Standardization of information technologies, scientific journals “Control Systems and Computers” and “Cibernatics and Computer Engineering”, presentations of our scientists at prestigious international conferences, symposia and exhibitions make an important contribution for increasing the authority of the International Center.

The International Center has formed a program of work for the nearest years and defined the mechanisms for its implementation in the context of the rapid development of intellectualization of information technologies in all spheres of our society. As the comprehensive analysis showed, this program fully corresponds to global trends that the term “digital transformation” characterizes and covers the research priorities in information technology for a period of 5–10 years.

Keywords: intelligent information technology, imaginative thinking, intelligent management, digital medicine, e-learning, robotics, information society.

Download full text (ua)!