Issue 4 (194), article 3
DOI:https://doi.org/10.15407/kvt194.04.041
Kibern. vyčisl. teh., 2018, Issue 4 (194), pp.
Stepashko V.S., DSc (Engineering), Professor,
Head of Dep. for Information Technologies of Inductive Modeling
e-mail: stepashko@irtc.org.ua
International Research and Training Center for Information Technologies
and Systems of the National Academy of Sciences of Ukraine
and Ministry of Education and Science of Ukraine,
Acad. Glushkov av., 40, Kyiv, 03187, Ukraine
FORMATION AND DEVELOPMENT OF SELF-ORGANIZING INTELLIGENT TECHNOLOGIES OF INDUCTIVE MODELING
Introduction. Effective solution of control and decision-making tasks in complex systems should use the results of mathematical modeling. To construct adequate predictive models, many modern methods and tools are available which may be generally based on two principal approaches: theory-driven (deductive) and data-driven (inductive) ones. The data-driven methods are basic for solving typical tacks of data mining; they implement an inductive process of transition from particular data to models generalizing the data. Among all such methods, very notable are those being developed within the area of GMDH-based inductive modeling founded several decades ago by academician O.H. Ivakhnenko.
The purpose of this paper is analysing the background of the GMDH invention by Ivakhnenko and the evolution of model self-organization ideas, methods and tools during the half-century historical period of successful development of the inductive modeling methodology.
Results. Professor Ivakhnenko acquired broad knowledge in the areas of automatic control, engineering cybernetics and emerging neuroscience initiated by the idea of percep-tron. These were those prerequisites which helped Ivakhnenko to synthesize his original self-organizing approach to solving tasks of constructing models of objects and processes on the basis of experimental data. The paper tracks evolution of scientific ideas and views of Ivakhnenko and main achievements in development of GMDH during the period 1968-1997. Contributions of researchers from different countries to the GMDH modification and application are characterized. Results of further developments of inductive modeling meth-ods and tools in the ITIM department are presented and the most promising prospects of in-vestigations in this field are indicated.
Conclusions. Main prerequisites facilitating the creation of the GMDH by O.H. Ivakhnenko were analysed, basic fundamental, technological and applied achievements of the half-century development of inductive modeling both in Ukraine and abroad were characterized, as well as the most prospective ways of further research were formulated.
Keywords: mathematical modeling, data-driven modeling, model self-organization, GMDH, inductive modeling, noise-immune modeling, information technology, case study.
REFERENCES
1. Ivakhnenko A.G., Muller J.-A. Recent Developments of Self-Organizing Modeling in Prediction and Analysis of Stock Market. Microelectronics Reliability. 1997. No. 37. P. 1053–1072.
2. Anastasakis L., Mort N. The Development of Self-Organization Techniques in Modelling: A Review of the Group Method of Data Handling (GMDH). ACSE Research Report 813. The University of Sheffield, 2001. 39 p.
3. Snorek M., Kordik P. Inductive Modelling World Wide the State of the Art. Proc. of 2nd Int. Workshop on Inductive Modelling (Prague, 19—23rd of Sept., 2007) Prague, 2007. P. 302–304.
4. Stepashko V. Developments and Prospects of GMDH-Based Inductive Modeling. In: Advances in Intelligent Systems and Computing II: Selected Papers from the Intern. Conf. on Computer Science and Information Technologies, CSIT 2017, Lviv, Ukraine / N. Shakhovska, V. Stepashko, Editors. AISC book series, Vol. 689. Cham: Springer, 2018, pp. 474–491. https://doi.org/10.1007/978-3-319-70581-1_34
5. Ivakhnenko A.G. Electroautomatics. Kiev: Gostekhizdat UkrSSR, 1957. 452 p. (In Russian)
6. Ivakhnenko A.G., Petina N.V. Voltage stabilizers with combined control. Kiev: AS UkrSSR publisher, 1958. 247 p. (In Russian)
7. Ivakhnenko A.G. Engineering cybernetics. Kiev: Gostekhizdat UkrSSR, 1959. 432 p. (In Russian)
8. Ivakhnenko A.G., Lapa V.G. Cybernetic predicting devices. Kiev: Naukova dumka, 1965. 213 p. (In Russian)
9. Rosenblatt F. Principles of Neurodynamic: Perceptrons and the Theory of Brain Mechanisms. Washington: Spartan Books, 1962. 616 p.
10. Ivakhnenko A.G. Method of Group Using of Arguments as a Rival of Stochastic Approximation Method. Avtomatyka. 1968. No 3. P. 58–72. (In Ukrainian)
11. Ivakhnenko A.G. Group Method of Data Handling as a Rival of Stochastic Approximation Method. Soviet Automatic Control. 1968. No. 3. P. 43–55.
12. Ivakhnenko A.G. Heuristic Self-Organization in Problems of Automatic Control. Automatica (IFAC). 1970. No. 6. P. 207–219. https://doi.org/10.1016/0005-1098(70)90092-0
13. Ivakhnenko A.G. Polynomial theory of complex systems. IEEE Trans. Sys., Man and Cyb. 1971. 1, No 4. P. 364–378.
14. Ivakhnenko A.G. Heuristic self-organization systems in engineering cybernetics. Kiev: Tekhnika, 1971. 392 p. (In Russian)
15. Ivakhnenko A.G. Inductive method of self-organization of complex systems. Kiev: Naukova dumka, 1982. 296 p. (In Russian)
16. Madala H.R., Ivakhnenko A.G. Inductive Learning Algorithms for Complex Systems Modeling. London, Tokyo: CRC Press Inc., 1994. 384 p.
17. Ivakhnenko A.G., Zaichenko Yu.P., Dimitrov V.D. Decision making based on self-organization. Moscow: Sov. radio, 1976. 280 p. (In Russian)
18. Ivakhnenko A.G., Stepashko V.S. Noise-immunity of modeling. Kiev: Naukova dumka, 1985. 216 p. (In Russian)
19. Ivakhnenko A.G., Yurachkovskiy Yu.P. Modeling of complex systems from experimental data. Moscow: Radio i svyaz, 1987. 120 p. (In Russian)
20. Stepashko V.S. A Combinatorial Algorithm of the Group Method of Data Handling with Optimal Model Scanning Scheme. Soviet Automatic Control. 1981. 14(3). P. 24–28.
21. Stepashko V.S. A Finite Selection Procedure for Pruning an Exhaustive Search of Models. Soviet Automatic Control. 1983. 16(4). P. 88–93.
22. Shelud’ko O.I. GMDH Algorithm with Orthogonalized Complete Description for Synthesis of Models by the Results of a Planned Experiment. Soviet Automatic Control. 1974. 7(5). P. 24–33.
23. Yurachkovsky Yu.P. Convergence of Multilayer Algorithms of the Group Method of Data Handling. Soviet Automatic Control. 1981. 14(3). P. 29–34.
24. Yurachkovsky Yu.P. Restoration of Polynomial Dependencies Using Self-Organization. Soviet Automatic Control. 1981. 14(4). P. 17–22.
25. Ivakhnenko A.G. Long-term forecasting and control of complex systems. Kiev: Tekhnika, 1975. 311 p. (In Russian)
26. Ivakhnenko A.G., Karpinsky A.M. Computer-Aided Self-Organization of Models in Terms of the General Communication Theory (Information Theory). Soviet Automatic Control. 1982. 15(4). P. 7–15.
27. Stepashko V.S. Potential noise stability of modelling using the combinatorial GMDH algorithm without information regarding the noise. Soviet Automatic Control. 1983. 16(3). P. 15–25.
28. Stepashko V.S., Kostenko Yu.V. A GMDH Algorithm for Two-level Modeling of Multidimensional Cyclic Processes. Soviet Automatic Control. 1987. 20(4). P. 49–57.
29. Ivakhnenko A.G., Osipenko V.V., Strokova T.I. Prediction of Two-dimensional Physical Fields Using Inverse Transition Matrix Transformation. Soviet Automatic Control. 1983. 16(4). P. 10–15.
30. Ivakhnenko A.G. Inductive Sorting Method for the Forecasting of Multidimensional Random Processes and Events with the Help of Analogs Forecast Complexing. Pattern Recogn. and Image Analysis. 1991. 1(1). P. 99–108.
31. Ivakhnenko A.G. Kostenko Yu.V. System Analysis and Long-Term Prediction on the Basis of Model Self-organisation (OSA algorithm). Soviet Automatic Control. 1982. 15(3). P. 11–17.
32. Ivakhnenko A.G. Objective Computer Clasterization Based on Self-Organisation Theory. Soviet Automatic Control. 1987. 20(6). P. 1–7.
33. Vysotskiy V.N., Ivakhnenko A.G., Cheberkus V.I. Long Term Prediction of Oscillatory Processes by Finding a Harmonic Trend of Optimum Complexity by the Balance-of-Variables Criterion. Soviet Automatic Control. 1975. 8(1). P. 18–24.
34. Ivakhnenko A.G., Krotov G.I. A Multiplicative-Additive Nonlinear GMDH Algorithm with Optimization of the Power of Factors. Soviet Automatic Control. 1984. 17(3). P. 10–15.
35. Kocherga Yu.L. J-optimal Reduction of Model Structure in the Gauss-Markov Scheme. Soviet J. of Automation and Information Sciences. 1988. 21(4). P. 34–36.
36. Aksenova T.I., YurachkovskyYu.P. A Characterization at Unbiased Structure and Conditions of Their J-Optimality. Sov. J. of Automation and Information Sciences. 1988. 21(4). P.36–42.
37. Ivakhnenko A.G., Kovalchuk P.I., Todua M.M., Shelud’ko O.I., Dubrovin O.F. Unique Construction of Regression Curve Using a Small Number of Points — Part 2. Soviet Automatic Control. 1973. 6(5). P. 29–41.
38. Stepashko V.S. Asymptotic Properties of External Criteria for Model Selection Soviet Journal of Automation and Information Sciences. 21, No. 6. (1988). P. 84–92.
39. Aksenova T.I. Sufficient conditions and convergence rate using different criteria for model selection, Systems Analysis Modelling Simulation 1995. vol. 20, no. 1–2. P.69–78.
40. Ivakhnenko A.G., Ivakhnenko G.A., Mueller J.A. Self-Organization of Neuronets with Active Neurons. Pattern Recognition and Image Analysis. 1994. 4(4). P. 177–188.
41. Muller J.-A., Lemke F. Self-organizing data mining. An intelligent approach to extract knowledge from data. Berlin, Dresden: Libri BoD, 1999. 225 p.
42. Self-organizing methods in modeling: GMDH type algorithms / Ed. S.J. Farlow. New York, Basel: Marcel Decker Inc., 1984. 350 p.
43. Voss M.S., Xin Feng. A new methodology for emergent system identification using particle swarm optimization (PSO) and the group method of data handling (GMDH). Proc. of the Genetic and Evolutionary Computation Conference. Morgan Kaufmann Publishers, 2002. (9 – 13th of July, 2002, New-York). New-York, 2002. P. 1227–1232.
44. Kondo T., Ueno J. Feedback GMDH-Type Neural Network Self-Selecting Optimum Neural Network Architecture and Its Application to 3-Dimensional Medical Image Recognition of the Lungs. Proc. of the II Intern. Workshop on Inductive Modelling IWIM-2007 (Prague, 19-23rd of Sept. 2007) Prague, 2007. P. 63–70.
45. Jirina M., Jirina M. jr. Genetic Selection and Cloning in GMDH MIA Method. Proc. of the II Intern. Workshop on Inductive Modelling IWIM 2007 (Prague, 23–26th of Sept., 2007) Prague, 2007. P. 165–171.
46. Lytvynenko V., Bidyuk P., Myrgorod V. Application of the Method and Combined Algorithm on the Basis of Immune Network and Negative Selection for Identification of Turbine Engine Surging. Proc. of the II Intern. Conf. on Inductive Modelling ICIM-2008 (Kyiv, 15–19th of Sept. 2008). Kyiv, 2008. P. 116–123.
47. Kordik P. Fully automated knowledge extraction using group of adaptive model evolution: PhD thesis. Prague: CTU, 2006. 150 p.
48. Oh S.K., Pedrycz W., Park H.S. Multi-layer hybrid fuzzy polynomial neural networks: a design in the framework of computational intelligence. Neurocomputing. 2005. 64. P. 397–431. https://doi.org/10.1016/j.neucom.2004.08.001
49. Zaychenko Yu. The Investigations of Fuzzy Group Method of Data Handling with Fuzzy Inputs in the Problem of Forecasting in Financial Sphere. Proc. of the II Intern. Conf. on Inductive Modelling ICIM-2008. (Kyiv, 15–19th of Sept., 2008). Kyiv, 2008. P. 129–133.
50. Bodyanskiy Ye., Vynokurova O., Teslenko N. Cascade GMDH-Wavelet-Neuro-Fuzzy Network. Proc. of the IV Intern. Workshop on Inductive Modelling IWIM-2011 (Kyiv-Zhukyn, 4–11th of July, 2011). Kyiv-Zhukyn, 2011. P. 16–21.
51. Voytyuk I., Dyvak M., Spilchuk V. The Method of Structure Identification of Macromodels as Difference Operators Based on the Analysis of Interval Data and Genetic Algorithm. Proc. of the IV Intern. Workshop on Inductive Modelling IWIM-2011 (Kyiv-Zhukyn 4–11th of July, 2011). Kyiv-Zhukyn, 2011. P. 114–118.
52. Lemke F. Parallel Self-Organizing Modeling. Proc. of the II Int. Conf. on Inductive Modelling ICIM-2008 (Kyiv, 15-19th of Sept. 2008). Kyiv, 2008. P. 176–183.
53. Koshulko O.A., Koshulko A.I. Multistage combinatorial GMDH algorithm for parallel processing of high-dimensional data. Proc. of III Int. Workshop on Inductive Modelling IWIM-2009. (15-19th of Sept., 2009, Krynica). Krynica, Poland, 2009. Prague: CTU, 2009. P. 114–116.
54. Kordik P., Cerny J. Advanced Ensemble Strategies for Polynomial Models. Proc. of the III Intern. Conf. on Inductive Modelling ICIM-2010 (Yevpatoria, 16–22nd of May, 2010). Yevpatoria, 2010. Kherson: KNTU, 2010. P. 77–82.
55. Cepek M., Kordik P., Snorek M. The Effect of Modelling Method to the Inductive Preprocessing Algorithm. Proc. of the III Intern. Conf. on Inductive Modelling ICIM-2010. (Yevpatoria, 16–22nd of May 2010) Yevpatoria, 2010. Kherson: KNTU, 2010. P. 131–138.
56. Sarychev A.P. System Regularity Criterion of Group Method of Data Handling. Journal of Automation and Information Sciences. 2006. 38(11). P. 25–37. https://doi.org/10.1615/J
Automat Inf Scien.v38.i11.30
57. Sarycheva L. Quality Criteria for GMDH-based Clustering. Proc. of the II International Conference on Inductive Modelling ICIM-2008 (Kyiv, 15–19th of Sept., 2008), Kyiv, 2008.
58. Lemke, F. Insights v.2.0, Self-organizing knowledge mining and forecasting tool, 2013.URL: http://www.knowledgeminer.eu. (Last accessed: 01.11.2018)
59. URL: https://www.gmdhshell.com. (Last accessed: 15.11.2018)
60. URL: www.mgua.irtc.org.ua (Last accessed: 01.12.2018)
61. Stepashko V.S. Method of Critical Variances as Analytical Tool of Theory of Inductive Modeling. Journal of Automation and Information Sciences. 2008. 40(2). P. 4–22. https://doi.org/10.1615/JAutomatInfScien.v40.i3.20
62. Ivakhnenko A.G., Savchenko E.A. Investigation of Efficiency of Additional Determination Method of the Model Selection in the Modeling Problems by Application of the GMDH Algorithm. Journal of Automation and Information sciences. 2008. 40(3). P. 47–58. https://doi.org/10.1615/JAutomatInfScien.v40.i3.50
63. Stepashko V.S., Efimenko S.M. Sequential Estimation of the Parameters of Regression Models. Cybernetics and Systems Analysis. 2005. 41(4). P.631–634. https://doi.org/10.1007/s10559-005-0099-3
64. Stepashko V., Yefimenko S. Parallel algorithms for solving combinatorial macromodelling problems. Przeglad Elektrotechniczny (Electrical Review). 2009. 85(4). P 98–99.
65. Samoilenko O., Stepashko V. Method of Successive Elimination of Spurious Arguments for Effective Solution the Search-Based Modelling Tasks. Proc. of the II Intern. Conf. on Inductive Modelling ICIM-2008 (Kyiv,15–19th of Sept. 2008), Kyiv, 2008. P. 36–39.
66. Moroz O., Stepashko V. Hybrid sorting-out algorithm COMBI-GA with evolutionary growth of model complexity. In: Advances in Intelligent Systems and Computing II: Selected Papers from the International Conference on Computer Science and Information Technologies, CSIT 2017, Lviv, Ukraine / N. Shakhovska, V. Stepashko, Ed. AISC, Vol. 689. Cham: Springer, 2018. P. 346–360. https://doi.org/10.1007/978-3-319-70581-1_25
67. Stepashko V., Bulgakova O., Zosimov V. Construction and Research of the Generalized Iterative GMDH Algorithm with Active Neurons. In: Advances in Intelligent Systems and Computing II: Selected Papers from the International Conference on Computer Science and Information Technologies, CSIT 2017, Lviv, Ukraine / N. Shakhovska, V. Stepashko, Editors. AISC book series, Vol. 689. Cham: Springer, 2018. P. 492–510. https://doi.org/10.1007/978-3-319-70581-1_35
68. Pavlov A.V. Generalized relaxational iterative algorithm of GMDH. Inductive Modeling of Complex Systems. Collected papers. Issue 3. Kyiv: IRTC ITS NASU, 2011. P. 121–134. (In Ukrainian)
69. Stepashko, V.S.: Conceptual fundamentals of intelligent modeling. Control Systems and Machines (USiM). 2016. 4, pp. 3–15. (In Russian)
70. Yefimenko S.N., Stepashko V.S. Fundamentals of recurrent-and-parallel computing in the combinatorial algorithm COMBI GMDH. USiM. 2014. 6. P. 27–33. (In Russian)
71. Yefimenko S.N., Stepashko V.S. Simulation experiment as a means of effectiveness research of modeling methods from observation data. USiM. 2009. 1. P. 69–78. (In Russian)
72. Bulgakova O., Zosimov V., Stepashko V. Software package for modeling of complex systems based on iterative GMDH algorithms with the network access capability. System Research and Information Technologies. 1. 2014. P. 43–55. (In Ukrainian)
73. Pavlov A. Designing an automated structural-parametric identification system. Inductive Modeling of Complex Systems. Collected papers. Issue 7. Kyiv: IRTC ITS NASU, 2015. P. 202–219. (In Ukrainian)
74. Yefimenko S. Building Vector Autoregressive Models Using COMBI GMDH with Recurrent-and-Parallel Computations. In: Advances in Intelligent Systems and Computing II: Selected Papers from the International Conference on Computer Science and Information Technologies, CSIT 2017, Lviv, Ukraine / N. Shakhovska, V. Stepashko, Editors. AISC book series, Vol. 689. Cham: Springer, 2018. P. 601–613. https://doi.org/10.1007/978-3-319-70581-1_42
75. Stepashko V., Samoilenko O., Voloschuk R. Informational Support of Managerial Decisions as a New Kind of Business Intelligence Systems. In: Computational Models for Business and Engineering Domains. G. Setlak, K. Markov (Eds.). Rzeszow, Poland; Sofia, Bulgaria: ITHEA. 2014. P. 269–279.
76. Moroz O., Stepashko V. Data reconstruction of seasonal changes of amylolytic microorganisms amount in copper polluted soils. Proc. of the 13th IEEE Intern. Conf. CSIT-2018 & International Workshop on Inductive Modeling. (Lviv, 11–14th of Sept., 2018), Lviv, 2018. P. 479–482. https://doi.org/10.1109/STC-CSIT.2018.8526587
77. Stepashko V.S., Yefimenko S.M., Savchenko Ye.A. Computerized experiment in inductive modeling. Kyiv: Naukova Dumka, 2014. 222 p. (In Ukrainian)
78. Pavlov A.V., Stepashko V.S., Kondrashova N.V. Effective methods of models self-organization. Kyiv: Akademperiodika, 2014. 200 p. (In Russian)
79. Stepashko V.S., Bulgakova O., Zosimov V. Iterational algorithms of inductive modeling. Kyiv: Naukova Dumka, 2014. 190 p. (In Ukrainian)
80. Schmidhuber J. Deep learning in neural networks: An overview. Neural Networks. 2015. 61, pp. 85–117. https://doi.org/10.1016/j.neunet.2014.09.003
Received 20.09.2018