DOI:https://doi.org/10.15407/kvt213.03.004
Cybernetics and Computer Engineering, 2023, 3(213)
Kyyko V.M., PhD (Engineering), Senior Researcher of the Pattern Recognition Department, https://orcid.org/0009-0005-6605-0339,
e-mail: vkiiko@gmail.com,
Matsello V.V., PhD (Engineering), Head of the Pattern Recognition Department, https://orcid.org/0000-0001-7640-4077,
e-mail: matsello@gmail.com
International Research and Training Center for Information Technologies and Systems of the National Academy of Sciences of Ukraine and the Ministry of Education and Science of Ukraine, 40, Acad. Glushkova av., Kyiv, 03187, Ukraine
REAL-TIME TRACKING OF OBJECTS IN VIDEO BASED ON ADAPTIVE HISTOGRAM FEATURES
Introduction. Object tracking in video is one of the open problems in computer vision and has a wide range of practical applications. The main difficulties of this task are that the object in the process of tracking can significantly change its appearance due to changes in lighting conditions, size and orientation in space, as well as disappear from the field of view. Analysis of known algorithms shows that each of them does not fully ensure reliable tracking of objects under the above conditions. One approach to improving tracking reliability is to develop a means of using multiple algorithms that complement each other in their capabilities.
The purpose of the paper is to develop an algorithm for long-term real-time tracking of objects in video based on the use of complementary features and algorithms to obtain more reliable tracking results in difficult conditions.
The results. An algorithm for long-term real-time tracking of objects in video has been developed based on the combined use of two algorithms with complementary features and capabilities – the well-known KCF algorithm with HOG features of brightness gradients and the developed CH algorithm using HSV histogram features of color representations of the object and background. It is shown that the algorithm has wider possibilities for its use compared to KCF and CH filters. The developed algorithm was tested on video from the VOT (Visual Object Tracking) database.
Conclusions. The developed algorithm ensures restoration of object localization after its disappearance from the field of view, as well as increasing the accuracy and reliability of localization in comparison with KCF and CH algorithms. Localization recovery is performed by searching for an object on an enlarged area of the image using KCF or another algorithm. The high-speed CH algorithm is used to preliminarily reduce the number of cells in the search area that can match the object and reduce its search time. Increasing the accuracy and reliability of localization is achieved by using a more informative criterion in the form of a weighted sum of the responses of two filters, as well as a more accurate definition of the rectangle bounding the object based on the segmentation of the color representation of the image.
Keywords: object tracking in video, KCF tracking algorithm, HOG features, histogram features of colors in the image.
Download full text!
REFERENCES
1 C. Bao, Y.Wu, H. Ling, and H. Ji. Real time robust l1 tracker using accelerated proximal gradient approach. In CVPR, pp.1830-1837, 2012.
2 B. Liu, J. Huang, L. Yang, and C. Kulikowski. Robust tracking using local sparse appearance model and k-selection. In CVPR, pp. 1313-1320, 2011.
https://doi.org/10.1109/CVPR.2011.5995730
3 J. F. Henriques, R. Caseiro, P. Martins, and J. Batista. High-speed tracking with kernelized correlation filters. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), vol. 37, no. 3, pp. 583-596, 2015.
https://doi.org/10.1109/TPAMI.2014.2345390
4 S. Hare, A. Saffari, and P. Torr. Struck: Structured output tracking with kernels. In IEEE Trans. on PAMI, Vol. 2, No 7, pp. 1-14, 2015.
5 M. Danelljan, F. S. Khan, M. Felsberg, and J. van de Weijer. Adaptive Color Attributes for Real-Time Visual Tracking. In Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA, 23-28 June 2014; pp. 1090-1097.
https://doi.org/10.1109/CVPR.2014.143
6 Weiwei Xing, Weibin Liu, Jun Wang, Shunli Zhang, Lihui Wang, Yuxiang Yang, Bowen Song. Visual Object Tracking from Correlation Filter to Deep Learning. Springer Nature, 2021, P. 193.
https://doi.org/10.1007/978-981-16-6242-3
7 L. Bertinetto, J. Valmadre, S. Golodetz, O. Miksik, P.H. Torr. Staple: Complementary learners for real-time tracking. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27-30 June 2016, pp. 1401-1409.
https://doi.org/10.1109/CVPR.2016.156
8 Zhou, H. Fu, S. You, and C. Kuo. Unsupervised lightweight single object tracking with UHP-SOT++. ArXiv: 2111.07548, 2021, P. 13.
9 M. Dunnhofer and C. Micheloni. CoCoLoT: Combining Complementary Trackers in Long-Term Visual Tracking, arXiv: 2205, 04261v1 [cs.CV] 9 May, 2022, P. 8.
https://doi.org/10.1109/ICPR56361.2022.9956082
10 Kyyko V.M., Matsello V.V. Object tracking by co-operative actions of background Control systems and machines, 2020. No 2, pp. 23-29. (In Ukrainian).
https://doi.org/10.15407/csc.2020.02.012
11 N. Dalal, B. Triggs. Histograms of oriented gradients for human detection. In Proc. of the IEEE Conf. on CVPR, San Diego, USA, 20-25 June 2005; pp. 886-893.
12 Felzenszwalb, P.F.; Girshick, R.B.; McAllester, D. Object detection discriminatively trained part-based models, IEEE Trans. PAMI. 2009 32, pp. 1627-1645.
https://doi.org/10.1109/TPAMI.2009.167
13 Schlesinger, M.I. Fast implementation of one class of linear convolution. Theoretical and applied questions of image recognition: Kiev: Institute of Cybernetics, 1991, pp. 61-69. (In Russian).
14 J. van de Weijer, C. Schmid, J. J. Verbeek, and D. Larlus. Learning color names for real-world applications. TIP, 18(7):1512-1524, 2009.
https://doi.org/10.1109/TIP.2009.2019809
15 T. Ojala, M. Pietika.inen, and T. Ma.enpa. Multiresolution Gray Scale and Rotation Invariant Texture Classification with Local Binary Patterns. IEEE Trans. PAMI, vol. 24, no. 7, pp. 971-987, July 2002.
https://doi.org/10.1109/TPAMI.2002.1017623
16 Chao Ma, Xiaokang Yang, Chongyang Zhang1, and Ming-Hsuan Yang. Long-term Correlation Tracking. IEEE Conf. On Computer Vision and Pattern Recognition (CVPR), 2015, pp. 5388-5396.
17 Schlesinger, M.I. Pattern recognition as an implementation of a certain subclass of thought processes. Control systems and machines, 2017. No 2, pp. 20-37. (In Russian).
https://doi.org/10.15407/usim.2017.02.020
18 Yuri Boykov, Gareth Funka-Lea. Graph Cuts and Efficient N-D Image Segmentation. Int. Journal of Computer Vision, Vol. 70, 2006, pp. 109-131.
https://doi.org/10.1007/s11263-006-7934-5
19 Q. Chen and V. Koltun. Fast mrf optimization with application to depth reconstruction. in Proc. of IEEE Conf. on Computer Vision and Pattern Recognition. 2014, pp. 3914-3921.
https://doi.org/10.1109/CVPR.2014.500
20 Nobuyuki Otsu. A threshold selection method from gray-level histograms. IEEE Trans. Sys., Man., Cyber. 9(1) 1979, pp. 62-66.
https://doi.org/10.1109/TSMC.1979.4310076
21 M. Everingham, L. J. V. Gool, C. K. I. Williams, J. M. Winn, and A. Zisserman. The pascal visual object classes (VOC) challenge. IJCV, vol. 88, no. 2, pp. 303-338, 2010.
https://doi.org/10.1007/s11263-009-0275-4
Received 23.06.2023