DOI:https://doi.org/10.15407/kvt214.04.040
Cybernetics and Computer Engineering, 2023, 4(214)
Melnychenko A.S., PhD Student,
the Pattern Recognition Department
https://orcid.org/0009-0009-2445-8271
e-mail: toscha.1232013@gmail.com
Vodolazskyi Ye. V. , PhD Engineering,
Senior Researcher, the Pattern Recognition Department
https://orcid.org/0000-0003-3906-256X
e-mail: waterlaz@gmail.com
International Research and Training Center for Information
Technologies and Systems of the National Academy of Science
of Ukraine and the Ministry of Education and Science of Ukraine
40, Acad. Glushkov av., 03187, Kyiv, Ukraine
TEXTURE MISSING PARTS GENERATION BASED ON IMAGE STATISTICAL ANALYSIS
Introduction. Restoration of damaged images is a long lasting problem that currently does not have a generalized solution. Many methods which are being used nowadays are damage type specific, which means that for each case of damaged image an algorithm must be picked by a human. A state of the art generative algorithms, which may handle many of the damage types, still lack the precision and require huge training datasets. Thus an algorithm that is able to handle most common damage types and does not demand lots of time and computational power is still in need.
The purpose of the paper is to research the current state of the art algorithms that solve texture missing part generation problem as well as to propose a new method, which might provide both precision and ease of use for solving said problem for most of the damage types using the same approach.
Methods. Research and analytics are used for processing found literature on the topic to substantiate the main approaches and best practices for the solution of the texture missing parts generation problem. As for purposed method, Gibbs sampling is used as a means of generating missing pixels of the image. Some additional algorithms, which might be used to generate probabilistic distribution for sampler and the means of getting the pixel value from the sampling process, are mentioned in the article itself.
Results. State of the art approaches for solving texture missing parts generation are analyzed and compared. Main groups of generative, texture reparation, gradient filling and combined methods are described and compared. New method for generating missing parts of the texture based on statistical analysis of the scene images is proposed. The generation of the pixel values in said method is based on Gibbs sampling. The first results of purposed method with patch based probabilistic distribution generation are shown.
Conclusions. The proposed Gibbs sampling based method is able to provide results, which are comparable with those generated by other modern methods. As a future work, it is planned to develop new more sophisticated and precise patches matching algorithms as well as to research other methods of both generating probability distribution and gathering pixel value from the sampling process.
Keywords: Gibbs sampling, texture restoration, image restoration, patches matching
REFERENCES
1 Tien-Ying Kuo, Yu-Jen Wei, Po-Chyi Su and Tzu-Hao Lin. Learning-Based Image Damage Area Detection for Old Photo Recovery. Sensors. Vol. 22. No. 21. 2022. 8580.
https://doi.org/10.3390/s22218580
2 Peyman Milanfar. A Tour of Modern Image Filtering: New Insights and Methods, Both Practical and Theoretical. IEEE Signal Processing Magazine. Vol. 30. No. 1. 2013. P. 106-126.
https://doi.org/10.1109/MSP.2011.2179329
3 Pitas I., Venetsanopoulos A.N. Median Filters. Nonlinear Digital Filters. Vol 84. 1990. P. 63-116.
https://doi.org/10.1007/978-1-4757-6017-0_4
4 Kamel Boukerrou, Ludwik Kurz. Suppression of “salt and pepper” noise based on Youden designs. Information Sciences. Vol.110. No. 3-4. 1998. P. 217-235.
https://doi.org/10.1016/S0020-0255(98)00004-8
5 A. Polesel, G. Ramponi, V. J. Mathews. Image enhancement via adaptive unsharp masking. IEEE Transactions on Image Processing. Vol. 9, No. 3. 2000. P. 505-510.
https://doi.org/10.1109/83.826787
6 Rui Wang, Wei Li, Rui Li, Liang Zhang. Automatic blur type classification via ensemble SVM. Signal Processing: Image Communication. Vol. 71. 2019. P. 24-35.
https://doi.org/10.1016/j.image.2018.08.003
7 Yu Huihui, Li Daoliang, Chen Yingyi. A state-of-the-art review of image motion deblurring techniques in precision agriculture. Heliyon. Vol. 9. No. 6. 2023. P. e17332.
https://doi.org/10.1016/j.heliyon.2023.e17332
8 Norbert Wiener. Extrapolation, Interpolation, and Smoothing of Stationary Time Series. The MIT Press. New York. NY. USA. 1964.
9 W. H. Richardson, Bayesian-Based Iterative Method of Image Restoration. Journal of the Optical Society of America. Vol. 62, No. 1. 1972. P. 55-59.
https://doi.org/10.1364/JOSA.62.000055
10 L. B. Lucy. An iterative technique for the rectification of observed distributions. Astronomical Journal. Vol. 79. No. 6. 1974. P. 745-754.
https://doi.org/10.1086/111605
11 A. K. Katsaggelos, K. T. Lay. Maximum likelihood blur identification and image restoration using the EM algorithm. IEEE Transactions on Signal Processing. Vol. 39. No. 3. 1991. P. 729-733.
https://doi.org/10.1109/78.80894
12 Seung-Gu Kim. Explicit Solution of EM Algorithm in Image Deblurring: Image Restoration without EM iterations. Communications for Statistical Applications and Methods. Vol. 16. 2009. P. 409-419.
https://doi.org/10.5351/CKSS.2009.16.3.409
13 J. F. Cai, Hui Ji, Chaoqiang Liu, Z. Shen. Blind motion deblurring from a single image using sparse approximation. IEEE Conference on Computer Vision and Pattern Recognition. Miami. FL. USA. 2009. P. 104-111.
14 Dong Yang, Shiyin Qin. Restoration of Partial Blurred Image Based on Blur Detection and Classification. Journal of Electrical and Computer Engineering. 2016. P. 1-12.
https://doi.org/10.1155/2016/2374926
15 W. Dong, L. Zhang, G. Shi, X. Wu. Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization. IEEE Transactions on Image Processing. Vol. 20. No. 7. 2011. P. 1838-1857.
https://doi.org/10.1109/TIP.2011.2108306
16 Sudha Yadav, Charu Jain, Aarti Chugh. Evaluation of Image Deblurring Techniques. Vol. 139. No. 12. 2016. P. 32-36.
https://doi.org/10.5120/ijca2016909492
17 Shamik Tiwari and V. P. Shukla and Awantika Singh and S. R. Biradar. Review of Motion Blur Estimation Techniques. Journal of Image and Graphics. Vol. 1. No. 4. 2013. P. 176-184.
https://doi.org/10.12720/joig.1.4.176-184
18 D. Li, R. M. Mersereau, S. Simske. Atmospheric Turbulence-Degraded Image Restoration Using Principal Components Analysis. IEEE Geoscience and Remote Sensing Letters. Vol. 4. No. 3. 2007. P. 340-344.
https://doi.org/10.1109/LGRS.2007.895691
19 L. Yan, M. Jin, H. Fang, H. Liu, T. Zhang. Atmospheric-Turbulence-Degraded Astronomical Image Restoration by Minimizing Second-Order Central Moment. IEEE Geoscience and Remote Sensing Letters. Vol. 9. No. 4. 2012. P. 672-676.
https://doi.org/10.1109/LGRS.2011.2178016
20 Tristan Dagobert, Yohann Tendero, Stephane Landeau. Study of the Principal Component Analysis Method for the Correction of Images Degraded by Turbulence. Image Processing On Line. Vol. 8. 2018. P. 388-407.
https://doi.org/10.5201/ipol.2018.47
21 X. Hua, C. Pan, Y. Shi, J. Liu, H. Hong. Removing Atmospheric Turbulence Effects Via Geometric Distortion and Blur Representation. IEEE Transactions on Geoscience and Remote Sensing. Vol. 60. 2022. P. 1-13.
https://doi.org/10.1109/TGRS.2020.3043627
22 A. D. Dongare, R. R. Kharde, A. D. Kachare. Introduction to Artificial Neural Network. International Journal of Engineering and Innovative Technology. Vol. 2. No. 1. 2012.
23 Nantheera Anantrasirichai. Atmospheric turbulence removal with complex-valued convolutional neural network. Pattern Recognition Letters. Vol. 171. 2023. P. 69-75.
https://doi.org/10.1016/j.patrec.2023.05.017
24 Jiuming Cheng et al. Restoration of Atmospheric Turbulence-Degraded Short-Exposure Image Based on Convolution Neural Network. Photonics. Vol. 10. 2023. P. 666.
https://doi.org/10.3390/photonics10060666
25 M. Asim, F. Shamshad, A. Ahmed. Blind Image Deconvolution Using Deep Generative Priors. IEEE Transactions on Computational Imaging. Vol. 6. 2020. P. 1493-1506.
https://doi.org/10.1109/TCI.2020.3032671
26 Qian Zhao, Hui Wang, Zongsheng Yue, Deyu Meng. A deep variational Bayesian framework for blind image deblurring. Knowledge-Based Systems. V. 249. 2022. P. 109008.
https://doi.org/10.1016/j.knosys.2022.109008
27 Amudhavel Jayavel et al. Improved Classification of Blurred Images with Deep-Learning Networks Using Lucy-Richardson-Rosen Algorithm. Photonics. Vol. 10. No. 4. 2023.
https://doi.org/10.3390/photonics10040396
28 C. K. Liang, L. W. Chang, H. H. Chen. Analysis and Compensation of Rolling Shutter Effect. IEEE Transactions on Image Processing. Vol. 17. No. 8. 2008. P. 1323-1330.
https://doi.org/10.1109/TIP.2008.925384
29 D. Bradley, B. Atcheson, I. Ihrke, W. Heidrich. Synchronization and rolling shutter compensation for consumer video camera arrays. Proceedings of the International Workshop on Projector-Camera Systems. 2009.
https://doi.org/10.1109/CVPRW.2009.5204340
30 Simon Baker, Eric Bennett, Sing Bing Kang, Richard Szeliski. Removing rolling shutter wobble. The Twenty-Third IEEE Conference on Computer Vision and Pattern Recognition. San Francisco. CA. USA. 2010. P. 2392-2399.
https://doi.org/10.1109/CVPR.2010.5539932
31 M. Grundmann, V. Kwatra, D. Castro, I. Essa. Calibration-free rolling shutter removal. 2012 IEEE International Conference on Computational Photography. Seattle. WA. USA. 2012. P. 1-8.
https://doi.org/10.1109/ICCPhot.2012.6215213
32 Y. Lao, O. Ait-Aider. A Robust Method for Strong Rolling Shutter Effects Correction Using Lines with Automatic Feature Selection. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City. UT. USA. 2018. P. 4795-4803.
https://doi.org/10.1109/CVPR.2018.00504
33 P. Liu, Z. Cui, V. Larsson, M. Pollefeys. Deep Shutter Unrolling Network. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle. WA. USA. 2020. P. 5940-5948.
https://doi.org/10.1109/CVPR42600.2020.00598
34 B. Fan, Y. Mao, Y. Dai, Z. Wan, Q. Liu. Joint Appearance and Motion Learning for Efficient Rolling Shutter Correction. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver. BC. Canada. 2023. P. 5671-5681.
https://doi.org/10.1109/CVPR52729.2023.00549
35 Marcelo Bertalmio, Guillermo Sapiro, Vincent Caselles, Coloma Ballester. Image inpainting. Proceedings of the 27th annual conference on Computer graphics and interactive techniques. 2000. USA. P. 417-424.
https://doi.org/10.1145/344779.344972
36 M. Ashikhmin. Synthesizing natural textures. ACM Symposium on Interactive 3D Graphics. 2001. P. 217-226.
https://doi.org/10.1145/364338.364405
37 A. Criminisi, P. Perez, K. Toyama. Region filling and object removal by exemplar-based image inpainting. IEEE Transactions on Image Processing. Vol. 13. No. 9. 2004. P. 1200-1212.
https://doi.org/10.1109/TIP.2004.833105
38 Zihan Liu. Literature Review on Image Restoration. Journal of Physics: Conference Series. Vol. 2386. P. 012041.
https://doi.org/10.1088/1742-6596/2386/1/012041
39 Jianmin Bao, Dong Chen, Fang Wen, Houqiang Li, Gang Hua. CVAE-GAN: Fine-Grained Image Generation through Asymmetric Training. 2017.
40 Diederik P. Kingma, Max Welling. An Introduction to Variational Autoencoders. 2019.
41 S. Geman, D. Geman. Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images. IEEE Transactions on Pattern Analysis and Machine Intelligence. Vol. PAMI-6, No. 6. 1984. P. 721-741.
https://doi.org/10.1109/TPAMI.1984.4767596
42 Georgy L. Gimel’farb. Image Textures and Gibbs Random Fields. Springer Dordrecht. 1999.
https://doi.org/10.1007/978-94-011-4461-2
Received 03.10.2023