Journal of Textile Research ›› 2024, Vol. 45 ›› Issue (02): 101-111.doi: 10.13475/j.fzxb.20231005101

• Textile Engineering • Previous Articles     Next Articles

Digitized restoration of textile pattern through edge-guided image inpainting method

ZHANG Jing, XIN Binjie(), YUAN Zhijie, XU Yingqi   

  1. School of Textiles and Fashion, Shanghai University of Engineering Science, Shanghai 201620, China
  • Received:2023-10-15 Revised:2023-11-24 Online:2024-02-15 Published:2024-03-29

RichHTML

9

PDF (PC)

44

Abstract:

Objective This study aims to further improve the Criminisi algorithm to effectively restore traditional Chinese textile patterns in the presence of damage. Given the complex nature of these patterns, structural restoration is essential to ensure their accurate recovery. Efforts will be directed towards improving the inpainting algorithm's performance and expanding its applicability in practical textile restoration work, with the aim of achieving faster and more accurate repair results. Our research aims to provide more feasible solutions for the preservation of cultural heritage and textile restoration, ensuring the enduring legacy of traditional Chinese textile patterns.

Method Firstly, it uses linear or second-order BÉzier curves to fit the missing edges and restore the structure. Then, it calculates more effective priority using structural information from a multi-resolution image to determine the current patch to be repaired. Next, it computes multiple candidates matching patches in the multi-resolution image based on color, gradient, and boundary features, selecting the best-matched patch to reduce randomness in the selection process. Finally, the replicated best-matched patch is segmented before being used for filling the damaged area, reducing the overlap with known information regions and achieving iterative completion of the restoration for all damaged areas.

Results Real traditional textile images were collected, and artificial damage was introduced by adding masks to simulate challenges encountered when dealing with damaged textiles. The effectiveness of the proposed algorithm was evaluated using a variety of objective metrics, including the peak signal-to-noise ratio, structural similarity feature similarity index measure, feature similarity index measure, and edge preservation rate. These metrics provided a comprehensive and quantifiable assessment of the restoration results. Apart from the quantitative assessments, a subjective evaluation of the inpainting results was also carried out. This qualitative assessment revealed that the proposed algorithm excelled in fitting the main structures in the damaged areas, ultimately resulting in a more visually pleasing restoration effect. Experimental results demonstrate that our method achieves higher objective evaluation scores and natural restoration effects for real textile color images with significant structural damages. It is worth noting that, despite the superior inpainting quality achieved by the proposed algorithm, there was a trade-off in terms of the time required for restoration. Nevertheless, this minor sacrifice in time was considered acceptable, given the significant enhancement in inpainting quality.

Conclusion Digital restoration of textile artifacts is crucial for preserving our cultural heritage. It protects these artifacts from decay and loss, allowing researchers and educators to explore their historical and artistic value. Digital restoration enables easy sharing and accessibility of artifacts, both online and physically, reaching a broader audience. Its speed compared to manual restoration expedites the process, swiftly presenting artifacts in their original aesthetic state for appreciation and study. Moreover, it helps preserve historical information within artifacts, contributing to the revitalization of their aesthetic value and fostering a deeper recognition of their artistic significance.

Key words: textile, cultural relics restoration, image inpainting, multi-resolution image, structural information

CLC Number: 

  • TS941.26

Fig. 1

Data acquisition environment"

Fig. 2

Image preprocessing. (a) Original image; (b) Edge detection of Fig.(a); (c) Apply RTV model to filter texture of Fig.(a); (d) Edge detection of Fig.(c)"

Fig. 3

Inpainting flowchart"

Fig. 4

Criminisi algorithm. (a) Patch ψp to be repaired and corresponding most similar patch ψq; (b) Fill patch ψp with most similar patch ψq for repair"

Fig. 5

Second-order Bézier curve fitting for missing edge"

Fig. 6

Reconstruction effect of missing edges. (a) Damaged image; (b) Breakpoint recognition; (c) Draw missing edge with Bezier curve"

Fig. 7

Comparison of edge extraction from images of different resolutions. (a) High resolution fabric image; (b) Low resolution image of Fig.(a); (c) Edges extracted according to Fig.(a); (d) Edges extracted according to Fig.(b)"

Fig. 8

Boundary term calculation. (a) Patch ψp to be repaired, centered on pixel point p inside edge δΩ of damaged area Ω, and similar patch ψq in complete area Φ, centered on point q; (b) Complete area ψ ? p ? and damaged area ψ p ? in ψp correspond to areas ψ ? q ? and ψ q ? with the same shape in ψq; (c) Pixel Boutp on outside of δΩ in ψp and pixel Binq on inside of corresponding position of δΩ in ψq"

Fig. 9

Segmentation of best-matched patch"

Fig. 10

Repair results of man-made damaged textile patterns I by different algorithms. (a) Original textile image; (b) Mask image; (c) Criminisi algorithm; (d) Algorithm of Ref.[12]; (e) Algorithm of Ref.[23]; (f) Algorithm of Ref.[22]; (g) Image reconstruction with missing edges; (h) Method of this paper"

Fig. 11

Repair results of man-made damaged textile patterns II by different algorithms. (a) Original textile image; (b) Mask image; (c) Criminisi algorithm; (d) Algorithm of Ref.[12]; (e) Algorithm of Ref.[23]; (f) Algorithm of Ref.[22]; (g) Image reconstruction with missing edges; (h) Method of this paper"

Tab. 1

Objective evaluation scores of different algorithms"

样本
编号		 方法 PSNR SSIM FSIM EPRa 样本
编号		 方法 PSNR SSIM FSIM EPRa
1 文献[7] 30.672 3 0.981 9 0.978 7 0.966 0 6 文献[7] 29.805 3 0.968 0 0.971 9 0.970 3
文献[12] 27.318 2 0.976 7 0.969 7 0.959 9 文献[12] 29.149 5 0.966 2 0.963 3 0.954 6
文献[23] 31.609 3 0.983 2 0.980 4 0.964 7 文献[23] 29.325 7 0.967 7 0.970 4 0.970 5
文献[22] 31.881 0 0.985 0 0.980 0 0.963 4 文献[22] 31.329 5 0.971 8 0.975 7 0.964 1
文献[15] 29.319 5 0.975 3 0.967 6 0.939 5 文献[15] 27.814 1 0.943 3 0.951 8 0.949 9
本文算法 34.072 6 0.988 5 0.985 3 0.969 2 本文算法 31.730 7 0.973 5 0.977 9 0.973 1
2 文献[7] 30.327 4 0.983 7 0.980 4 0.979 1 7 文献[7] 27.174 6 0.981 8 0.960 1 0.949 5
文献[12] 29.379 2 0.981 3 0.975 4 0.976 9 文献[12] 28.561 0 0.984 3 0.967 5 0.975 8
文献[23] 30.076 9 0.983 2 0.980 6 0.979 4 文献[23] 29.534 3 0.983 7 0.971 0 0.977 3
文献[22] 31.435 0 0.984 9 0.983 7 0.980 0 文献[22] 26.500 0 0.976 8 0.956 1 0.977 2
文献[15] 28.279 5 0.972 9 0.965 4 0.964 5 文献[15] 28.893 8 0.984 1 0.968 9 0.968 9
本文算法 32.673 5 0.986 7 0.984 8 0.979 6 本文算法 29.621 8 0.984 1 0.973 3 0.982 7
3 文献[7] 24.351 2 0.936 7 0.938 4 0.891 8 8 文献[7] 28.142 3 0.984 2 0.981 0 0.980 1
文献[12] 20.045 5 0.914 6 0.917 5 0.457 1 文献[12] 28.008 9 0.982 6 0.981 1 0.980 2
文献[23] 23.969 2 0.936 6 0.936 8 0.828 3 文献[23] 28.245 0 0.984 5 0.982 1 0.979 8
文献[22] 23.712 3 0.932 4 0.933 2 0.804 2 文献[22] 28.678 0 0.985 7 0.981 8 0.980 1
文献[15] 25.127 5 0.956 3 0.939 1 0.881 7 文献[15] 27.148 0 0.978 0 0.974 8 0.973 3
本文算法 26.037 2 0.953 5 0.947 4 0.900 2 本文算法 30.716 0 0.989 6 0.985 2 0.982 0
4 文献[7] 24.233 3 0.956 4 0.937 3 0.873 2 9 文献[7] 30.993 0 0.985 3 0.983 2 0.976 6
文献[12] 24.070 1 0.955 0 0.937 5 0.952 9 文献[12] 30.856 5 0.984 0 0.983 7 0.975 8
文献[23] 26.222 9 0.959 0 0.952 2 0.956 8 文献[23] 31.213 1 0.985 1 0.984 2 0.976 8
文献[22] 26.333 5 0.958 3 0.954 7 0.955 9 文献[22] 32.377 2 0.986 8 0.985 2 0.977 0
文献[15] 25.167 3 0.941 4 0.941 6 0.939 9 文献[15] 29.551 4 0.972 1 0.975 6 0.952 9
本文算法 27.816 4 0.964 0 0.964 7 0.959 1 本文算法 33.375 9 0.987 5 0.988 8 0.979 0
5 文献[7] 31.037 5 0.981 8 0.986 5 0.980 6 10 文献[7] 30.170 8 0.988 3 0.983 6 0.984 5
文献[12] 28.979 7 0.977 0 0.977 1 0.976 7 文献[12] 28.643 4 0.986 3 0.982 4 0.981 3
文献[23] 30.505 7 0.979 4 0.985 5 0.978 5 文献[23] 32.161 0 0.989 9 0.987 3 0.981 5
文献[22] 30.849 5 0.980 7 0.985 2 0.980 2 文献[22] 32.957 3 0.990 8 0.991 1 0.984 7
文献[15] 29.146 6 0.966 7 0.978 4 0.960 8 文献[15] 33.299 3 0.984 0 0.991 2 0.886 4
本文算法 31.294 9 0.983 1 0.986 8 0.981 6 本文算法 37.783 9 0.993 1 0.997 0 0.985 6

Fig. 12

Comparison of repair details of man-made damaged textile patterns 1 of this paper. (a) Original textile image; (b) Mask image; (c) LaMa model; (d) Method of this paper"

Fig. 13

Comparison of repair details of man-made damaged textile patterns 2. (a) Original textile image; (b) Mask image; (c) LaMa model; (d) Method of this paper"

[1] BERTALMIO M, SAPIRO G, CASELLES V, et al. Image inpainting[C]// Proceedings of the 27th Annual Conference On Computer Graphics And Interactive Techniques. New York: ACM, 2000: 417-424.
[2] SHEN J, CHAN T F. Mathematical models for local nontexture inpaintings[J]. SIAM Journal on Applied Mathematics, 2001, 62(3): 1019-1043.
doi: 10.1137/S0036139900368844
[3] RUDIN L I, OSHER S, FATEMI E. Nonlinear total variation based noise removal algorithms[J]. Physica D: Nonlinear Phenomena, 1992, 60(1-4): 259-268.
doi: 10.1016/0167-2789(92)90242-F
[4] RUDIN L I, OSHER S. Total variation based image restoration with free local constraints[C]// Proceedings of 1st International Conference on Image Processing Austin, IEEE, 1994: 31-35.
[5] CHAN T F, SHEN J. Nontexture inpainting by curvature-driven diffusions[J]. Journal of Visual Communication and Image Representation, 2001, 12(4): 436-449.
doi: 10.1006/jvci.2001.0487
[6] CRIMINISI A, PÉREZ P, TOYAMA K. Region filling and object removal by exemplar-based image inpain-ting[J]. IEEE Transactions on Image Processing, 2004, 13(9): 1200-1212.
doi: 10.1109/TIP.2004.833105
[7] 李张翼. 针对古织物图像的改进Criminisi修复算法[J]. 电脑知识与技术, 2016, 12(26):193-195.
LI Zhangyi. Improved criminisi repair algorithm for ancient fabric images[J]. Computer Knowledge and Technology, 2016, 12(26):193-195.
[8] 朱耀麟, 李张翼, 武桐. 针对规则古织物纹理的图像修复[J]. 棉纺织技术, 2017, 45(10):9-12.
ZHU Yaolin, LI Zhangyi, WU Tong. Image restoration for regular ancient fabric texture[J]. Cotton Textile Technology, 2017, 45(10):9-12.
[9] 韦秋菊, 孙晓婉, 徐平华, 等. 机织物局部组织复原与密度自动测定[J]. 现代纺织技术, 2022, 30(6): 102-109.
WEI Qiuju, SUN Xiaowan, XU Pinghua, et al. Local weave restoration and automatic density measurement for fabrics[J]. Advanced Textile Technology, 2022, 30(6): 102-109.
[10] BARNES C, SHECHTMAN E, FINKELSTEIN A, et al. PatchMatch: a randomized correspondence algorithm for structural image editing[C]// ACM SIGGRAPH Conference 2009. New Orleans: ACM. DOI:10.1145/1531326.1531330.
[11] KORMAN S, AVIDAN S. Coherency sensitive hash-ing[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 38(6): 1099-1112.
doi: 10.1109/TPAMI.2015.2477814
[12] LECUN Y, BOSER B, DENKER J S, et al. Backpropagation applied to handwritten zip code recognition[J]. Neural computation, 1989, 1(4): 541-551.
doi: 10.1162/neco.1989.1.4.541
[13] RUMELHART D E, HINTON G E, WILLIAMS R J. Learning internal representations by error propaga-tion[J]. Parallel Distributed Processing, 1986, 1(1): 318-363.
[14] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]// Advances in Neural Information Processing Systems 27 (NIPS 2014). Montreal: Curran Associates, Inc., 2014: 2672-2680.
[15] SUVOROV R, LOGACHEVA E, MASHIKHIN A, et al. Resolution-robust large mask inpainting with fourier convolutions[C]// Proceedings of the IEEE/CVF Winter Conference On Applications Of Computer Vision. Waikoloa, IEEE, 2022: 2149-2159.
[16] 刘羿漩, 葛广英, 齐振岭, 等. 基于改进DCGAN的刺绣图像修复的研究[J]. 激光与光电子学进展, 2023, 60(20):1-20.
LIU Yixuan, GE Guangying, QI Zhenling, et al. Research on embroidery image restoration based on improved DCGAN[J]. Laser & Optoelectronics Progress, 2023, 60(20):1-20.
[17] TUKEY J W. Exploratory data analysis[M]. Reading: Addison-Wesley, 1971: 210-236.
[18] TOMASI C, MANDUCHI R. Bilateral filtering for gray and color images[C]// Sixth International Conference on Computer Vision (IEEE Cat. No. 98CH36271). Bombay: IEEE, 1998: 839-846.
[19] HE K, SUN J, TANG X. Guided image filtering[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 35(6): 1397-1409.
doi: 10.1109/TPAMI.2012.213
[20] XU L, YAN Q, XIA Y, et al. Structure extraction from texture via relative total variation[J]. ACM Transactions on Graphics (TOG), 2012, 31(6): 1-10.
[21] CANNY J. A computational approach to edge detec-tion[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986 (6): 679-698.
[22] LIU H, BI X, LU G, et al. Exemplar-based image inpainting with multi-resolution information and the graph cut technique[J]. IEEE Access, 2019, 7: 101641-101657.
doi: 10.1109/Access.6287639
[23] LE Meur O, EBDELLI M, GUILLEMOT C. Hierarchical super-resolution-based inpainting[J]. IEEE Transactions on Image Processing, 2013, 22(10): 3779-3790.
doi: 10.1109/TIP.2013.2261308 pmid: 23661318
[24] KWATRA V, SCHÖDL A, ESSA I, et al. Graphcut textures: image and video synthesis using graph cuts[J]. ACM Transactions on Graphics (TOG), 2003, 22(3): 277-286.
doi: 10.1145/882262.882264
[25] LEE J, LEE D K, PARK R H. Robust exemplar-based inpainting algorithm using region segmentation[J]. IEEE Transactions on Consumer Electronics, 2012, 58(2): 553-561.
doi: 10.1109/TCE.2012.6227460
[26] CHUNG B, YIM C. Hybrid error concealment method combining exemplar-based image inpainting and spatial interpolation[J]. Signal Processing: Image Communication, 2014, 29(10): 1121-1137.
doi: 10.1016/j.image.2014.09.009
[27] ZHANG L, ZHANG L, MOU X, et al. FSIM: A feature similarity index for image quality assessment[J]. IEEE Transactions on Image Processing, 2011, 20(8): 2378-2386.
doi: 10.1109/TIP.2011.2109730 pmid: 21292594
[28] CHEN L, JIANG F, ZHANG H, et al. Edge preservation ratio for image sharpness assessment[C]// 2016 12th World Congress on Intelligent Control and Automation (WCICA). Guilin:IEEE, 2016: 1377-1381.
[1] YANG Meihui, LI Bo, SHEN Yanqin, WU Hailiang. Sorption properties of regenerated keratin gels to size macromolecules in textile desizing wastewater [J]. Journal of Textile Research, 2024, 45(02): 142-152.
[2] SUN Langtao, YANG Yushan. Preparation of thermoregulation and antibacterial microcapsules and its application in cotton fabrics [J]. Journal of Textile Research, 2024, 45(02): 171-178.
[3] CHENG Ziqi, LU Yehu, XU Jingxian. Heat transfer simulation and parametric design of electric heating textile system [J]. Journal of Textile Research, 2024, 45(02): 206-213.
[4] YAN Pengxiang, CHEN Fuxing, LIU Hong, TIAN Mingwei. Preparation of flexible force-sensing electronic textiles and construction of human motion monitoring system [J]. Journal of Textile Research, 2024, 45(02): 59-66.
[5] FAN Shuo, YANG Peng, ZENG Jinhao, SONG Xiaodi, GONG Yudan, XIAO Yao. Preparation of multi-component organic polysiloxane for flame retardancy of polyamide 6 fabrics with anti-dripping behavior [J]. Journal of Textile Research, 2024, 45(01): 152-160.
[6] LU Weijian, TU Jiajia, WANG Junru, HAN Sijie, SHI Weimin. Model for empty bobbin recognition based on improved residual network [J]. Journal of Textile Research, 2024, 45(01): 194-202.
[7] YANG Zhichao, LIU Shuqiang, WU Gaihong, JIA Lu, ZHANG Man, LI Fu, LI Huimin. Research progress in absorbable surgical sutures [J]. Journal of Textile Research, 2024, 45(01): 230-239.
[8] DONG Kai, LÜ Tianmei, SHENG Feifan, PENG Xiao. Advances in smart textiles oriented to personalized healthcare [J]. Journal of Textile Research, 2024, 45(01): 240-249.
[9] CHEN Shun, QIAN Kun, LIANG Fuwei, GUO Wenwen. Preparation and properties of flame retardant hydrophobic cotton fabric with eugenol-based composite coating [J]. Journal of Textile Research, 2023, 44(12): 115-122.
[10] LI Xun, LI Zhewen, ZHANG Tingwen, JING Junfeng, LI Pengfei. Mobile robot positioning method in textile production environments [J]. Journal of Textile Research, 2023, 44(12): 170-180.
[11] ZHANG Yongfang, FEI Pengfei, YAN Zhifeng, WANG Shuhua, GUO Hong. Research progress of hydrothermal degradation of waste cellulose textiles [J]. Journal of Textile Research, 2023, 44(12): 216-224.
[12] ZHANG Jun, HU Song, TONG Mengxia, XIAO Wenling. Research progress in Kansei engineering for textile and clothing applications [J]. Journal of Textile Research, 2023, 44(11): 240-249.
[13] ZHANG Jianlei, SHEN Pandeng, HE Lin, CHENG Longdi. Impact of heterogeneous environmental regulations on carbon emissions with China's textile and garment industry [J]. Journal of Textile Research, 2023, 44(10): 149-156.
[14] WANG Yutao, CONG Honglian, GU Hongyang. Structural design and thermal-moist comfort of weft knitted knee pads [J]. Journal of Textile Research, 2023, 44(10): 68-74.
[15] XU Ruidong, WANG Hang, QU Lijun, TIAN Mingwei. Preparation and properties of polyactic acid nonwoven substrate touch-sensing electronic textile [J]. Journal of Textile Research, 2023, 44(09): 161-167.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
京ICP备14006648号
Copyright 2021 © Editorial office of Journal of Textile Research
Supported by Beijing Magtech Co.Ltd