Journal of Textile Research ›› 2023, Vol. 44 ›› Issue (01): 219-227.doi: 10.13475/j.fzxb.20211105509

• Comprehensive Review • Previous Articles     Next Articles

Research progress in fabric defect detection based on deep learning

WANG Bin1,2, LI Min2,3(), LEI Chenglin1,2, HE Ruhan2,3   

  1. 1. Engineering Research Center of Hubei Province for Clothing Information, Wuhan Textile University, Wuhan, Hubei 430200, China
    2. Hubei Provincial Engineering Research Center for Intelligent Textile and Fashion, Wuhan Textile University, Wuhan, Hubei 430200, China
    3. School of Computer Science and Artificial Intelligence, Wuhan Textile University, Wuhan, Hubei 430200, China
  • Received:2021-11-10 Revised:2022-10-04 Online:2023-01-15 Published:2023-02-16

RichHTML

57

PDF (PC)

423

Abstract:

Significance With the development of science and technology, the improvement of product quality is highly demanded. Although the technologies used in producing textile products have undergone revolutionary advancement which contributes to the textile quality dramatically, defects in textile products such as fabrics remain to be a reality. Fabric defect detection plays an important role in textile industry, and fabric defect detection technology based on deep learning has been paid more and more attention. This paper reports a research and development progress in fabric defect detection based on deep learning.
Progress Deep learning is mainly composed of four steps, i.e., defining model and loss function, training the model, finding optimization method and loop iteration. The research focus for fabric defect detection method based on deep learning mainly includes deep learning models such as convolution neural network (CNN) and automatic encoder (AE). The stack denoising automatic encoder based on Fisher criterion introduces deep learning into this field for the first time, which provides a new idea for the application of deep learning to the field of defect detection. Convolution neural network has achieved good results in the field of image recognition because of its strong nonlinear fitting ability. More precision-based detection algorithms based on candidate regions and more speed-based algorithms based on regression analysis are present. While the advantages of convolution neural network is exploited, other methods are used for exploring the possibility of combined use of models, and provide new ways for defect detection.
Conclusion and Prospect Fabric defect detection methods based on deep learning in recent years are reviewed and summarized, and the effects of different models are compared in detail. Advantages, disadvantages and applicable scope of each model are analyzed, and future development of fabric defect detection method based on deep learning model is prospected. Deep learning models can improve the detection efficiency, but still have some deficiencies. In order to optimize the accuracy of fabric image defect detection, breakthrough should be made from the following aspects in the future. 1) High quality data sets should be established. 2) Specific evaluation criteria need to be established. 3) The applicability should be extended. A single detection method often has limitations, but when different defect detection methods are utilized to deal with different detection needs, the detection results are often different, therefore hybrid methods would have better applicability.

Key words: deep learning, defect detection, fabric, neural network, image segmentation, machine vision

CLC Number: 

  • TP181

Fig.1

Basic steps for deep learning"

Tab.1

Defect detection method based on convolutional neural network model"

序号 文献 关键技术 疵点召回 疵点分类 疵点位置回归
1 [18] VGG16 √ √ √
2 [25] CNN √ — —
3 [26] FPN √ √ √
4 [35] CNN √ — √
5 [36] PE TMCNN √ — √
6 [37] CNN √ √ √
7 [15] Faster RCNN, ResNet √ — √
8 [27] Faster RCNN √ √ √
9 [28] GAN √ √ √
10 [29] Faster RCNN, GAN √ — √
11 [30] DCGAN √ — —
12 [14] YOLOv3 √ √ √
13 [31] Faster RCNN √ — √
14 [32] RefinDet √ √ √
15 [33] DUW-CAM, L-SE √ — —
16 [34] CNN √ — √

Tab.2

Performance of fabric defect detection methods based on convolutional neural network"

序号 文献 数据集 精度 检测时间/(张·s-1) 疵点类型 硬件资源
1 [18] 自建数据集 亚麻布mAP:93.4%;花纹织物mAP:95.1% 亚麻布:21.5;花纹织物:21.1 破洞、污渍等5种 GTX 1060 GPU
2 [25] TILDA数据集 Acc:96.55% 19.6 油污、跳纱等6种 GTX 1280 GPU
3 [26] FBDF数据集 mAP:72.6% — 断裂、错纬等20种 TITAN RTX GPU
4 [35] TILDA数据集和自建数据集 TILDA数据集Acc:90.25%
自建数据集Acc:94.6%		 TILDA数据集:18.3
自建数据集:20.6		 纬缩、拆痕等7种 NVIDIA GTX 1080TI GPU
5 [36] Kylberg数据集和自建数据集 Kylberg数据集Acc:80.32%
自建数据集Acc:83.81%		 — 浆斑、油纱等14种 GTX 980Ti Nvidia GPU
6 [37] 自建数据集,Knitting数据集和Nano fiber数据集 自建数据集Acc:93.81%
Knitting数据集Acc:95.61%
Nano fiber数据集Acc:81.82%		 自建数据集:23.9
Knitting数据集:23.9
Nano fiber数据集:23.0		 脱纬、跳花等18种 GeForce RTX 2080 Ti
7 [15] 现场采集坯布 坯布mAP:94.71% 7.7 褶皱、线条等4种 NVIDIA GTX1080Ti
8 [27] 天池布匹数据集 Acc:94.67%, Recall:98.63% 4.3 污渍、断纬和破洞 Intel Core i5-82500U
9 [28] 自建数据集 mAP:94.8% — 棉结、污渍等8种 —
10 [29] 机器采集花纹织物 mAP:93.62% 5.4 带纱、破洞等6种 NVIDIA GeForce GTX 745
11 [30] TILDA数据集和自建数据集 TILDA数据集mAP:85.14%;自建数据集mAP:93.45% TILDA数据集:37;自建数据集:33 错纬、污渍等14种 NVIDIA Tesla P4
12 [14] 机器采集坯布和格纹布 坯布错误率:1.3%;格纹布错误率:1.02% 坯布:27.7;格纹布:21.8 破洞、油污等6种 GTX1080 Ti
13 [31] 机器采集斜纹和牛仔布 斜纹mAP:62.3%;
牛仔布mAP:92.5%		 斜纹:9.7;牛仔布:25.4 错纬、折痕等12种 RTX2080Ti GPU
14 [32] TILDA数据集,香港图案纹理数据集和DAGM200数据集 TILDA数据集mAP:80.2%;
香港图案数据集mAP:85.9%;
DAGM2007数据集mAP:96.7%		 TILDA数据集:34.0;香港图案数据集:21.9;DAGM2007数据集:45.2 破洞、棉球等13种 NVIDA TITAN XP GPU
15 [33] DAGM2007数据集和机器采集 DAGM2007数据集mAP:90.29%;
机器采集数据集mAP:93.27%		 — 污渍、破洞等8种 Nvidia Quadro M5000 GPU
16 [34] 天池布匹数据集 mAP:90.4% 10.3 污渍、断纬和破洞 NVIDIA TITAN XP GPU

Tab.3

Performance of fabric defect detection methods based on automatic encoder"

序号 文献 数据集 精度 检测时间/(张·s-1) 疵点类型 硬件资源
1 [38] 自建数据集 mAP:98.21% 4.76 浆斑、破洞等4种 Intel i5 processor
2 [39] TILDA数据集 mAP:83.9% — 穿错、毛边等12种 GTX 980Ti Nvidia GPU
3 [40] TILDA数据集和机器采集数据集 TILDA数据集mAP:87.2%;
机器采集数据集mAP:79.2%		 — 断经、经缩等11种 Intel®Core i5-4460
4 [41] VTC-2K10.5G-C19数据集 Recall:69.7%
Precision:82.2%		 — 跳花、百脚等15种 —

Tab.4

Performance of fabric defect detection methods based on other deep learning models"

序号 文献 数据集 精度/% 检则时间/(张·s-1) 疵点类型 硬件资源
1 [42] TILDA数据集 — — 油污、跳纱等6种 —
2 [35] 自建数据集 mAP:94.57% 16.2 破洞、污渍等8种 Titan RTX GPU (24G)
3 [35] 自建数据集 Acc:0.95%
AP:95.8%		 — 毛边、断经等6种 NVIDIA GTX 1080TI
[1] ZHU Q P, WU M Y, LI J, et al. Fabric defect detection via small scale over-complete basis set[J]. Textile Research Journal, 2014, 84(15):1634-1649.
doi: 10.1177/0040517514525880
[2] 纺织行业“十四五”发展纲要[J]. 纺织科学研究, 2021(7): 40-49.
Development outline of textile industry during the 14th five-year plan[J]. Textile Science Research, 2021(7): 40-49.
[3] 吕文涛, 林琪琪, 钟佳莹, 等. 面向织物疵点检测的图像处理技术研究进展[J]. 纺织学报, 2021, 42(11): 197-206.
LÜ Wentao, LIN Qiqi, ZHONG Jiaying, et al. Research progress of image processing technology for fabric defect detection[J]. Journal of Textile Research, 2021, 42(11): 197-206.
[4] 郑胤, 陈权崎, 章毓晋. 深度学习及其在目标和行为识别中的新进展[J]. 中国图像图形学报, 2014, 19(2): 175-184.
ZHENG Yin, CHEN Quanqi, ZHANG Yujin. Deep Learning and its new progress in goal and behavior recognition[J]. Journal of Image and Graphics, 2014, 19(2): 175-184.
[5] 景军锋, 李江南, 李鹏飞. 改进型奇异值分解在织物疵点检测上的应用[J]. 纺织学报, 2014, 35(6): 62-67.
JING Junfeng, LI Jiangnan, LI Pengfei. Application of improved singular value decomposition in fabric defect detection[J]. Journal of Textile Research, 2014, 35(6): 62-67.
[6] 刘洲峰, 赵全军, 李春雷, 等. 基于局部统计与整体显著性的织物疵点检测算法[J]. 纺织学报, 2014, 35(11): 62-67.
LIU Zhoufeng, ZHAO Quanjun, LI Chunlei, et al. Fabric defect detection algorithm based on local statistics and global significance[J]. Journal of Textile Research, 2014, 35(11): 62-67.
[7] 张缓缓, 李仁忠, 景军锋, 等. Frangi滤波器和模糊C均值算法相结合的织物瑕疵检测[J]. 纺织学报, 2015, 36(9): 120-124.
ZHANG Huanhuan, LI Renzhong, JING Junfeng, et al. Fabric defect detection based on Frangi filter and Fuzzy C-means algorithm[J]. Journal of Textile Research, 2015, 36(9): 120-124.
[8] 景军锋, 赵娟. 基于MeanShift滤波的织物疵点检测方法[J]. 电子测量与仪器学报, 2016, 30(5): 739-747.
JING Junfeng, ZHAO Juan. Fabric defect detection method based on MeanShift filter[J]. Journal of Electronic Measurement and Instruments, 2016, 30(5): 739-747.
[9] DI L, LONG H, LIANG J. Fabric defect detection based on illumination correction and visual salient features[J]. Sensors (Basel), 2020. DOI: 10.3390/s20185147.
doi: 10.3390/s20185147
[10] PRACH Y, BUMRUNG K. Defect detection in textile fabrics with snake active contour and support vector machines[J]. Journal of Physics: Conference Series, 2019. DOI: 10.1088/1742-6596/1195/1/012006.
doi: 10.1088/1742-6596/1195/1/012006
[11] LI Y Y, LUO H C, YU M M, et al. Fabric defect detection algorithm using RDPSO-based optimal Gabor filter[J]. Journal of The Textile Institute, 2019, 110(4):487-495.
doi: 10.1080/00405000.2018.1489951
[12] 杨晓波. 基于GMRF模型的统计特征畸变织物疵点识别[J]. 纺织学报, 2013, 34(4): 137-142.
YANG Xiaobo. Fabric defect detection of statistic aberration feature based on GMRF model[J]. Journal of Textile Research, 2013, 34(4): 137-142.
[13] 杨晓波. 基于Gabor滤波器的织物疵点检测[J]. 纺织学报, 2010, 31(4): 55-59.
YANG Xiaobo. Fabric defect detection based on Gabor filter[J]. Journal of Textile Research, 2010, 31(4): 55-59.
[14] JING J F, ZHUO D, ZHANG H H, et al. Fabric defect detection using the improved YOLOv3 model[J]. Journal of Engineered Fibers and Fabrics, 2020, 2(4):1-10.
[15] 晏琳, 景军锋, 李鹏飞. Faster RCNN模型在坯布疵点检测中的应用[J]. 棉纺织技术, 2019, 47(2): 24-27.
YAN Lin, JING Junfeng, LI Pengfei. Application of faster RCNN model in defect detection of gray fabric[J]. Cotton Textile Technology, 2019, 47(2): 24-27.
[16] 杜磊, 李立轻, 汪军, 等. 几种基于图像自适应阈值分割的织物疵点检测方法比较[J]. 纺织学报, 2014, 35(6): 56-61.
DU Lei, LI Liqing, WANG Jun, et al. Comparison of several fabric defect detection methods based on image adaptive threshold segmentation[J]. Journal of Textile Research, 2014, 35(6): 56-61.
[17] 管声启, 石秀华. 基于频域滤波的织物疵点检测[J]. 计算机应用, 2008(10): 2673-2675.
GUAN Shengqi, SHI Xiuhua. Fabric defect detection based on frequency domain filtering[J]. Journal of Computer Application, 2008(10): 2673-2675.
[18] ZHAO S X, YIN L, ZANG J, et al. Real-time fabric defect detection based on multi-scale convolutional neural network[J]. IET Collaborative Intelligent Manufacturing, 2020, 2(4):189-196.
doi: 10.1049/iet-cim.2020.0062
[19] 汪坤. 基于深度学习的织物疵点检测算法研究[D]. 杭州: 浙江理工大学, 2020:22-23.
WANG Kun. Research on fabric defect detection algorithm based on deep learning[D]. Hangzhou: Zhejiang Sci-Tech University, 2020:22-23.
[20] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.
doi: 10.1145/3065386
[21] ZEILER M D, KRISHNAN D, TAYLOR G W, et al. Deconvolutional networks[C] // 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Illinois :IEEE, 2010:2528-2535.
[22] 邓俊锋, 张晓龙. 基于自动编码器组合的深度学习优化方法[J]. 计算机应用, 2016, 36(3): 697-702.
doi: 10.11772/j.issn.1001-9081.2016.03.697
DENG Junfeng, ZHANG Xiaolong. Deep learning optimization method based on automatic encoder combination[J]. Journal of Computer Application, 2016, 36(3): 697-702.
[23] GEOFFREY H, RUSLAN S. Discovering binary codes for documents by learning deep generative models[J]. Topics in Cognitive Science, 2011, 3(1):74-91.
doi: 10.1111/j.1756-8765.2010.01109.x pmid: 25164175
[24] GEOFFREY E, SIMON O, YEE T. A fast learning algorithm for deep belief nets[J]. Neural Computation, 2006, 18(7):1527-1554.
doi: 10.1162/neco.2006.18.7.1527 pmid: 16764513
[25] PANDIA R J, EDWARD R S J. Computer vision for automatic detection and classification of fabric defect employing deep learning algorithm[J]. International Journal of Clothing Science and Technology, 2019, 31(4):510-521.
doi: 10.1108/IJCST-11-2018-0135
[26] WU Y, ZHANG X D, FANG F Z. Automatic fabric defect detection using cascaded mixed feature pyramid with guided localization[J]. Sensors, 2020, 20(3):871.
doi: 10.3390/s20030871
[27] 安萌, 郑飂默, 王诗宇, 等. 一种改进Faster R-CNN的面料疵点检测方法[J]. 小型微型计算机系统, 2021, 42(5): 1029-1033.
AN Meng, ZHENG Liaomo, WANG Shiyu, et al. A fabric defect detection method based on improved Faster R-CNN[J]. Journal of Chinese Computer Systems, 2021, 42(5): 1029-1033.
[28] LIU J H, WANG C Y, SU H, et al. Multistage GAN for fabric defect detection[J]. IEEE Transactions on Image Processing, 2019, 29(1):3388-3400.
doi: 10.1109/TIP.2019.2959741
[29] 李明, 景军锋, 李鹏飞. 应用GAN和Faster R-CNN的色织物缺陷识别[J]. 西安工程大学学报, 2018, 32(6): 663-669.
LI Ming, JING Junfeng, LI Pengfei. Defect recognition of dyed fabric using GAN and Faster R-CNN[J]. Journal of Xi'an Polytechnic University, 2018, 32(6): 663-669.
[30] HU G H, HUANG J F, WANG Q H, et al. Unsupervised fabric defect detection based on a deep convolutional generative adversarial network[J]. Textile Research Journal, 2020, 90(3/4):247-270.
doi: 10.1177/0040517519862880
[31] PENG P R, WANG Y, HAO C, et al. Automatic fabric defect detection method using PRAN-Net[J]. Applied Sciences, 2020, 10(23):8434-8437.
doi: 10.3390/app10238434
[32] XIE H, WU Z. A robust fabric defect detection method based on improved RefineDet[J]. Sensors (Basel), 2020, 20(15):4260-4284.
doi: 10.3390/s20154260
[33] LIU Z F, HUO Z C, LI C L, et al. DLSE-net: a robust weakly supervised network for fabric defect detection[J]. Displays, 2021, 68(12):102-110.
[34] JUN X, WANG J G, ZHOU J, et al. Fabric defect detection based on a deep convolutional neural network using a two-stage strategy[J]. Textile Research Journal, 2021, 91(1/2):130-142.
doi: 10.1177/0040517520935984
[35] LI C X, LIU T, YE W. A novel feature-based network with sequential information for textile defect detection[J]. Coloration Technology, 2020, 136(6):476-484.
doi: 10.1111/cote.12493
[36] WEN Z J, ZHAO Q K, TONG L N. CNN-based minor fabric defects detection[J]. International Journal of Clothing Science and Technology, 2020, 33(1):1-12.
doi: 10.1108/IJCST-11-2019-0177
[37] HO C C, CHOU W C, SU E G. Deep convolutional neural network optimization for defect detection in fabric inspection[J]. Sensors, 2021, 21(21):7074.
doi: 10.3390/s21217074
[38] LI Y D, ZHAO W G, PAN J H. Deformable patterned fabric defect detection with fisher criterion-based deep learning[J]. IEEE Transactions on Automation Science and Engineering, 2017, 14(2):1256-1264.
doi: 10.1109/TASE.2016.2520955
[39] MEI S, WANG Y, WEN G. Automatic fabric defect detection with a multi-scale convolutional denoising autoencoder network model[J]. Sensors (Basel), 2018. DOI:10.3390/s18041064.
doi: 10.3390/s18041064
[40] 景军锋, 党永强, 苏泽斌, 等. 基于改进SAE网络的织物疵点检测算法[J]. 电子测量与仪器学报, 2017, 31(8): 1321-1329.
JING Junfeng, DANG Yongqiang, SU Zebin, et al. Fabric defect detection algorithm based on improved SAE network[J]. Journal of Electronic Measurement and Instrument, 2017, 31(8): 1321-1329.
[41] HAN Y J, YU H J. Fabric defect detection system using stacked convolutional denoising auto-encoders trained with synthetic defect data[J]. Applied Sciences, 2020, 10(7):26-36.
doi: 10.3390/app10010026
[42] 刘洲峰, 闫磊, 李春雷, 等. 基于稀疏优化的织物疵点检测算法[J]. 纺织学报, 2016, 37(5):56-61.
LIU Zhoufeng, YAN Lei, LI Chunlei, et al. Fabric defect detection algorithm based on sparse optimization[J]. Journal of Textlie Research, 2016, 37(5):56-61.
[43] CHEN M Q, YU L J, ZHI C, et al. Improved faster R-CNN for fabric defect detection based on Gabor filter with Genetic algorithm optimization[J]. Computers in Industry, 2022.DOI: j.compind.2021.103551.
doi: j.compind.2021.103551
[1] YING Zhiping, WANG Weiqing, WU Zhenyu, HU Xudong. Compression after impact performance of three-dimensional orthogonal woven composites [J]. Journal of Textile Research, 2023, 44(01): 129-135.
[2] ZHANG Diandian, LI Min, GUAN Yu, WANG Sixiang, HU Huanchuan, FU Shaohai. Preparation and performance of disperse dye printed fabrics with characteristics of vegetation-like Vis-NIR reflectance spectrum [J]. Journal of Textile Research, 2023, 44(01): 142-148.
[3] WAN Ailan, SHEN Xinyan, WANG Xiaoxiao, ZHAO Shuqiang. Preparation and sensing response characterization of polydopamine modified reduced graphene oxide/polypyrrole conductive fabrics [J]. Journal of Textile Research, 2023, 44(01): 156-163.
[4] CHEN Jia, YANG Congcong, LIU Junping, HE Ruhan, LIANG Jinxing. Cross-domain generation for transferring hand-drawn sketches to garment images [J]. Journal of Textile Research, 2023, 44(01): 171-178.
[5] CHEN Jiahui, MEI Tao, ZHAO Qinghua, YOU Haining, WANG Wenwen, WANG Dong. Research progress in smart fabrics for thermal and humidity management [J]. Journal of Textile Research, 2023, 44(01): 30-37.
[6] DAI Lu, HU Zexu, WANG Yan, ZHOU Zhe, ZHANG Fan, ZHU Meifang. Combustion and charring behavior of polyphenylene sulfide/graphene nanocomposite fibers [J]. Journal of Textile Research, 2023, 44(01): 71-78.
[7] ZHANG Chudan, WANG Rui, WANG Wenqing, LIU Yanyan, CHEN Rui. Synthesis and properties of cationic modified flame retardant polyester fabrics [J]. Journal of Textile Research, 2022, 43(12): 109-117.
[8] MEI Min, QIAN Jianhua, ZHOU Yukai, YANG Jingjing. Preparation and application of nano-SiO2/fluorine-containing silicon waterproof and moisture-permeable finishing agent [J]. Journal of Textile Research, 2022, 43(12): 118-124.
[9] AN Yijin, XUE Wenliang, DING Yi, ZHANG Shunlian. Evaluation of textile color rubbing fastness based on image processing [J]. Journal of Textile Research, 2022, 43(12): 131-137.
[10] SONG Jiexin, FU Tianyu, LI Fengming, SONG Rui, LI Yibin. Prediction method for tension of fabric sewn by robot based on extensibility [J]. Journal of Textile Research, 2022, 43(12): 173-180.
[11] CHEN Yushan, JIANG Gaoming, LI Bingxian. Design and 3-D simulation of weft knitted wrap fabric [J]. Journal of Textile Research, 2022, 43(12): 62-68.
[12] FANG Yinchun, CHEN Lüxin, LI Junwei. Preparation and properties of flame retardant and superhydrophobic polyester/cotton fabrics [J]. Journal of Textile Research, 2022, 43(11): 113-118.
[13] YANG Hongying, QI Mengyuan, YANG Zhihui, YANG Yanli, ZHANG Jingjing, XIE Wanzi. Whiteness formulae evaluation of non-fluorescent fabrics by psychophysical experiments [J]. Journal of Textile Research, 2022, 43(11): 119-126.
[14] QIAO Xiran, FANG Kuanjun, LIU Xiuming, GONG Jixian, ZHANG Shuai, ZHANG Min. Different influence of hydroxyethyl methyl cellulose pretreatment on surface properties of cotton and polyamide [J]. Journal of Textile Research, 2022, 43(11): 127-132.
[15] ZHANG Fumu, LIU Duanwu, HU Yueming. Construction and experiment of intelligent chemicals distribution system for dyeing machine [J]. Journal of Textile Research, 2022, 43(11): 179-187.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
京ICP备14006648号
Copyright 2021 © Editorial office of Journal of Textile Research
Supported by Beijing Magtech Co.Ltd