纺织学报 ›› 2023, Vol. 44 ›› Issue (01): 219-227.doi: 10.13475/j.fzxb.20211105509

• 综合述评 • 上一篇    下一篇

基于深度学习的织物疵点检测研究进展

王斌1,2, 李敏2,3(), 雷承霖1,2, 何儒汉2,3   

  1. 1.武汉纺织大学 湖北省服装信息化工程技术研究中心, 湖北 武汉 430200
    2.武汉纺织大学 计算机与人工智能学院,湖北 武汉 430200
    3.武汉纺织大学 纺织服装智能化湖北省工程研究中心, 湖北 武汉 430200
  • 收稿日期:2021-11-10 修回日期:2022-10-04 出版日期:2023-01-15 发布日期:2023-02-16
  • 通讯作者: 李敏(1978—),女,副教授,博士。主要研究方向为图像处理与模式识别。E-mail:2008031@wtu.edu.cn
  • 作者简介:王斌(1998—),男,硕士生。主要研究方向为计算机视觉。
  • 基金资助:
    中国高校产学研创新基金项目(2020HYA02015)

Research progress in fabric defect detection based on deep learning

WANG Bin1,2, LI Min2,3(), LEI Chenglin1,2, HE Ruhan2,3   

  1. 1. Engineering Research Center of Hubei Province for Clothing Information, Wuhan Textile University, Wuhan, Hubei 430200, China
    2. Hubei Provincial Engineering Research Center for Intelligent Textile and Fashion, Wuhan Textile University, Wuhan, Hubei 430200, China
    3. School of Computer Science and Artificial Intelligence, Wuhan Textile University, Wuhan, Hubei 430200, China
  • Received:2021-11-10 Revised:2022-10-04 Published:2023-01-15 Online:2023-02-16

摘要:

为提高疵点检测的准确性和通用性,实现使用简洁而有效的形式对织物图像的特点和疵点的本质特征进行综合表达,首先,介绍了深度学习技术,对引入了深度学习的疵点检测方法进行综述,同时对深度学习与疵点检测的内在关系进行阐述;然后,分析总结了深度学习的概念及代表性的计算模型,并对引入深度学习的疵点检测方法进行归纳、总结和分类;最后,对典型的方法进行了分析,讨论了各种方法的优缺点,并对未来的研究趋势进行了展望。指出:随着深度学习的发展,探索更加通用的检测方法是推进深度学习在织物疵点检测领域应用的努力方向。

关键词: 深度学习, 疵点检测, 纺织品, 神经网络, 图像分割, 机器视觉

Abstract:

Significance With the development of science and technology, the improvement of product quality is highly demanded. Although the technologies used in producing textile products have undergone revolutionary advancement which contributes to the textile quality dramatically, defects in textile products such as fabrics remain to be a reality. Fabric defect detection plays an important role in textile industry, and fabric defect detection technology based on deep learning has been paid more and more attention. This paper reports a research and development progress in fabric defect detection based on deep learning.
Progress Deep learning is mainly composed of four steps, i.e., defining model and loss function, training the model, finding optimization method and loop iteration. The research focus for fabric defect detection method based on deep learning mainly includes deep learning models such as convolution neural network (CNN) and automatic encoder (AE). The stack denoising automatic encoder based on Fisher criterion introduces deep learning into this field for the first time, which provides a new idea for the application of deep learning to the field of defect detection. Convolution neural network has achieved good results in the field of image recognition because of its strong nonlinear fitting ability. More precision-based detection algorithms based on candidate regions and more speed-based algorithms based on regression analysis are present. While the advantages of convolution neural network is exploited, other methods are used for exploring the possibility of combined use of models, and provide new ways for defect detection.
Conclusion and Prospect Fabric defect detection methods based on deep learning in recent years are reviewed and summarized, and the effects of different models are compared in detail. Advantages, disadvantages and applicable scope of each model are analyzed, and future development of fabric defect detection method based on deep learning model is prospected. Deep learning models can improve the detection efficiency, but still have some deficiencies. In order to optimize the accuracy of fabric image defect detection, breakthrough should be made from the following aspects in the future. 1) High quality data sets should be established. 2) Specific evaluation criteria need to be established. 3) The applicability should be extended. A single detection method often has limitations, but when different defect detection methods are utilized to deal with different detection needs, the detection results are often different, therefore hybrid methods would have better applicability.

Key words: deep learning, defect detection, fabric, neural network, image segmentation, machine vision

中图分类号: 

  • TP181

图1

深度学习基本步骤"

表1

基于卷积神经网络模型的疵点检测方法"

序号 文献 关键技术 疵点召回 疵点分类 疵点位置回归
1 [18] VGG16
2 [25] CNN
3 [26] FPN
4 [35] CNN
5 [36] PE TMCNN
6 [37] CNN
7 [15] Faster RCNN, ResNet
8 [27] Faster RCNN
9 [28] GAN
10 [29] Faster RCNN, GAN
11 [30] DCGAN
12 [14] YOLOv3
13 [31] Faster RCNN
14 [32] RefinDet
15 [33] DUW-CAM, L-SE
16 [34] CNN

表2

基于卷积神经网络的织物疵点检测方法性能"

序号 文献 数据集 精度 检测时间/(张·s-1) 疵点类型 硬件资源
1 [18] 自建数据集 亚麻布mAP:93.4%;花纹织物mAP:95.1% 亚麻布:21.5;花纹织物:21.1 破洞、污渍等5种 GTX 1060 GPU
2 [25] TILDA数据集 Acc:96.55% 19.6 油污、跳纱等6种 GTX 1280 GPU
3 [26] FBDF数据集 mAP:72.6% 断裂、错纬等20种 TITAN RTX GPU
4 [35] TILDA数据集和自建数据集 TILDA数据集Acc:90.25%
自建数据集Acc:94.6%
TILDA数据集:18.3
自建数据集:20.6
纬缩、拆痕等7种 NVIDIA GTX 1080TI GPU
5 [36] Kylberg数据集和自建数据集 Kylberg数据集Acc:80.32%
自建数据集Acc:83.81%
浆斑、油纱等14种 GTX 980Ti Nvidia GPU
6 [37] 自建数据集,Knitting数据集和Nano fiber数据集 自建数据集Acc:93.81%
Knitting数据集Acc:95.61%
Nano fiber数据集Acc:81.82%
自建数据集:23.9
Knitting数据集:23.9
Nano fiber数据集:23.0
脱纬、跳花等18种 GeForce RTX 2080 Ti
7 [15] 现场采集坯布 坯布mAP:94.71% 7.7 褶皱、线条等4种 NVIDIA GTX1080Ti
8 [27] 天池布匹数据集 Acc:94.67%, Recall:98.63% 4.3 污渍、断纬和破洞 Intel Core i5-82500U
9 [28] 自建数据集 mAP:94.8% 棉结、污渍等8种
10 [29] 机器采集花纹织物 mAP:93.62% 5.4 带纱、破洞等6种 NVIDIA GeForce GTX 745
11 [30] TILDA数据集和自建数据集 TILDA数据集mAP:85.14%;自建数据集mAP:93.45% TILDA数据集:37;自建数据集:33 错纬、污渍等14种 NVIDIA Tesla P4
12 [14] 机器采集坯布和格纹布 坯布错误率:1.3%;格纹布错误率:1.02% 坯布:27.7;格纹布:21.8 破洞、油污等6种 GTX1080 Ti
13 [31] 机器采集斜纹和牛仔布 斜纹mAP:62.3%;
牛仔布mAP:92.5%
斜纹:9.7;牛仔布:25.4 错纬、折痕等12种 RTX2080Ti GPU
14 [32] TILDA数据集,香港图案纹理数据集和DAGM200数据集 TILDA数据集mAP:80.2%;
香港图案数据集mAP:85.9%;
DAGM2007数据集mAP:96.7%
TILDA数据集:34.0;香港图案数据集:21.9;DAGM2007数据集:45.2 破洞、棉球等13种 NVIDA TITAN XP GPU
15 [33] DAGM2007数据集和机器采集 DAGM2007数据集mAP:90.29%;
机器采集数据集mAP:93.27%
污渍、破洞等8种 Nvidia Quadro M5000 GPU
16 [34] 天池布匹数据集 mAP:90.4% 10.3 污渍、断纬和破洞 NVIDIA TITAN XP GPU

表3

基于自动编码器的织物疵点检测方法性能"

序号 文献 数据集 精度 检测时间/(张·s-1) 疵点类型 硬件资源
1 [38] 自建数据集 mAP:98.21% 4.76 浆斑、破洞等4种 Intel i5 processor
2 [39] TILDA数据集 mAP:83.9% 穿错、毛边等12种 GTX 980Ti Nvidia GPU
3 [40] TILDA数据集和机器采集数据集 TILDA数据集mAP:87.2%;
机器采集数据集mAP:79.2%
断经、经缩等11种 Intel®Core i5-4460
4 [41] VTC-2K10.5G-C19数据集 Recall:69.7%
Precision:82.2%
跳花、百脚等15种

表4

基于其它深度学习模型的织物疵点检测方法性能"

序号 文献 数据集 精度/% 检则时间/(张·s-1) 疵点类型 硬件资源
1 [42] TILDA数据集 油污、跳纱等6种
2 [35] 自建数据集 mAP:94.57% 16.2 破洞、污渍等8种 Titan RTX GPU (24G)
3 [35] 自建数据集 Acc:0.95%
AP:95.8%
毛边、断经等6种 NVIDIA GTX 1080TI
[1] ZHU Q P, WU M Y, LI J, et al. Fabric defect detection via small scale over-complete basis set[J]. Textile Research Journal, 2014, 84(15):1634-1649.
doi: 10.1177/0040517514525880
[2] 纺织行业“十四五”发展纲要[J]. 纺织科学研究, 2021(7): 40-49.
Development outline of textile industry during the 14th five-year plan[J]. Textile Science Research, 2021(7): 40-49.
[3] 吕文涛, 林琪琪, 钟佳莹, 等. 面向织物疵点检测的图像处理技术研究进展[J]. 纺织学报, 2021, 42(11): 197-206.
LÜ Wentao, LIN Qiqi, ZHONG Jiaying, et al. Research progress of image processing technology for fabric defect detection[J]. Journal of Textile Research, 2021, 42(11): 197-206.
[4] 郑胤, 陈权崎, 章毓晋. 深度学习及其在目标和行为识别中的新进展[J]. 中国图像图形学报, 2014, 19(2): 175-184.
ZHENG Yin, CHEN Quanqi, ZHANG Yujin. Deep Learning and its new progress in goal and behavior recognition[J]. Journal of Image and Graphics, 2014, 19(2): 175-184.
[5] 景军锋, 李江南, 李鹏飞. 改进型奇异值分解在织物疵点检测上的应用[J]. 纺织学报, 2014, 35(6): 62-67.
JING Junfeng, LI Jiangnan, LI Pengfei. Application of improved singular value decomposition in fabric defect detection[J]. Journal of Textile Research, 2014, 35(6): 62-67.
[6] 刘洲峰, 赵全军, 李春雷, 等. 基于局部统计与整体显著性的织物疵点检测算法[J]. 纺织学报, 2014, 35(11): 62-67.
LIU Zhoufeng, ZHAO Quanjun, LI Chunlei, et al. Fabric defect detection algorithm based on local statistics and global significance[J]. Journal of Textile Research, 2014, 35(11): 62-67.
[7] 张缓缓, 李仁忠, 景军锋, 等. Frangi滤波器和模糊C均值算法相结合的织物瑕疵检测[J]. 纺织学报, 2015, 36(9): 120-124.
ZHANG Huanhuan, LI Renzhong, JING Junfeng, et al. Fabric defect detection based on Frangi filter and Fuzzy C-means algorithm[J]. Journal of Textile Research, 2015, 36(9): 120-124.
[8] 景军锋, 赵娟. 基于MeanShift滤波的织物疵点检测方法[J]. 电子测量与仪器学报, 2016, 30(5): 739-747.
JING Junfeng, ZHAO Juan. Fabric defect detection method based on MeanShift filter[J]. Journal of Electronic Measurement and Instruments, 2016, 30(5): 739-747.
[9] DI L, LONG H, LIANG J. Fabric defect detection based on illumination correction and visual salient features[J]. Sensors (Basel), 2020. DOI: 10.3390/s20185147.
doi: 10.3390/s20185147
[10] PRACH Y, BUMRUNG K. Defect detection in textile fabrics with snake active contour and support vector machines[J]. Journal of Physics: Conference Series, 2019. DOI: 10.1088/1742-6596/1195/1/012006.
doi: 10.1088/1742-6596/1195/1/012006
[11] LI Y Y, LUO H C, YU M M, et al. Fabric defect detection algorithm using RDPSO-based optimal Gabor filter[J]. Journal of The Textile Institute, 2019, 110(4):487-495.
doi: 10.1080/00405000.2018.1489951
[12] 杨晓波. 基于GMRF模型的统计特征畸变织物疵点识别[J]. 纺织学报, 2013, 34(4): 137-142.
YANG Xiaobo. Fabric defect detection of statistic aberration feature based on GMRF model[J]. Journal of Textile Research, 2013, 34(4): 137-142.
[13] 杨晓波. 基于Gabor滤波器的织物疵点检测[J]. 纺织学报, 2010, 31(4): 55-59.
YANG Xiaobo. Fabric defect detection based on Gabor filter[J]. Journal of Textile Research, 2010, 31(4): 55-59.
[14] JING J F, ZHUO D, ZHANG H H, et al. Fabric defect detection using the improved YOLOv3 model[J]. Journal of Engineered Fibers and Fabrics, 2020, 2(4):1-10.
[15] 晏琳, 景军锋, 李鹏飞. Faster RCNN模型在坯布疵点检测中的应用[J]. 棉纺织技术, 2019, 47(2): 24-27.
YAN Lin, JING Junfeng, LI Pengfei. Application of faster RCNN model in defect detection of gray fabric[J]. Cotton Textile Technology, 2019, 47(2): 24-27.
[16] 杜磊, 李立轻, 汪军, 等. 几种基于图像自适应阈值分割的织物疵点检测方法比较[J]. 纺织学报, 2014, 35(6): 56-61.
DU Lei, LI Liqing, WANG Jun, et al. Comparison of several fabric defect detection methods based on image adaptive threshold segmentation[J]. Journal of Textile Research, 2014, 35(6): 56-61.
[17] 管声启, 石秀华. 基于频域滤波的织物疵点检测[J]. 计算机应用, 2008(10): 2673-2675.
GUAN Shengqi, SHI Xiuhua. Fabric defect detection based on frequency domain filtering[J]. Journal of Computer Application, 2008(10): 2673-2675.
[18] ZHAO S X, YIN L, ZANG J, et al. Real-time fabric defect detection based on multi-scale convolutional neural network[J]. IET Collaborative Intelligent Manufacturing, 2020, 2(4):189-196.
doi: 10.1049/iet-cim.2020.0062
[19] 汪坤. 基于深度学习的织物疵点检测算法研究[D]. 杭州: 浙江理工大学, 2020:22-23.
WANG Kun. Research on fabric defect detection algorithm based on deep learning[D]. Hangzhou: Zhejiang Sci-Tech University, 2020:22-23.
[20] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.
doi: 10.1145/3065386
[21] ZEILER M D, KRISHNAN D, TAYLOR G W, et al. Deconvolutional networks[C] // 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Illinois :IEEE, 2010:2528-2535.
[22] 邓俊锋, 张晓龙. 基于自动编码器组合的深度学习优化方法[J]. 计算机应用, 2016, 36(3): 697-702.
doi: 10.11772/j.issn.1001-9081.2016.03.697
DENG Junfeng, ZHANG Xiaolong. Deep learning optimization method based on automatic encoder combination[J]. Journal of Computer Application, 2016, 36(3): 697-702.
[23] GEOFFREY H, RUSLAN S. Discovering binary codes for documents by learning deep generative models[J]. Topics in Cognitive Science, 2011, 3(1):74-91.
doi: 10.1111/j.1756-8765.2010.01109.x pmid: 25164175
[24] GEOFFREY E, SIMON O, YEE T. A fast learning algorithm for deep belief nets[J]. Neural Computation, 2006, 18(7):1527-1554.
doi: 10.1162/neco.2006.18.7.1527 pmid: 16764513
[25] PANDIA R J, EDWARD R S J. Computer vision for automatic detection and classification of fabric defect employing deep learning algorithm[J]. International Journal of Clothing Science and Technology, 2019, 31(4):510-521.
doi: 10.1108/IJCST-11-2018-0135
[26] WU Y, ZHANG X D, FANG F Z. Automatic fabric defect detection using cascaded mixed feature pyramid with guided localization[J]. Sensors, 2020, 20(3):871.
doi: 10.3390/s20030871
[27] 安萌, 郑飂默, 王诗宇, 等. 一种改进Faster R-CNN的面料疵点检测方法[J]. 小型微型计算机系统, 2021, 42(5): 1029-1033.
AN Meng, ZHENG Liaomo, WANG Shiyu, et al. A fabric defect detection method based on improved Faster R-CNN[J]. Journal of Chinese Computer Systems, 2021, 42(5): 1029-1033.
[28] LIU J H, WANG C Y, SU H, et al. Multistage GAN for fabric defect detection[J]. IEEE Transactions on Image Processing, 2019, 29(1):3388-3400.
doi: 10.1109/TIP.2019.2959741
[29] 李明, 景军锋, 李鹏飞. 应用GAN和Faster R-CNN的色织物缺陷识别[J]. 西安工程大学学报, 2018, 32(6): 663-669.
LI Ming, JING Junfeng, LI Pengfei. Defect recognition of dyed fabric using GAN and Faster R-CNN[J]. Journal of Xi'an Polytechnic University, 2018, 32(6): 663-669.
[30] HU G H, HUANG J F, WANG Q H, et al. Unsupervised fabric defect detection based on a deep convolutional generative adversarial network[J]. Textile Research Journal, 2020, 90(3/4):247-270.
doi: 10.1177/0040517519862880
[31] PENG P R, WANG Y, HAO C, et al. Automatic fabric defect detection method using PRAN-Net[J]. Applied Sciences, 2020, 10(23):8434-8437.
doi: 10.3390/app10238434
[32] XIE H, WU Z. A robust fabric defect detection method based on improved RefineDet[J]. Sensors (Basel), 2020, 20(15):4260-4284.
doi: 10.3390/s20154260
[33] LIU Z F, HUO Z C, LI C L, et al. DLSE-net: a robust weakly supervised network for fabric defect detection[J]. Displays, 2021, 68(12):102-110.
[34] JUN X, WANG J G, ZHOU J, et al. Fabric defect detection based on a deep convolutional neural network using a two-stage strategy[J]. Textile Research Journal, 2021, 91(1/2):130-142.
doi: 10.1177/0040517520935984
[35] LI C X, LIU T, YE W. A novel feature-based network with sequential information for textile defect detection[J]. Coloration Technology, 2020, 136(6):476-484.
doi: 10.1111/cote.12493
[36] WEN Z J, ZHAO Q K, TONG L N. CNN-based minor fabric defects detection[J]. International Journal of Clothing Science and Technology, 2020, 33(1):1-12.
doi: 10.1108/IJCST-11-2019-0177
[37] HO C C, CHOU W C, SU E G. Deep convolutional neural network optimization for defect detection in fabric inspection[J]. Sensors, 2021, 21(21):7074.
doi: 10.3390/s21217074
[38] LI Y D, ZHAO W G, PAN J H. Deformable patterned fabric defect detection with fisher criterion-based deep learning[J]. IEEE Transactions on Automation Science and Engineering, 2017, 14(2):1256-1264.
doi: 10.1109/TASE.2016.2520955
[39] MEI S, WANG Y, WEN G. Automatic fabric defect detection with a multi-scale convolutional denoising autoencoder network model[J]. Sensors (Basel), 2018. DOI:10.3390/s18041064.
doi: 10.3390/s18041064
[40] 景军锋, 党永强, 苏泽斌, 等. 基于改进SAE网络的织物疵点检测算法[J]. 电子测量与仪器学报, 2017, 31(8): 1321-1329.
JING Junfeng, DANG Yongqiang, SU Zebin, et al. Fabric defect detection algorithm based on improved SAE network[J]. Journal of Electronic Measurement and Instrument, 2017, 31(8): 1321-1329.
[41] HAN Y J, YU H J. Fabric defect detection system using stacked convolutional denoising auto-encoders trained with synthetic defect data[J]. Applied Sciences, 2020, 10(7):26-36.
doi: 10.3390/app10010026
[42] 刘洲峰, 闫磊, 李春雷, 等. 基于稀疏优化的织物疵点检测算法[J]. 纺织学报, 2016, 37(5):56-61.
LIU Zhoufeng, YAN Lei, LI Chunlei, et al. Fabric defect detection algorithm based on sparse optimization[J]. Journal of Textlie Research, 2016, 37(5):56-61.
[43] CHEN M Q, YU L J, ZHI C, et al. Improved faster R-CNN for fabric defect detection based on Gabor filter with Genetic algorithm optimization[J]. Computers in Industry, 2022.DOI: j.compind.2021.103551.
doi: j.compind.2021.103551
[1] 乌婧, 江振林, 吉鹏, 谢锐敏, 陈烨, 陈向玲, 王华平. 纺织品前瞻性制备技术及应用研究现状与发展趋势[J]. 纺织学报, 2023, 44(01): 1-10.
[2] 陈佳, 杨聪聪, 刘军平, 何儒汉, 梁金星. 手绘草图到服装图像的跨域生成[J]. 纺织学报, 2023, 44(01): 171-178.
[3] 蒲海红, 贺芃鑫, 宋柏青, 赵丁莹, 李欣峰, 张天一, 马建华. 纤维素/碳纳米管复合纤维的制备及其功能化应用[J]. 纺织学报, 2023, 44(01): 79-86.
[4] 张楚丹, 王锐, 王文庆, 刘燕燕, 陈睿. 阳离子改性阻燃涤纶织物的制备及其性能[J]. 纺织学报, 2022, 43(12): 109-117.
[5] 尹喆, 赵海浪, 徐红, 毛志平, 谭玉静. 纺织品中喹啉含量的快速测定[J]. 纺织学报, 2022, 43(12): 125-130.
[6] 安亦锦, 薛文良, 丁亦, 张顺连. 基于图像处理的纺织品耐摩擦色牢度评级[J]. 纺织学报, 2022, 43(12): 131-137.
[7] 赵伦玉, 隋晓锋, 毛志平, 李卫东, 冯雪凌. 气凝胶材料在纺织品上的应用研究进展[J]. 纺织学报, 2022, 43(12): 181-189.
[8] 方寅春, 陈吕鑫, 李俊伟. 阻燃超疏水涤/棉混纺织物的制备及其性能[J]. 纺织学报, 2022, 43(11): 113-118.
[9] 顾梅花, 刘杰, 李立瑶, 崔琳. 结合特征学习与注意力机制的服装图像分割[J]. 纺织学报, 2022, 43(11): 163-171.
[10] 张福沐, 刘端武, 胡跃明. 染色机助剂智能配送系统的构建及实践[J]. 纺织学报, 2022, 43(11): 179-187.
[11] 曹聪聪, 汤龙世, 刘元军, 赵晓明. 无机抗菌织物的研究进展[J]. 纺织学报, 2022, 43(11): 203-211.
[12] 王铭亮, 张慧乐, 岳晓丽, 陈慧敏. 基于数字图像相关法的纱线和织物微观变形测量[J]. 纺织学报, 2022, 43(11): 52-58.
[13] 陈金广, 李雪, 邵景峰, 马丽丽. 改进YOLOv5网络的轻量级服装目标检测方法[J]. 纺织学报, 2022, 43(10): 155-160.
[14] 程宁波, 缪东洋, 王先锋, 王朝晖, 丁彬, 俞建勇. 用于个人热湿舒适管理的功能纺织品研究进展[J]. 纺织学报, 2022, 43(10): 200-208.
[15] 肖渊, 李倩, 张威, 胡汉春, 郭鑫雷. 微喷印原电池置换成型织物基柔性导电线路的影响因素研究[J]. 纺织学报, 2022, 43(10): 89-96.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!