基于机器视觉的缝纫线迹缺陷检测方法

doi:10.13475/j.fzxb.20230708401

纺织学报 ›› 2024, Vol. 45 ›› Issue (07): 173-180.doi: 10.13475/j.fzxb.20230708401

基于机器视觉的缝纫线迹缺陷检测方法

陈育帆¹, 郑小虎²^,³^,⁴(), 徐修亮⁵, 刘冰⁶

1.东华大学信息科学与技术学院, 上海 201620
2.东华大学人工智能研究院, 上海 201620
3.纺织工业人工智能技术教育部工程研究中心, 上海 201620
4.上海工业大数据与智能系统工程技术研究中心, 上海 201620
5.上海富山精密机械科技有限公司, 上海 201599
6.杭州中服科创研究院有限公司, 浙江杭州 311199

收稿日期:2023-07-31 修回日期:2024-03-26 出版日期:2024-07-15 发布日期:2024-07-15
通讯作者: 郑小虎(1983—),男,副教授,博士。主要研究方向为工业人工智能应用。E-mail:xhzheng@dhu.edu.cn。
作者简介:陈育帆(2000—),男,硕士生。主要研究方向为机器视觉、目标检测。
基金资助:
中央高校基本科研业务费专项资金资助项目(2232021D-15);上海市科技计划项目(20DZ2251400)

Machine vision-based defect detection method for sewing stitch traces

CHEN Yufan¹, ZHENG Xiaohu²^,³^,⁴(), XU Xiuliang⁵, LIU Bing⁶

1. College of Information Science and Technology, Donghua University, Shanghai 201620, China
2. Institute of Artificial Intelligence, Donghua University, Shanghai 201620, China
3. Engineering Research Center of Artificial Intelligence for Textile Industry, Ministry of Education, Shanghai 201620, China
4. Shanghai Industrial Big Data and Intelligent Systems Engineering Technology Center, Shanghai 201620, China
5. HIKARI (Shanghai) Precise Machinery Scientific & Technology Co., Ltd., Shanghai 201599, China
6. Hangzhou Zhongfu Technology & Innovation Research Institute Co., Ltd., Hangzhou, Zhejiang 311199, China

Received:2023-07-31 Revised:2024-03-26 Published:2024-07-15 Online:2024-07-15

摘要/Abstract

摘要：

缝纫线迹缺陷检测过程易受缝纫机抖动和面料移动过快等影响,针对缺陷检测过程中的扰动影响,以高精度和快速检测缺陷特征为目标,提出一种基于机器视觉的缝纫线迹缺陷检测方法。首先将主干网络的标准卷积改用蓝图卷积的DeblurGAN-v2算法和拉普拉斯算法联用,分辨模糊与清晰图像,并对运动模糊图像去模糊。然后将师生特征金字塔匹配算法应用到缝纫线迹缺陷检测上,将困难样本挖掘技术应用到师生特征金字塔匹配算法中提高了算法的检测精度与速度。结果表明:图像去模糊算法有效地去除了由外部干扰引起的图像模糊问题,缺陷检测算法检测正确率保持在95%以上,单张图片检测速度在0.04 s以下。本文方法能有效检测线迹缺陷特征,保障生产的高效性和连续性。

关键词: 机器视觉, 缝纫线迹, 缺陷检测, DeblurGAN-v2算法, 服装质量

Abstract:

Objective In order to solve the problems of slow speed, low efficiency, and high cost in conventional manual quality inspection methods for sewing thread, this study proposes a machine vision-based method for sewing thread defect detection in seams. This study aims to achieve fast, accurate, and automated identification of common defects such as cast thread, jumper thread, and broken thread in seams. This study also highlights the importance and necessity of improving product quality and production efficiency in the textile and garment industry.

Method This study adopts a two-step approach for defect detection. Firstly, a low-cost array camera was adopted to capture real-time images of the sewing seam and the DeblurGAN-v2 method was employed to remove motion blurriness from the images, aiming at improving image clarity. Secondly, the student-teacher feature pyramid matching method was applied for anomaly detection, which transfers the knowledge from a pre-trained ResNet-34 model as the teacher network to a student network with the same architecture, so as to learn the distribution of normal images. By comparing the differences between the feature pyramids generated by the two networks as a scoring method, the defect detection system made decisions on whether the image has anomalies, and marked the abnormal areas with a heat distribution map.

Results The defects of flat stitch fabric and overstitch fabric were tested and the performance of the proposed method was evaluated in terms of recall and accuracy rates. The results show that the proposed method can effectively detect various sewing thread defects and has high recall and accuracy rates for different types of defects. This study also provided some examples of defect detection results and scores for different types of defects.

Conclusion The feature pyramid matching technique is applied in the field of stitch trace detection. By adding the difficult sample mining technology, the average detection accuracy is increased to more than 95%, and the detection speed of a single image is less than 0.04 s. Aiming at image motion blur ring caused by jitter and fast movement. The DeblurGAN-v2 framework is used as the framework of deblurring algorithm, and the blueprint convolution is added to change the backbone network, and the processing speed of a single image is kept below 0.06 s. The model has excellent interference resistance and high processing speed, and can meet the requirement of stitch trace recognition.

Key words: sewing thread defect detection, machine vision, DeblurGAN-v2, student-teacher feature pyramid matching, anomaly detection

中图分类号:

TP18

陈育帆, 郑小虎, 徐修亮, 刘冰. 基于机器视觉的缝纫线迹缺陷检测方法[J]. 纺织学报, 2024, 45(07): 173-180.

CHEN Yufan, ZHENG Xiaohu, XU Xiuliang, LIU Bing. Machine vision-based defect detection method for sewing stitch traces[J]. Journal of Textile Research, 2024, 45(07): 173-180.

导出引用管理器 EndNote|Reference Manager|ProCite|BibTeX|RefWorks

链接本文: http://www.fzxb.org.cn/CN/10.13475/j.fzxb.20230708401

http://www.fzxb.org.cn/CN/Y2024/V45/I07/173

图/表 15

图1

表1

图2

图3

图4

表2

表3

算法ResNet-34框架结构表"

卷积层	ResNet-34结构	输出尺度
Conv1x(第1层)	7×7,64,步长为2	112×112
Conv2x(第2层)	3×3,最大池化步长为2 $3 × 3,64 3 × 3,64$ ×3	56×56
Conv3x(第3层)	$3 × 3,128 3 × 3,128$ ×4	28×28
Conv4x(第4层)	$3 × 3,256 3 × 3,256$ ×6	14×14
Conv5x(第5层)	$3 × 3,512 3 × 3,512$ ×3	7×7
	平均池化,1 000维全连接层, softmax激活函数	1×1

表3

表4

图5

表5

图6

图7

表6

表7

图8

参考文献 14

[1]	吴柳波, 李新荣, 杜金丽. 基于轮廓提取的缝纫机器人运动轨迹规划研究进展[J]. 纺织学报, 2021, 42(4):191-200.
	WU Liubo, LI Xinrong, DU Jinli. Research progress of motion trajectory planning of sewing robot based on contour extraction[J]. Journal of Textile Research, 2021, 42(4): 191-200.
[2]	路浩, 陈原. 基于机器视觉的碳纤维预浸料表面缺陷检测方法[J]. 纺织学报, 2020, 41(4): 51-57.
	LU Hao, CHEN Yuan. A method for surface defect detection of carbon fiber prepreg based on machine vision[J]. Journal of Textile Research, 2020, 41(4): 51-57.
[3]	FANG B, LONG X, SUN F, et al. Tactile-based fabric defect detection using convolutional neural network with attention mechanism[J]. IEEE Transactions on Instrumentation and Measurement, 2022(71): 1-9.
[4]	ABATI D, PORRELLO A, CALDERARA S, et al. Latent space autoregression for novelty detection[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach: IEEE, 2019: 481-490.
[5]	WANG G, HAN S, DING E, et al. Student-teacher feature pyramid matching for unsupervised anomaly detection[J]. arXiv Preprint, 2021. DOI: 10.48550/arxiv.2103.04257.
[6]	BERGMANN P, FAUSER M, SATTLEGGER D, et al. Uninformed students: student-teacher anomaly detection with discriminative latent embeddings[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle: IEEE,2020:4183-4192.
[7]	KIM J H, KIM N, PARK Y W, et al. Object detection and classification based on YOLOv5 with improved maritime dataset[J]. Journal of Marine Science and Engineering, 2022, 10(3): 377.
[8]	REN S, HE K, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149. doi: 10.1109/TPAMI.2016.2577031 pmid: 27295650
[9]	ROZUMNYI D, OSWALD MR, FERRARI V, et al. Defmo: deblurring and shape recovery of fast moving objects[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Nashville: IEEE, 2021: 3456-3465.
[10]	KUPYN O, MARTYNIUK T, WU J, et al. Ddeblurring (orders-of-magnitude) faster and better[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul: IEEE, 201: 8878-8887.
[11]	HOWARD A, SANDLER M, CHU G, et al. Searching for mobilenetV3[C]// 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul: IEEE, 201: 1314-1324.
[12]	HAASE D, AMTHOR M. Rethinking depthwise separable convolutions: how intra-kernel correlations lead to improved mobilenets[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 14600-14609.
[13]	DEFARD T, SETKOV A, LOESCH A, et al. PaDiM: a patch distribution modeling framework for anomaly detection and localization[C]// International Conference on Pattern Recognition. Cham: Springer International Publishing, 2021: 475-489.
[14]	新刚, 蔡逸超, 周晓, 等. 基于机器视觉的筒子纱缺陷在线检测系统[J]. 纺织学报, 2018, 39 (1):139-145.
	MOU Xingang, CAI Yichao, ZHOU Xiao, et al. Online yarn cone defects detection system based on machine vision[J]. Journal of Textile Research, 2018, 39(1):139-145.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

算法类型	PSNR值/dB	SSIM值	时间/s
DeblurGAN-v2	23.85	0.65	0.05
DeblurGAN	25.23	0.72	0.86
DeblurGAN-bsv2	27.52	0.84	0.05
MIMO-UNet	20.36	0.41	0.02

ResNet模型	数据集	正确率/%	处理速度/(帧·s^-1)
ResNet-18	测试数据集	92.5	28
ResNet-34	测试数据集	94.6	22
ResNet-50	测试数据集	95.3	15

缺陷类型	召回率/%		正确率/%
缺陷类型	优化前	优化后	优化前	优化后
抛线	98.0	99.0	98.0	99.0
跳线	92.0	95.0	90.0	93.0
断线	98.0	100.0	100.0	100.0
主针线松	92.0	97.0	96.0	98.0
链针线断	97.0	98.0	98.0	100.0
上弯线紧	98.0	98.0	88.0	92.0

面料类别	线迹种类	结果	异常得分	区分阈值
	正常线迹	ok	0.31	0.94
面料1	抛线线迹	nok	1.00	0.94
	跳线线迹	nok	0.97	0.94
	断线线迹	nok	0.99	0.94
	正常线迹	ok	0.44	0.64
面料2	主针线松	nok	0.76	0.64
	链针线断	nok	0.78	0.64
	上弯线紧	nok	0.81	0.64

线迹种类	检测图像总数	正确检测个数	漏检个数	正确率/%	漏检率/%
跳线线迹	100	91	5	91.00	5.00
断线线迹	100	100	0	100.00	0.00
抛线线迹	100	98	1	98.00	1.00
主针线松	100	98	3	98.00	3.00
上弯线紧	100	96	2	96.00	2.00
链针线断	100	100	1	100.00	1.00

基于机器视觉的缝纫线迹缺陷检测方法

Machine vision-based defect detection method for sewing stitch traces

RichHTML

PDF (PC)

摘要/Abstract

引用本文

使用本文

图/表 15

参考文献 14

相关文章 15

Metrics

本文评价

推荐阅读 0

[1]	文嘉琪, 李新荣, 冯文倩, 李瀚森. 印花面料的边缘轮廓快速提取方法[J]. 纺织学报, 2024, 45(05): 165-173.
[2]	杨金鹏, 景军锋, 李吉国, 王渊博. 基于机器视觉的玻璃纤维合股纱缺陷检测系统设计[J]. 纺织学报, 2024, 45(05): 193-201.
[3]	白恩龙, 张周强, 郭忠超, 昝杰. 基于机器视觉的棉花颜色检测方法[J]. 纺织学报, 2024, 45(03): 36-43.
[4]	葛苏敏, 林瑞冰, 徐平华, 吴思熠, 罗芊芊. 基于机器视觉的曲面枕个性化定制方法[J]. 纺织学报, 2024, 45(02): 214-220.
[5]	史伟民, 韩思捷, 屠佳佳, 陆伟健, 段玉堂. 基于机器视觉的空纱筒口定位方法[J]. 纺织学报, 2023, 44(11): 105-112.
[6]	马创佳, 齐立哲, 高晓飞, 王子恒, 孙云权. 基于改进YOLOv4-Tiny的缝纫线迹质量检测方法[J]. 纺织学报, 2023, 44(08): 181-188.
[7]	陈罡, 金贵阳, 吴菁, 罗千. 智能服装缝制关键技术及成套装备研发[J]. 纺织学报, 2023, 44(08): 197-204.
[8]	陈泰芳, 周亚勤, 汪俊亮, 徐楚桥, 李冬武. 基于视觉特征强化的环锭纺细纱断头在线检测方法[J]. 纺织学报, 2023, 44(08): 63-72.
[9]	纪越, 潘东, 马杰东, 宋丽梅, 董九志. 基于机器视觉的弦振动纱线张力非接触检测系统[J]. 纺织学报, 2023, 44(05): 198-204.
[10]	陶静, 汪俊亮, 徐楚桥, 张洁. 基于视觉校准的环锭纺细纱条干特征在线提取方法[J]. 纺织学报, 2023, 44(04): 70-77.
[11]	王斌, 李敏, 雷承霖, 何儒汉. 基于深度学习的织物疵点检测研究进展[J]. 纺织学报, 2023, 44(01): 219-227.
[12]	郑兆伦, 鲁玉军. 基于奇异值分解的双算法织物缺陷检测[J]. 纺织学报, 2022, 43(11): 59-67.
[13]	金守峰, 侯一泽, 焦航, 张鹏, 李宇涛. 基于改进AlexNet模型的抓毛织物质量检测方法[J]. 纺织学报, 2022, 43(06): 133-139.
[14]	周其洪, 彭轶, 岑均豪, 周申华, 李姝佳. 基于机器视觉的细纱接头机器人纱线断头定位方法[J]. 纺织学报, 2022, 43(05): 163-169.
[15]	吕文涛, 林琪琪, 钟佳莹, 王成群, 徐伟强. 面向织物疵点检测的图像处理技术研究进展[J]. 纺织学报, 2021, 42(11): 197-206.