基于多尺度纹理合成的刺绣风格迁移模型

doi:10.13475/j.fzxb.20220504801

纺织学报 ›› 2023, Vol. 44 ›› Issue (09): 84-90.doi: 10.13475/j.fzxb.20220504801

基于多尺度纹理合成的刺绣风格迁移模型

姚琳涵¹, 张颖¹, 姚岚¹, 郑晓萍², 魏文达³, 刘成霞¹^,⁴^,⁵()

1.浙江理工大学服装学院, 浙江杭州 310018
2.中国纺织工程学会, 北京 100025
3.中国纺织建设规划院, 北京 100125
4.丝绸文化传承与产品设计数字化技术文化和旅游部重点实验室, 浙江杭州 310018
5.服装数字化技术浙江省工程实验室, 浙江杭州 310018

收稿日期:2022-05-16 修回日期:2022-12-21 出版日期:2023-09-15 发布日期:2023-10-30
通讯作者: 刘成霞(1975—),女,教授,博士。主要研究方向为服装数字化。E-mail:glorior_liu@163.com。
作者简介:姚琳涵(1998—),女,硕士生。主要研究方向为服装数字化、服装图像处理。
基金资助:
国家自然科学基金项目(51405446);浙江省自然科学基金项目(LY20E050017);浙江省大学生科技创新活动计划暨新苗人才计划项目(2020R406084)

Embroidery style transfer modeling based on multi-scale texture synthesis

YAO Linhan¹, ZHANG Ying¹, YAO Lan¹, ZHENG Xiaoping², WEI Wenda³, LIU Chengxia¹^,⁴^,⁵()

1. School of Fashion Design & Engineering,Zhejiang Sci-Tech University, Hangzhou, Zhejiang 310018, China
2. China Textile Engineering Society, Beijing 100025,China
3. China Textile Planning Institute of Construction, Beijing 100125, China
4. Key Laboratory of Silk Culture Heritage and Product Design Digital Technology, Hangzhou, Zhejiang 310018, China
5. Zhejiang Province Engineering Laboratory of Clothing Digital Technology, Hangzhou, Zhejiang 310018, China

Received:2022-05-16 Revised:2022-12-21 Published:2023-09-15 Online:2023-10-30

摘要/Abstract

摘要：

针对现有的刺绣风格模拟算法产生的图像细节不够精确,缺乏语义深度等缺点,提出了一种多尺度纹理合成的刺绣风格迁移模型(MTE-NST),该模型主要由生成网络和损失网络2部分组成,其中生成网络又包含内容匹配模块、结构增强模块和纹理精细模块。并通过引入多程式损失联合训练,分层迭代优化刺绣迁移图像,减少各个损失项对迁移效果的影响。结果表明:与现有卷积神经网络风格迁移算法对比,MTE-NST能生成更清晰的刺绣织线纹理和多方向的针脚轨迹,显著减少图片匹配错误产生的伪影,生成更逼真的刺绣艺术作品,本文研究结果有助于提高刺绣产品的外观仿真设计水平,促进刺绣技艺的发展及创新。

关键词: 风格迁移, 刺绣作品, 深度神经网络, 非真实渲染, 拉普拉斯损失

Abstract:

Objective There are still many problems in the existing embroidery image generation algorithms, such as singular style of the generated image, rough features, and many artifacts. Aiming at this situation, embroidery style transfer modeling based on multi-scale texture synthesis is proposed. By improving the existing style transfer algorithm, embroidery-style images with higher perceptual quality are expected to be created.

Method Multi-stylized loss function was adopted to extract the edge information and detailed structure of the content image and the style image, and the results were compared with the effect of the transfer image generated by only using the content loss and the style loss.

Results Experiments presults showed that the multi-stylizd loss function could generate embroidery migration images with clear texture. Gatys' model was shown to have more artifacts and deformations in the conversion process of the image with dense lines, and Johnson's model also had artifacts and conversion errors in the conversion of feather details. Li and Wand's model lacks embroidery details such as stitching and texture variations (Fig. 6). The multi-scale texture synthesis embroidery style transfer model (MTE-NST) proposed in this research transfered the detailed structure of the style map well, which was closer to the real embroidery work, and was better than the first three models in terms of style and details. MTE-NST had the smallest MSE (the smallest style loss) and the smallest LPIPS (the highest image perceptual similarity), and the image quality and transfer effect were better. The test time and occupied memory were second only to Johnson, which are 0.58 s, 3 900 MB, respectively which are quite close, further verifying that MTE-NST can generate more realistic embroidery style image (Tab. 2).

Conclusion This paper proposes a MTE-NST, which learns hierarchically the multi-scale embroidery art styles. MTE-NST can not only restore the style image color but also preserve the texture structure and fine details of the image edge, which solves the problem of texture conversion mismatch and can generate embroidery style transfer pictures with better visual effects.

Key words: style transfer, embroidery simulation, deep neural network, non-realistic rendering, Laplacian loss

中图分类号:

TS195.644

姚琳涵, 张颖, 姚岚, 郑晓萍, 魏文达, 刘成霞. 基于多尺度纹理合成的刺绣风格迁移模型[J]. 纺织学报, 2023, 44(09): 84-90.

YAO Linhan, ZHANG Ying, YAO Lan, ZHENG Xiaoping, WEI Wenda, LIU Chengxia. Embroidery style transfer modeling based on multi-scale texture synthesis[J]. Journal of Textile Research, 2023, 44(09): 84-90.

图/表 8

图1

图2

图3

表1

图4

图5

图6

表2

参考文献 15

[1]	JING Y, YANG Y, FENG Z, et al. Neural style transfer: a review[J]. IEEE Transactions on Visualization and Computer Graphics, 2019, 26(11): 3365-3385. doi: 10.1109/TVCG.2945
[2]	ZHANG W, CAO C, CHEN S, et al. Style transfer via image component analysis[J]. IEEE Transactions on multimedia, 2013, 15(7): 1594-1601. doi: 10.1109/TMM.2013.2265675
[3]	DONG Y, TAN W, TAO D, et al. CartoonLoss GAN: learning surface and coloring of images for cartoon ization[J]. IEEE Transactions on Image Processing, 2021, 31: 485-498. doi: 10.1109/TIP.2021.3130539
[4]	ZHANG S, GAO X, WANG N, et al. Robust face sketch style synthesis[J]. IEEE Transactions on Image Processing, 2015, 25(1): 220-232. doi: 10.1109/TIP.2015.2501755
[5]	FU F, LV J, TANG C, et al. Multi-style Chinese art painting generation of flowers[J]. IET Image Processing, 2021, 15(3): 746-762. doi: 10.1049/ipr2.v15.3
[6]	郑锐, 钱文华, 徐丹, 等. 基于卷积神经网络的刺绣风格数字合成[J]. 浙江大学学报, 2019, 46(3): 270-278.
	ZHENG Rui, QIAN Wenhua, XU Dan, et al Digital synthesis of embroidery style based on convolutional neural network[J] Journal of Zhejiang University, 2019, 46 (3): 270-278.
[7]	QIAN W, CAO J, XU D, et al. CNN-based embroidery style rendering[J]. International Journal of PRAI, 2020, 34(14): 205-225.
[8]	BEG M A, YU J Y. Generating embroidery patterns using image-to-image translation[EB/OL]. (2020-03-05) [2022-04-14]. https://arxiv.org/abs/2003.02909.
[9]	ZHU J Y, PARK T, ISOLA P, et al. Unpaired image to image translation using cycle consistent adversarial networks[C]// 2017 Proceedings of the ICCV. Venice: IEEE, 2017: 2223-2232.
[10]	GUAN X, LUO L, LI H, et al. Automatic embroidery texture synthesis for garment design and display online[J]. The Visual Computer, 2021, 37(9): 2553-2565. doi: 10.1007/s00371-021-02216-0
[11]	LI S, XU X, NIE L, et al. Laplacian-steered neural style transfer[C]// Proceeding of the 25th ACM International Conference on Multimedia. New York: Association for Computing Machinery, 2017: 1716-1724.
[12]	RISSER E, WILMOT P, BARNES C. Stable and controllable neural texture synthesis and style transfer using histogram losses[EB/OL]. (2017-01-31) [2022-04-14] https://arxiv.org/abs/1701.08893.
[13]	GATYS L, ECHER A, BETHGE M. Neural algorithm of artistic style[J]. Journal of Vision, 2015, 16(12):326-336. doi: 10.1167/16.12.326
[14]	ULYANOV D, LEBEDEV V, VEDALDI A, et al. Texture networks: feed-forward synthesis of textures and stylized images[C]// International Conference on Machine Learning. New York: ACM, 2016:1349-1357.
[15]	LI C, WAND M. Combining markov random fields and convolutional neural networks for image synthesis[C]// 2016 Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016:2479-2486.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

参数	含义	数值
α	内容损失权重	0.01
β	风格损失权重	1
γ	拉普拉斯损失	0.001
Epoch	迭代次数	3 000
L_r	学习率	0.001

模型类型	MSE	LPIPS	测试时间/s	占用内存/MB
文献[13]	5.81	0.664	38.34	5 900
文献[14]	2.67	0.517	0.55	3 800
文献[15]	1.61	0.193	0.62	4 600
MTE-NST	1.43	0.175	0.58	3 900

基于多尺度纹理合成的刺绣风格迁移模型

Embroidery style transfer modeling based on multi-scale texture synthesis

RichHTML

PDF (PC)

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献 15

相关文章 2

Metrics

本文评价

推荐阅读 0

[1]	陈金文, 王鑫, 罗炜豪, 梅琛楠, 韦京艳, 钟跃崎. 面向虚拟现实的着装人体个性化头面部纹理生成技术[J]. 纺织学报, 2023, 44(09): 188-196.
[2]	刘锋, 徐杰, 柯文博. 基于深度强化学习的服装缝制过程实时动态调度[J]. 纺织学报, 2022, 43(09): 41-48.