基于深度强化学习的服装缝制过程实时动态调度

doi:10.13475/j.fzxb.20220407908

Abstract

Abstract:

The garment sewing process is prone to interference caused by dynamic events. With the objective of minimizing the completion cycle, a deep reinforcement learning-based real-time dynamic scheduling method for the garment sewing process was proposed for the dynamic events of real-time order arrival. A scheduling optimization model of garment sewing process was established, and the problem was transformed into a sequential decision problem based on Markov decision process. By defining state features, action set, reward function, exploration and exploitation strategy, combined with the DDQN algorithm to train a deep neural network to describe the state-action value, the most appropriate scheduling rule was selected at the decision node. Experiments show that for the sewing process of the front panel of jeans, the proposed method is 2.3% inferior to the genetic algorithm in achieving the scheduling goal, but the decision-making time is greatly reduced by 91.4%. The method is useful to achieve effective real-time response and ensure the efficiency and continuity of sewing production for the scheduling problem of dynamic order arrival.

Key words: garment sewing production, dynamic scheduling method, reinforcement learning method, deep neural network, smart manufacturing

CLC Number:

TS941

LIU Feng, XU Jie, KE Wenbo. Real-time dynamic scheduling for garment sewing process based on deep reinforcement learning[J].Journal of Textile Research, 2022, 43(09): 41-48.

Figures/Tables 7

Tab.1

Fig.1

Tab.2

Candidate actions set"

序号	动作代码	具体描述
1	SFPT	选择第1道工序加工时间最短的工件
2	LFPT	选择第1道工序加工时间最长的工件
3	SFPT/APT	选择第1道工序加工时间与所有工序总时间比值最小的工件
4	LFPT/APT	选择第1道工序加工时间与所有工序总时间比值最大的工件
5	SLPT	选择最后1道工序加工时间最短的工件
6	LLPT	选择最后1道工序加工时间最长的工件
7	SLPT/APT	选择最后1道工序加工时间与所有工序总时间比值最小的工件
8	LLPT/APT	选择最后1道工序加工时间与所有工序总时间比值最大的工件
9	SHPT	前一半工序时间最短
10	LHPT	前一半工序时间最长
11	SHPT/APT	前一半工序时间与所有工序总时间比值最小的工件
12	LHPT/APT	前一半工序时间与所有工序总时间比值最大的工件
13	SAPT	所有工序总时间最短
14	LAPT	所有工序总时间最长
15	Gupta	对每个工件计算斜度参数,即s_i= $C m i n j ∈ {1,2, 3, …, m - 1} (t i, j + t i, j + 1)$ ,按照s_i的非减顺序排列各工件,选择s_i最小的工件。其中C= $1, 若 t i, m ≤ t i, 1 - 1, 其他$ ,i=1,2,…,n
16	NEH_KK1	计算a_i= $∑ j = 1 m$ ((m-1)(m-2)/2+m-j)t_i_,_j b_i= $∑ j = 1 m$ ((m-1)(m-2)/2+j-1)t_i_,_j i=1,2,…,n 按照min(a_i,b_i)非增顺序排列各工件,选择min(a_i,b_i)最大的工件

Tab.2

Tab.3

Framework of DDQN based training algorithm"

步骤	算法描述
1	初始化回放缓存记忆体D并设定其最大容量为M
2	初始化在线评估网络Q及其参数θ
3	初始化目标网络 $Q^$ 并设定其参数 $θ^$ =θ
4	for e_iter=1∶M:
5	建立初始状态s₀(按本文3.1节所述方法)
6	更新ε值(按式(14))
7	设置done=False(done==False表示还有待加工服装未进入加工过程;done==True表示所有服装均进入加工过程)
8	while(done==False):
9	以1-ε的概率随机选择一个动作或者依据网络Q评估出各动作q值,并选择当前最优动作a(按式(13))
10	执行动作a_t得到新的状态s_t₊₁和回报r_t,以及更新done_t
11	将(s_t,a_t,r_t,s_t₊₁,done_t)存入D
12	从D中以批量形式采样(s_i,a_i,r_i,s_i₊₁,done_i)
13	更新目标值 y= $r i d o n e i = = T r u e r i + γ · Q^(s i + 1, a r g m a x a Q (s i + 1, a; θ); θ^) d o n e i = = F a l s e$
14	更新Q的参数θ,使Q(s_i,a_i)的值更接近y
15	每隔C步将参数θ赋予 $θ^$

Tab.3

Tab.4

Fig.2

Fig.3

References 15

[1]	LI Xinyu, GAO Liang. An effective hybrid genetic algorithm and tabu search for flexible job shop scheduling problem[J]. International Journal of Production Economics, 2016, 174:93-110. doi: 10.1016/j.ijpe.2016.01.016
[2]	DAI Min, TANG Dunbing, GIRET A, et al. Energy-efficient scheduling for a flexible flow shop using an improved genetic-simulated annealing algorithm[J]. Robotics and Computer-Integrated Manufacturing, 2013, 29(5):418-429. doi: 10.1016/j.rcim.2013.04.001
[3]	郑立平, 郝忠孝. 遗传算法理论综述[J]. 计算机工程与应用, 2003(21): 50-53.
	ZHENG Liping, HAO Zhongxiao. Review of genetic algorithm theory[J]. Computer Engineering and Applications, 2003(21): 50-53.
[4]	郑卫波, 周俊, 许正良. 基于遗传算法的服装企业计划调度建模与求解[J]. 江苏纺织, 2009(1):59-60.
	ZHENG Weibo, ZHOU Jun, XU Zhengliang. Modeling and solving of scheduling of garment enterprise based on genetic algorithm[J]. Jiangsu Textile, 2009(1):59-60.
[5]	谢子昂, 杜劲松, 赵国华. 衬衫吊挂流水线的自适应动态调度[J]. 纺织学报, 2020, 41(10):144-149.
	XIE Ziang, DU Jinsong, ZHAO Guohua. Adaptive dynamic scheduling of garment hanging production line[J]. Journal of Textile Research, 2020, 41(10):144-149.
[6]	NAWAZ Muhammad, ENSCORE Emory, HAM Inyong. A heuristic algorithm for the m-machine, n-job flow-shop sequencing problem[J]. Omega, 1983, 11(1): 91-95. doi: 10.1016/0305-0483(83)90088-9
[7]	FRAMINAN J M, GUPTA J N, LEISTEN R. A review and classification of heuristics for permutation flow-shop scheduling with makespan objective[J]. Journal of the Operational Research Society, 2004, 55(12): 1243-1255. doi: 10.1057/palgrave.jors.2601784
[8]	贺俊杰, 张洁, 张朋, 等. 基于多智能体强化学习的纺织面料染色车间动态调度方法[J/OL]. 计算机集成制造系统:1-31[2022-04-20]. http://kns.cnki.net/kcms/detail/11.5946.tp.20210227.1731.002.html.
	HE Junjie, ZHANG Jie, ZHANG Peng, et al. Multi-agent reinforcement learning based textile dyeing workshop dynamic scheduling method[J/OL]. Computer Integrated Manufacturing Systems:1-31[2022-04-20]. http://kns.cnki.net/kcms/detail/11.5946.tp.20210227.1731.002.html.
[9]	LUO Shu. Dynamic scheduling for flexible job shop with new job insertions by deep reinforcement learning[J]. Applied Soft Computing, 2020. DOI: 10.1016/j.asoc.2020.106208. doi: 10.1016/j.asoc.2020.106208
[10]	孔松涛, 刘池池, 史勇, 等. 深度强化学习在智能制造中的应用展望综述[J]. 计算机工程与应用, 2021, 57(2):49-59. doi: 10.3778/j.issn.1002-8331.2008-0431
	KONG Songtao, LIU Chichi, SHI Yong, et al. Review of application prospect of deep reinforcement learning in intelligent manufacturing[J]. Computer Engineering and Applications, 2021, 57(2):49-59. doi: 10.3778/j.issn.1002-8331.2008-0431
[11]	SUTTON R S, BARTO A G. Reinforcement learning: an introduction[M]. London: MIT press, 2018: 119-133.
[12]	MNIH V, KAVUKCUOGLU K, SILVER D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529-533. doi: 10.1038/nature14236
[13]	HASSELT Hado Van, GUEZ Arthur, SILVER David. Deep reinforcement learning with double Q-learning[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2016, 30(1): 2094-2100.
[14]	肖鹏飞, 张超勇, 孟磊磊, 等. 基于深度强化学习的非置换流水车间调度问题[J]. 计算机集成制造系统, 2021, 27(1): 192-205.
	XIAO Pengfei, ZHANG Chaoyong, MENG Leilei, et al. Non-permutation flow shop scheduling problem based on deep reinforcement learning[J]. Computer Integrated Manufacturing Systems, 2021, 27(1):192-205.
[15]	HAN Baoan, YANG Jianjun. Research on adaptive job shop scheduling problems based on dueling double DQN[J]. IEEE Access, 2020, 8: 186474-186495. doi: 10.1109/ACCESS.2020.3029868

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

参数符号	符号说明
n	需要加工的服装件数
m	服装的加工工序数
x_j	第j个缝制工序上同时并行工位的数量
J_i	需要加工的服装集合{J₁,…, J_i,…, J_n}
O_i_,_j	服装J_i的第j个加工工序
M_j_,_k	第j个工序上的第k个并行工位
A_i	服装J_i加工任务的下达时间
C_i_,_j	服装J_i在工序j上加工结束的时间
t_i_,_j_,_k	服装i在工序j的并行工位M_j_,_k的加工持续时间
X_i_,_j_,_k	决策变量,服装i在工序j加工时,是否处于并行工位M_j_,_k。如果是,X_i_,_j_,_k=1;否则,X_i_,_j_,_k=0
Y_i_,_j_,_t	决策变量,t时刻服装J_i是否正在O_j工序上加工。如果是,Y_i_,_j_,_t=1;否则,Y_i_,_j_,_t=0
F_i_,_j_,_k_,_l	决策变量,服装J_i是否在O_j工序的M_j_,_k并行工位上的第l顺位加工。如果是,F_i_,_j_,_k_,_l=1;否则,F_i_,_j_,_k_,_l=0

Real-time dynamic scheduling for garment sewing process based on deep reinforcement learning

RichHTML

PDF (PC)

Abstract

Cite this article

share this article

Figures/Tables 7

References 15

Related Articles 3

Metrics

Comments

Recommended 0

工序序号	工序内容	加工时间/s								设备(数量)
工序序号	工序内容	款1	款2	款3	款4	款5	款6	款7	款8	设备(数量)
1	平车装表袋口	30	36	32	30	40	27	30	35	平缝机(1)
2	烫装表袋实样	30	30	30	30	30	30	30	30	烫台(1)
3	平车装表袋内外线	90	105	100	95	110	85	90	100	平缝机(3)
4	绷缝前袋布	36	43	40	40	45	32	36	40	绷缝机(1)
5	平缝袋布于前片上	33	40	35	33	45	30	33	35	平缝机(1)
6	车袋布底	60	60	60	60	60	60	60	60	平缝机(2)
7	双针车前袋口	50	60	55	55	65	45	50	55	双针机(2)
8	平车定前袋口	70	85	75	70	85	65	70	70	平缝机(2)
9	门襟黏衬	25	25	25	25	25	25	25	25	烫台(1)
10	三线车门襟、三线车前浪	50	60	55	55	65	48	53	60	三线缝边机(2)
11	双针车拉链	50	60	55	55	65	50	55	60	双针机(2)
12	平缝门襟于前中间暗边线	60	70	65	65	75	55	65	70	平缝机(2)
13	双针车门襟	30	35	33	35	40	25	35	35	双针机(1)
14	里襟三层合一	30	35	33	35	40	25	35	35	三线缝边机(1)
15	双针车封前浪	60	70	65	65	75	55	65	70	双针机(2)

[1]	ZHANG Jie, XU Chuqiao, WANG Junliang, ZHENG Xiaohu. Advancement in data-driven intelligent control system for roboticized textile production [J]. Journal of Textile Research, 2022, 43(09): 1-10.
[2]	MAO Huimin, SUN Lei, TU Jiajia, SHI Weimin. Key technology for yarn automatic splicer [J]. Journal of Textile Research, 2022, 43(09): 21-26.
[3]	GAO Xiaofei, QI Lizhe, SUN Yunquan. Design of shape-following manipulator for three-dimensional sewing of flexible fabrics [J]. Journal of Textile Research, 2022, 43(09): 27-33.