Journal of Textile Research ›› 2019, Vol. 40 ›› Issue (07): 174-181.doi: 10.13475/j.fzxb.20180900808

• Management & Information • Previous Articles     Next Articles

Clothing image retrieval by salient region detection and sketches

WU Chuanbin1, LIU Li1,2(), FU Xiaodong1,2, LIU Lijun1,2, HUANG Qingsong1,2   

  1. 1. Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, Yunnan 650500, China
    2. Computer Technology Application Key Laboratory of Yunnan Province, Kunming University of Science and Technology, Kunming, Yunnan 650500, China
  • Received:2018-09-03 Revised:2019-04-15 Online:2019-07-15 Published:2019-07-25
  • Contact: LIU Li E-mail:kmust_mary@163.com

RichHTML

9

PDF (PC)

86

Abstract:

In order to solve the problems of unsatisfactory accuracy and low efficiency in the clothing image retrieval, a sketch based clothing image retrieval method by visual salient regions and re-ranking was proposed. Firstly, clothing salient edge map was obtained by saliency detection method with regularized random walks walking and the edge map. Then, histogram of oriented gradeient features of user sketches and the salient edge in clothing images were extracted, respectively, and the feature matching was achieved by similarity calculation between the input sketches and clothing images. Finally, the retrieval results were output in descending order according to the similarity. Using the re-ranking optimization based on distance correlation coefficients, final results were obtained. Experimental results show that the method can effectively provide clothing retrieval results and significantly improve accuracy and robustness comparison with other approaches. The accuracy ratio of the algorithm is higher than 78.5%.

Key words: clothing retrieval, sketch-based image retrieval, saliency detection, feature matching

CLC Number: 

  • TP391.41

Fig.1

Process flow diagram of proposed method"

Fig.2

GF-HOG feature extraction map. (a) Salient edge; (b) Salient edge feature; (c) User sketch; (d) User sketch feature"

Fig.3

Sketch data. (a) Dress; (b) Overcoat; (c) Jersey; (d) Kilt; (e) Shorts; (f) Trousers"

Tab.1

Comparison of detection results"

方法 召回率 精确率 F值
MR 0.852 0.658 0.745
RC 0.596 0.921 0.728
本文方法 0.813 0.821 0.826

Fig.4

Results of salient detection. (a) Clothing image; (b) Salient detection; (c)Edge detection; (d) Salient edge"

Fig.5

Comparison with other methods"

Fig.6

Influence of codebook size"

Fig.7

Influence of size of parameter w"

Fig.8

Retrieval results without re-ranking. (a) Retrieve images; (b) Top 10 of search results"

Fig.9

Retrieval results with re-ranking. (a) Retrieve images; (b) Top 10 of search results"

Tab.2

Comparison with several algorithms on NDCG(K20)"

方法 NDCG(K20) 时间/s
文献[9] 0.658 3.18
文献[10] 0.391 5.56
HOG[13] 0.367 6.53
Edgel[18] 0.280 2.96
HLR[19] 0.584 3.23
RST-SHELO[20] 0.679 2.39
本文方法 0.785 2.42

Fig.10

Retrieval results of connected pants in different drawn styles; (a) Drawn style 1; (b) Top 5 of search results of drawn style 1; (c) Drawn style 2; (d) Top 5 of search results of drawn style 2; (e) Drawn style 3; (f) Top 5 of search results of drawn style 3"

[1] KIAPOUR M H, HAN X, LAZEBNIK S, et al. Where to buy it: matching street clothing photos in online shops[C]// IEEE International Conference on Computer Vision. Piscataway: IEEE Xplore, 2015: 3343-3351.
[2] LIU Z, LUO P, QIU S, et al. DeepFashion: powering robust clothes recognition and retrieval with rich annotations[C]// Computer Vision and Pattern Recognition. Piscataway: IEEE Xplore, 2016: 1096-1104.
[3] TSENG C H, HUNG S S, TSAY J J . An efficient garment visual search based on shape context[J]. Wseas Transactions on Computers, 2009,8(7):223-230.
[4] KONDO S, TOYOURA M, MAO X, et al. Sketch based skirt image retrieval[C]// Non Photorealistic Animation and Rendering. New York: ACM Digital Library, 2014: 11-16.
[5] 李东, 万贤福, 汪军 . 采用傅里叶描述子和支持向量机的服装款式识别方法[J]. 纺织学报, 2017,38(5):122-127.
LI Dong, WAN Xianfu, WANG Jun . Clothing style recongnition approach using Fourier descriptor and support vector machines[J]. Journal of Textile Research, 2017,38(5):122-127.
[6] 项明, 姚雪存, 崔振东 , 等. 利用相关反馈技术和形状特征的服装图像检索算法[J]. 纺织学报, 2014,35(2):94-98.
XIANG Ming, YAO Xuecun, CUI Zhendong , et al. Research on image retrieval algorithm by relevance feedback technology and shape features[J]. Journal of Textile Research, 2014,35(2):94-0.
[7] LI K, PANG K, SONG Y Z, et al. Fine-grained sketch-based image retrieval: the role of part-aware attri-butes[C]// Applications of Computer Vision. Piscataway: IEEE Xplore, 2016: 1-9.
[8] AZIZPOUR H, LAPTEV I. Object detection using strongly-supervised deformable part models[C]// European Conference on Computer Vision. Berlin: Springer-Verlag, 2014: 836-849.
[9] ZHANG Y, QIAN X, TAN X , et al. Sketch-based image retrieval by salient contour reinforcement[J]. IEEE Transactions on Multimedia, 2016,18(8):1604-1615.
[10] PEDRONETTE D C G, TORRES R D S . Exploiting clustering approaches for image re-ranking[J]. Journal of Visual Languages & Computing, 2011,22(6):453-466.
[11] ARBELÁEZ P, MAIRE M, FOWLKES C , et al. Contour detection and hierarchical image segment-ation[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2011,33(5):898-916.
[12] ACHANTA R, SHAJI A, SMITH K , et al. SLIC superpixels compared to state-of-the-art superpixel methods.[J]. IEEE Trans Pattern Anal Mach Intell, 2012,34(11):2274-2282.
doi: 10.1109/TPAMI.2012.120 pmid: 22641706
[13] DALAL N, TRIGGS B. Histograms of oriented gradients for human detection[C]// IEEE Computer Society. Piscataway: IEEE Xplore, 2005: 886-893.
[14] HU R, COLLOMOSSE J . A performance evaluation of gradient field HOG descriptor for sketch based image retrieval[J]. Computer Vision & Image Understanding Cviu, 2013,117(7):790-806.
[15] SZÉKELY G J, RIZZO M L, BAKIROV N K. Measuring and testing dependence by correlation of distances.[C]// ACM Symposium on Virtual Reality Software and Technology. New York: ACM Digital Library, 2007: 141-144.
[16] YANG C, ZHANG L, LU H, et al. Saliency detection via graph-based manifold ranking[C]// IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society. Piscataway: IEEE Xplore, 2013: 3166-3173.
[17] CHENG M M, ZHANG G X, MITRA N J, et al. Global contrast based salient region detection[C]// Computer Vision and Pattern Recognition. Piscataway: IEEE Xplore, 2011: 409-416.
[18] CAO Y, WANG C, ZHANG L, et al. Edgel index for large-scale sketch-based image search[C]// IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Xplore, 2011: 761-768.
[19] WANG S, ZHANG J, HAN T X , et al. Sketch-based image retrieval through hypojournal-driven object boundary selection with HLR descriptor[J]. IEEE Transactions on Multimedia, 2015,17(7):1045-1057.
[20] SAAVEDRA J M , RST-SHELO. Sketch-based image retrieval using sketch tokens and square root normalization[J]. Multimedia Tools & Applications, 2017,76(1):931-951.
[21] SEZGIN T M, STAHOVICH T, DAVIS R. Sketch based interfaces:early processing for sketch under-standing[C]// The Workshop on Perceptive User Interfaces. New York: ACM Digital Library, 2001: 1-8.
[1] WANG Hui, SUN Jie, DING Xiaojun, LONG Ying, ZOU Fengyuan. Comparison of feature extracting and matching methods for fabric patterns [J]. Journal of Textile Research, 2020, 41(04): 45-50.
[2] WANG Wensheng, LI Tianjian, RAN Yuchen, LU Ying, HUANG Min. Method for position detection of cheese yarn rod [J]. Journal of Textile Research, 2020, 41(03): 160-167.
[3] . Fabric pattern retrieval based on maximally stable extremal regions [J]. Journal of Textile Research, 2015, 36(10): 156-160.
[4] . Necktie pattern retrieval method based on image edge characteristics [J]. JOURNAL OF TEXTILE RESEARCH, 2015, 36(05): 123-126.
[5] LIU Jun;LI Zhong;HU Jueliang. 3D clothing fitting research based on feature matching [J]. JOURNAL OF TEXTILE RESEARCH, 2009, 30(01): 122-126.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
京ICP备14006648号
Copyright 2021 © Editorial office of Journal of Textile Research
Supported by Beijing Magtech Co.Ltd