Abstract:

Objective Virtual fitting provides users with a digital and interactive fashion fitting experience and meets the requirements for garment customization in the fashion industry by using machine vision, artificial intelligence and other technologies. It has attracted keen attention from international brands and researchers. However, due to the influence of various posture, occlusion and interruption in non-fitting area, the existing virtual fitting methods still have problems, such as distortion, blurring and low accuracy. In order to overcome these problems, this paper proposed a high-precision virtual fitting model named as C-CGAN based on texture feature learning.

Method A garment reconstruction network based on the idea of CGAN was proposed, which used the garment mask positioning and garment texture constraints to learn intelligently the garment reconstruction model under various postures. The encoder-decoder network was utilized to fuse the reconstructed garment and character features. In addition, a variety of comprehensive loss functions were employed to optimize the network performance. A rich texture dataset was eventually constructed based on the international virtual fitting dataset, followed by the development of a garment fitting system in PyTorch environment and its performance evaluation.

Results The results of C-CGAN showed more significant FID (Fréchet distance) and IS (initial score) optimization effect than that of the newly reported VITON and CP-VTON statistical metrics (Tab.2). However, the PSNR (peak signal to noise ratio) accuracy of CP-VTON was still low, which means it had a lot of distortion. Compared with CP-VTON, in the case of comparable IS, the FID of C-CGAN was reduced by about 11%, the SSIM (structural similarity) is increased by about 25%, and the PSNR was increased by about 78%. Therefore, the performance metrics of this network had significant advantages. In order to compare the visual fitting effect, CP-VTON and C-CGAN were both adopted to synthesize the texture of the model's original tops on the test dataset for comparison of the subjective visual similarity between the virtual fitting results and the real sample in dataset. The comparison results of the virtual fitting (Fig.7) in 9 difficult scenes (Tab.1) showed that CP-VTON was prone to large deformation distortion for some complex textures, such as stripes and wave points, and the model's arm was distorted when occluded. In contrast, C-CGAN was shown to be able to suppress effectively the interference of occlusion and garment texture, truly and exquisitely preserve the details of characters and texture, and had a higher similarity with real samples. Furthermore, in order to verify the applicability of this method in practical applications, a model in test dataset was selected whose original top's texture is light pinstripe. There were ups and downs and pleats at the model's front and waist, respectively, relating to her posture. The virtual garment replacement preview results of seven textures (Fig.8) showed that textured details and features varied on the model's chest and waist corresponding to the posture, such as the fold changes of pure color, the density changes of the wave point and the waveform variation of the stripe. In addition, C-CGAN was shown to preserve well the model characteristics of models and clothing characteristics of other areas.

Conclusion This paper presented extensive qualitative and quantitative evaluations on the C-CGAN method. The statistical metrics on the test dataset show that the similarity between the C-CGAN virtual fitting results and the real samples is higher, the accuracy is higher, and the distortion is smaller. The subjective visual comparison results of virtual fitting show that C-CGAN has better adaptability to difficult fitting scenes such as stripes, wave points and occlusion, and the reconstructed texture is more natural and delicate, with high matching sense of human posture and good adaptability. The virtual garment replacement preview test results show that C-CGAN can generate texture deformation adapted to human posture for color, stripe and wave point, and the generated image is clear. C-CGAN can provide a realistic virtual fitting experience, which can be widely used in digital fashion application scenarios such as interactive texture reloading and garment assisted design.

Key words: conditional generative adversarial network, encoder-decoder network, positioning and reconstruction, virtual fitting, garment customization