Abstract:

Objective In recent years, the meta-universe concept based on virtual reality (VR) augmented reality and mixed reality technology has developed vigorously. Especially, virtual avatars based on users' personalized customization are widely used in virtual reality. From the perspective of technical composition, the realization of three-dimensional dressing of the human body has become more common. However, it is still a proposition worthy of further exploration to endow the user's head and facial features to the dressed human body and apply them in virtual reality and even metauniverse scenes.

Method By inputting the user's single frontal face image, detailed expression capture and animation (DECA) neural network was adopted to generate the corresponding 3-D head model with initial texture. The initial texture was then corrected by illumination, and the skin color of the main area and edge of the face was extracted by a given mask. The facial texture was in-painted twice to obtain a complete face texture. Using the face texture as a style image and using the original body texture of the dressed human body as a content image, style transfer was carried out by progressive attentional manifold alignment (PAMA) neural network.

Results In the analysis of the influencing factors of facial texture illumination correction, when the convolution kernel was 5×5, the difference of local brightness V₁ between the two sides was decreased by 23.1% (Fig. 2), which shows that illumination correction can reduce the brightness difference of the user's facial texture and improve the illumination uniformity of the face with the best scale factor being 5. In the analysis of automatic facial texture restoration, when the weight coefficient was 0.4, the head and neck texture was closest to the facial skin color. When repairing, the combination of two methods was shown to be the best (Fig. 12(c)). The results of texture restoration for different users using the method were presented (Fig. 13), where it can be seen that after the original texture was restored and completed, the texture of the user's head and face model was similar to the neck skin color in the original three-dimensional dressed mannequin with natural boundary fusion, and the original facial information of the user was retained. The comparison between the results of this method and other head and face texture restoration methods based on a single image was shown (Fig. 14). It can be seen that the generated results of this method have the highest similarity with users, and it took the shortest time (8 s per person) under the same configuration. In the analysis of style transfer of body texture, using the PAMA style transfer network could make the skin texture of the body texture in virtual reality similar to that of the user's face texture (Fig. 15). In the application of VR, the rendering result of the user's dressed human body was displayed in various virtual scenes (Fig. 17). It can be seen that the model can present real light and shade effects in different indoor and outdoor lighting environments, demonstrating the practicability of this method in VR scenes.

Conclusion Based on the application of virtual reality, this paper presented a study on the generation of personalized head and face textures of dressed human body with a proposed technique of head and face texture fusion and restoration. By inputting a single frontal face image, a complete head and face texture and corresponding body texture can be generated, which solves the problem of natural fusion of head and face texture and body texture skin color from different sources and can meet the virtual fitting applications in various VR environments.

Key words: users' personalized customization, face texture, illumination correction, texture repairing, neural style transfer, virtual reality