We are living in an increasingly digital world.
If you’ve been paying attention, that shouldn’t come as any great surprise. Even prior to the pandemic, there was considerable interest in technologies such as virtual reality and augmented reality. Amidst lockdowns and quarantine measures, that interest has only increased exponentially.
By now, you’ve probably heard all the hype around the metaverse. Nebulously-defined at best and little more than a marketing buzzword at worst, there’s at least one concept at its core that demonstrates considerable promise — the notion of one day connecting with one another through virtual reality not simply as customizable avatars, but as vivid digital representations of our actual selves.
The idea of connected online environments populated by realistic, simulated, and dynamic avatars of our friends and family.
That may seem like something taken right out of the annals of science fiction. But it isn’t. On the contrary, it’s something Chinese tech company OPPO just made more feasible than ever through its AI-guided 3D body reconstruction technology.
Building Virtual Bodies
When it comes to 3D body reconstruction, one of the most significant challenges has historically involved the realistic portrayal of clothes, especially loose-fitting ones. The wireframe itself isn’t the issue — we’ve had a handle on realistically-proportioned virtual bodies for a while now. The problem is making the clothing respond like…well, actual clothes.
Without getting too technical, traditional methods for Digital Avatar reconstruction tend to result in clothing that looks blurred or simply wrong. The issue is that there’s a significant complexity gap between different body poses. Ensuring clothing realistically adjusts to these movements requires an inordinate amount of effort.
Until recently, there hasn’t really been a solution to this problem aside from ‘throw more engineers at it.’ And that’s where OPPO comes in. As demonstrated at the 2022 Computer Vision and Pattern Recognition Conference (opens in new tab), the company finally addressed what may well have been one of the longest-standing issues in this field.
As described in this paper (opens in new tab), OPPO’s process began with training a refined algorithm to understand how fabric shapes and deforms with movement. Initially, it focused on identifying and understanding small details such as logos, hemlines, and textures. The key to success here lay in treating these fabrics as a separate layer from an avatar’s skeleton motions — something traditional rendering methods still fail to do.
Concurrent with this training, the algorithm leverages a series of predefined, moveable points attached to the body template of an SMPL model (basically the standard 3D model of the human body (opens in new tab)). These points are then combined with structured local radiance fields, processes that simultaneously model appearance and geometry to generate more convincing visuals (opens in new tab).
Leveraging its new method, OPPO can generate three-dimensional, one-to-one dynamic models through simple video analysis — an innovation with some very exciting applications.
OPPO’s Virtual Avatar Creation Process: A Step-by-Step Guide
As we established earlier, digital human modeling is traditionally an extremely complex undertaking. Scanning the model is only the first step. Cost aside, success typically requires a series of intensive, manual processing and engineering tasks.
As seen in the video above, OPPO’s process basically throws those difficulties out the window.
1. First, the model’s body data is collected through OPPO’s Digital Avatar Lab.
2. Once the smartphone’s software has collected enough data about the subject, the AI begins to generate the 3D avatar.
3. A professional dancer acted as the human body template and the motions were recorded. The first stage in the rendering process involves the generation of a skeleton with a series of predefined anchor nodes.
4. The algorithm continues to analyze the skeleton’s movements to differentiate it from the shape and motion of any clothing applied to the avatar.
5. Finally, just a few hours later neural scene rendering generates a fully-clothed digital character with realistic clothing physics and motion.
Have no rhythm and embarrassed that you can’t dance? Your avatar won’t have that problem as you twirl and spin, your clothes flowing around you just like they would on the dance floor. While the algorithm is still in development in the lab, it will gradually become more efficient and more effective as it’s given more training data, eventually making it applicable to limitless real-world scenarios beyond dancing.
Visualizing a Digital World With OPPO
This is an enormous step in the direction of full-immersion virtual reality. As the technology at the foundation of OPPO’s algorithm improves, digital and physical will blend to an ever-greater extent. As you might expect, this has some very exciting applications.
First, shopping malls and other retail stores can provide more immersive online experiences through technology like virtual dressing rooms and food courts. This can be combined with realistic, lifelike personalized avatars that effectively allow people to interact with the virtual world on a level that was previously impossible. Said avatars would also allow friends and loved ones to see the ‘real significant ones’ and behave as what we want them to even over vast distances.
There are plenty of other applications, as well.
More convincing and dynamic characters based on actual people movements in video games coupled with reduced development costs. Remote work supported by full-immersion virtual reality meeting rooms. AI-guided personal training and fitness instruction.
Realistically, the sky’s the limit here.
Believe it or not, technology like this is nothing new from OPPO — the company has been a frontrunner in the field of artificial intelligence for quite a few years. Initially beginning its research into AI development back in 2015, it’s since deepened its focus considerably. Today, it operates the Institute of Intelligent Perception and Interaction, with dedicated research & development teams focused on a range of different AI technologies and applications.
Aside from 3D body reconstruction, OPPO’s technology has also been used in products such as real-time spatial AR generator CybeReal (opens in new tab), OPPO Air Glass (opens in new tab), and Omoji (opens in new tab), which uses facial recognition to create 3D emojis.
All this is to say that as exciting as this is, this innovation only scratches the surface of OPPO’s research. The company is devising new technologies with each passing day. And there’s little doubt that, by this time next year, it’ll have something even more exciting to share.
