Unity buying skin sim tech company for fancy faces and flappy flesh
Tech seen in Hellblade 2 and Spider-Man: Miles Morales
Two months after announcing they're buying Weta Digital, the CGI studio behind King Of The Hoops and that, Unity are now buying virtual skin. Today the makers of the Unity engine announced the acquisition of Ziva Dynamics, a company whose tech specialises in simulating skin and meat to make fancy facial animations and meaty muscle and such. Their tech powered the flappy flesh of the troll in a recent Hellblade 2 trailer, and has been seen on PlayStations in Spider-Man: Miles Morales.
No, that face still makes me a bit unhappy to look at.
Ziva Dynamics have made software which models muscles, fat, and skin to create more lifelike characters with fancy faces and all that meat. It's been used in films including The Meg but isn't only for pre-rendered 3D. Along with the flappy fleshtroll seen in Hellblade 2's Game Awards trailer, Ziva tech powers the muscle deformation added to PlayStation-exclusive swinger Spider-Man: Miles Morales in a March 2021 patch.
Unity say they want "to further democratise Ziva's best-in-class tools to allow artists, regardless of skill level, to easily and quickly create digital characters unlike ever before". They also add that Ziva's experience can help them "accelerate the process of bringing Weta tools to real-time 3D through the cloud with the power of machine learning".
If you're into gameguts, you might enjoy Ziva's marketing explanation of their work on Hellbade's horrible troll. Starting as a simulation requiring hours of rendering time on a server farm, it ended up as animations running in real time:
"The Ziva Dynamics artists began by building the troll in Ziva VFX, the soft tissue simulation software. This process was expedited by a combination of Ziva's Anatomy Transfer tools and their proprietary generic male anatomy simulation. Ziva then added the troll's torn, hanging skin flaps to the simulation as a coupled skin pass. At this point, the asset was complete with excitable muscles, jiggling fat, and wrinkling skin, all while being over 40ft tall in world-space so accurate gravity would influence on all of those anatomy layers.
"The tradeoff for the gigantic film-quality results, however, was a render speed of 6 hours per 50 frames, leading to hundreds of hours of baking on a 15 machine AWS cluster. So, to transform this large film asset into a real-time game character, Ziva fed over 12GB of performance capture data along with the high-quality Ziva VFX simulation into their Ziva Real-Time Trainer. This technology used machine learning to train the troll asset to perform all of the animations along with novel poses in real-time while maintaining the rich dynamics of the original simulation. As a result, the final ML troll body performed at a fully-interactive frame rate of under 3 milliseconds per frame in Unreal Engine 4.26 and was ready to be handed back to the innovation-loving Ninja Theory team."
My personal hope is that this technology will be seized by walking simulator developers to revel in flesh. That's the game.
In less welcome news, Waypoint reported today that Unity have been making a dogfight simulator for the US Air Force. As the site previously reported, some Unity staff have become concerned that their work might end up part of something which goes against their morals, and they might not even know they'd contributed to it.
Then, and now, Unity repeated that when they form partnerships, "we ask ourselves if the specific engagement violates one of our key principles: that it does not directly involve the loss of life, harm of the planet, or a person's right to equity and inclusion. We don't invest our time and energy with customers to undermine these principles."