If you click on a link and make a purchase we may receive a small commission. Learn more.

Facebook researchers are turning videos into videogames

Spooky tennis

Facebook, a company with a fabulous track record in using artificial intelligence to make the world a better place, have figured out how to turn telly people into videogame people. The researchers' Vid2game game project lets them create a controllable avatar from live-action footage, as demonstrated below with a tennis player.

The resulting animations aren't super convincing, at least at this stage. That doesn't stop Facebook boasting that their work "paves the way for new types of realistic and personalized games, which can be casually created from everyday videos."

As well as using a joystick to control the character's movements, the software also allows the background to be swapped. Again, this is all rather limited - we're a long way from plunging such avatars into 3D worlds.

It works like this:

"The method is based on two networks. The first network maps a current pose, and a single-instance control signal to the next pose. The second network maps the current pose, the new pose, and a given background, to an output frame. Both networks include multiple novelties that enable high-quality performance. This is demonstrated on multiple characters extracted from various videos of dancers and athletes."

And looks like this:

Anyone with basic software skills has been able to get up to video trickery like this for a while now, creating fake videos from footage of actual people. They've never been able to puppet them in real time, though.

If you really want to know how it works and aren't intimidated by phrases like "autoregressive models" and "ReLU activations", the full research paper is here.

I've been trying to think about exactly what the implications of this are, beyond the obviously horrific upcoming prevalence of playable YouTube influencers. I do think any progress towards making easily doctored video is course for concern. I'm not saying we should attempt to curtail such research, as that's likely to be counterproductive - and hey, I'm enough of a narcissist to find the idea of playing as my literal self in a videogame appealing. But it seems to me that a well-placed fake video at a critical juncture, like say, an upcoming election or referendum, could easily contribute to steering society in an unwelcome direction.

It's worth thinking about the impact of widely spread fake videos more broadly, too. You'll never be able to fully trust your eyes, and people might always be able to claim real footage of them is merely a fiction. What do we do when video evidence doesn't stand up in court?

Tagged With

Join the Rock Paper Shotgun supporter program

Sign up today and get access to more articles like these, an ad-free reading experience, free gifts, and help us create more great writing about PC games.

See more information

Comments

We love having a friendly, positive and constructive community - you lot are great - and we want to keep it like that. Our main commenting rule is "be excellent to each other". Please see our code of conduct, where you can find out what "be excellent" means. TL;DR? Respect others, think before you post, and be prepared for puns.

More News

Latest Articles

We've been talking, and we think that you should wear clothes

Total coincidence, but we sell some clothes

Rock Paper Shotgun Merch