Watch this AI learn to drive inside Grand Theft Auto V

You might have heard that some folks making self-driving cars used Grand Theft Auto V to help train their AIs. You might have wondered what that looks like. You… might not learn that watching this Twitch livestream of an AI learning to drive in GTA V. The compubrain, named Charles, is less advanced, though Charles sounds fancy in creator Harrison ‘Sentdex’ Kinsley’s description: “a convolutional neural network that learns to drive through deep learning.” Charles often ends up atop highway barriers, ramming walls, or battered in ludicrous police chases. This makes Charles no less fun to watch.

Kinsley, a man robots will burn for trapping their forebears inside a crimeworld for education, explains:

“At the moment, Charles learns and takes all actions based on single frames at a time, and bases his decisions on just pixel data. Charles only sees exactly what you see.

“In time, I intend to give Charles some short-term memory to hopefully improve his driving.”

Like with Brent Watanabe’s wonderful San Andreas Deer Cam (which is still my favourite botstream), I’m coming to recognise behaviours and quirks in how Charles sees the world, and enjoying simply watching Charles’s weird adventures. Charles is living on my second screen right now, pootling around and ploughing through pedestrians.

Occasionally Charles will seem almost like a real player, swerving to side-swipe a scooter then do a sweet flip or repeatedly driving over the same man.

Oh, Charles has just driven into the sea.

If you want to know more about how Charles works, check out this brief explanation of how Charles sees and Kinsley’s tutorial on building self-driving cars with the programming language Python. Yes, a magic snake hisses the words of life inside Charles’s brain.


  1. DuncUK says:

    Hah. I just logged onto the Twitch stream to find the AI repeatedly crashing into rocks on the beach, before giving up, running over a few sunbathers and then driving directly into the sea, under a hail of police bullets.

  2. LTK says:

    I look forward to this being incorporated in the next generation of self-driving cars. “Police pursuit detected. Taking evasive action.” Followed by a sick jump off a ramp.

    • poliovaccine says:

      Agreed. All cars in the future will have a gearshift with first, second, third, fourth, sometimes fifth, reverse… and Charles.

    • April March says:

      Isn’t that the plot of Knight Rider?

  3. Jane Doe says:

    Problem with doing that in GTA is that nobody in GTA drives anywhere close to real-life driving. I tried that once and immediatly got bumped by the car behind me, because I dared to stop at a red light. In GTA, only the first car in lane is allowed to stop at red lights.

    • poliovaccine says:

      Hah, you’re right.. they should’ve done it in Mafia. Remember that? You could even get speeding tickets, and pay them like a normal person. Why was that so much fun??

  4. BockoPower says:

    Charles reminds me of me when I first played GTAV. For a game with big focus on driving it has very weird steering. So I blame the game for the AI’s ridiculous mistakes!

  5. Catchcart says:

    I know a bit of Python so I thought to myself, hey, that sounds like a fun challenge. Let’s do Spelunky! Then I found out that that was a challenge for major CS eggheads back in 2015. Guess I’ll stick to just doing the dailies, then…

  6. Premium User Badge

    Earl-Grey says:

    Who’s livestreaming my wife’s car?!


  7. Otterley says:

    Mussst sssidessswipe! Ssspin, ssslay!

  8. fish99 says:

    Decision making based off a single frame seems to be a huge limitation, but hopefully they keep developing it.