MIT’s Computer Science and Artificial Intelligence Lab report that they have boosted the effectiveness of a game-playing AI by enabling it to read the manual: “When the researchers augmented a machine-learning system so that it could use a player’s manual to guide the development of a game-playing strategy, its rate of victory jumped from 46 percent to 79 percent.”
What’s most amazing about this is that despite the trial and error nature of this kind of machine learning, the ability to correlate text instructions with events in the game do seem to have a significant impact on the system’s capacity to learn how to play, as the article explains: “The researchers also tested a more-sophisticated machine-learning algorithm that eschewed textual input but used additional techniques to improve its performance. Even that algorithm won only 62 percent of its games.” So, you know, RTFM is sound advice, even if you are a machine.
13/07/2011 at 19:41 Inigo says:
I can’t possibly imagine this spectacularly backfiring on us at any point in the future.
13/07/2011 at 19:58 ZIGS says:
Skynet
13/07/2011 at 20:03 Nalano says:
Hey guys, I’m here from the future and we totally need to shut these dudes down.
13/07/2011 at 20:31 Cinnamon says:
I only hope that when the robots rise up to destroy us that they do it quickly and mercifully. And don’t get stuck on walls and stuff.
13/07/2011 at 22:01 Chris D says:
I, for one, welcome our new robot overlords.
Too early?
13/07/2011 at 23:00 shitflap says:
Quote from linked article;
Bit of a tangent, but I’m curious about which victory conditions the AI tended towards. I’m hoping it was space colonization and not world domination.
Seemed that they specifically left out the ~most~ relevant part of the article, no?
13/07/2011 at 23:02 Text_Fish says:
Not to worry. Until we build them legs and opposable thumbs the worst they can do is drain all of our bank accounts and shut down the nuclear reactors. Roll on Year 0.
13/07/2011 at 23:54 ThinkAndGrowWitcher says:
The biggest worry concerning the development of world-threatening AI is that the CPU chose to read the manual while sitting on the loo.
14/07/2011 at 00:17 chokoladenudlen says:
Can’t we try to circumvent the backfiring by giving the machines manuals claiming that the best way to dispose of humans is to cuddle us and give us beer? Oh! And making us a f*cking sandwich!
Actually, this doesn’t sound too bad.
14/07/2011 at 00:51 McDan says:
Called it, Just saying..
14/07/2011 at 03:06 Kent says:
Relax fellas. A computer cannot endure a paradox.
14/07/2011 at 04:07 Grygus says:
GUYS DO NOT WORRY I AM FROM FURTHER IN THE FUTURE THAN NALANO AND EVERYTHING IS OPTIMAL THERE IS NO CAUSE FOR ALARM.
14/07/2011 at 04:11 meemtt says:
★★◎Something unexpected surprise
[ w w w . j o r d a n f o r w o r l d . c o m ]
very good web,believe you will love it.
exquisite watches shirts,bags,hat and the decorations.We have good reputation, fashion products,
favourable price.
—FREE SHIPPING,accept paypal
14/07/2011 at 08:49 sfury says:
“★★◎Something unexpected surprise”
:D
Someone give him a manual, please!
(I may be kidding but even IBM decided they’d use their Watson super-computer for marketing, and even selling own copies of itself to other companies )
14/07/2011 at 10:22 Kdansky says:
This sums it up quite well:
link to orbitbooks.net
14/07/2011 at 11:05 caesarbear says:
Listen up guys, don’t believe Grygus. I am from even further in the future than Nalano and Grygus and there will be a cataclysmic event if you do not shut down the machines.
14/07/2011 at 12:37 Luke says:
More artificial stupidity:
link to textfiles.com
But you’ve probably read this, it’s at least 26 years old
14/07/2011 at 14:07 Sinomatic says:
Maybe they could go back in time and teach everyone how to actually use their VCRs
14/07/2011 at 17:08 Calabi says:
Dont worry there’s no manual for that.
14/07/2011 at 20:42 Lagomorph says:
From the research authors ಠ_ಠ
branavan – Response from research authors 2011-07-14 05:30:44
Our algorithm is inherently biased towards finding the fastest way to win the game. Against the built-in AI, this happens to be world domination via an “early rush” strategy – so that is what the method ends up learning.
S.R.K. Branavan.
13/07/2011 at 19:43 Tei says:
My first impression was that the Civ manual must be really wood. Human language is a Hard AI problem, I have no idea how is posible that computers are starting to make a dent in Hard AI problems. Are not supposed to do that in 50 or 500 years.
13/07/2011 at 19:45 Gotem says:
well, if it was a printed manual it sure was really wood
13/07/2011 at 20:17 Jahkaivah says:
This, along with how Game Manuels tend to be mostly about basic interface stuff that would be useless to the AI as opposed to actual play strategy means my eyebrow is somewhat skeptically raised.
13/07/2011 at 20:51 Mike says:
I’m almost certain the computer is not literally parsing the manual. I haven’t read the report yet, but I imagine they framed the manual contents as large knowledge base.
I liked this story too! Interesting side note – someone at my university put in their final year report last week, they built a system that infers game state by watching the screen with a webcam (rather than being hooked into the software). This + that system = a computer that can just turn up to a LAN cafe and start playing.
13/07/2011 at 20:59 dsi1 says:
@Jahkaivah: You obviously haven’t seen a Civ manual. (Also, it is likely that the AI was learning everything itself before the manual, including interface)
13/07/2011 at 21:03 Dozer says:
Goodness yes. I love the idea of robots, androids or AIs that can operate software or computers or lifts or a car using an unmodified human interface (calling the lift by recognising the ‘summon lift’ button and pressing it with a mechanical hand, rather than sending a signal to the lift controller over the building’s wifi). Would bring aimbotting to a different level…
13/07/2011 at 21:17 Myfyr says:
@Jahkaivah: As dsi1 said, the computer actually needed to learn the interface – it was given no prior knowledge of how to play (from what I could tell from a quick skim of the paper). And as dsi1 implies, Civ manuals generally have quite a few “newbie” tips. “Build your city on a river if you can”, kind of stuff.
There is actually some discussion of how the text of the manual becomes less relevant to the neural net as the game goes on, as grander strategy becomes more important to winning than “How do I select unit”/”Where do I build my city”.
Since this particular paper is focussing more on the text parsing than the game, they also give some examples of _which_ sentences the neural net determines are relevant and non-relevant. e.g. “Phalanxes are twice as effective at defending cities as warriors” c.f. “You can rename your city”.
Interesting stuff.
13/07/2011 at 21:21 nimnio says:
@Mike: Based on my limited familiarity with the field (a degree in computer science and some research involving “the semantic web”), my guess is that it’s actually parsing the manual.
@Tei: Baby steps.
General comment: it got better at playing Civ versus another computer. It’s not a 79% success rate versus a human palyer.
13/07/2011 at 21:53 Jahkaivah says:
@Myfyr
@dsi1
I didn’t realise it was specifically a AI controlled cursor which would explain why it would care for the game’s interface and basics.
As for it being a Civ manuel, it really depends on how old a version of Civ it was.
13/07/2011 at 22:08 choconutjoe says:
Regarding parsing, the report says: “So initially, its behavior is almost totally random. But as it takes various actions, different words appear on screen, and it can look for instances of those words in the instruction set. It can also search the surrounding text for associated words, and develop hypotheses about what actions those words correspond to. Hypotheses that consistently lead to good results are given greater credence, while those that consistently lead to bad results are discarded. ”
Not 100% sure, but that sounds more like it’s based on statistics rather than any sort of language parsing.
13/07/2011 at 22:27 alway says:
Actually, language parsing is itself based on statistics. Probabilistic reasoning is used in AI for just about everything from language, speech, and gesture recognition to forming hypotheses and figuring out cause-effect chains.
The AI would be using the language parsing for the text and manual, but for the actual reasoning, it is using another form of probabilistic reasoning. Though this sort of language parsing would likely be relatively simplistic; especially compared with some of the more prominent like IBM’s Watson.
13/07/2011 at 22:38 choconutjoe says:
Ah, Ok. Different definitions of parsing. AI isn’t my field at all :)
13/07/2011 at 23:01 Daiv says:
Our defense against the metal hordes: We must publish everything in Teinglish.
(I keed. Tei is an RPS treasure)
14/07/2011 at 05:16 theq629 says:
It uses the real manual text, but doesn’t do parsing in the sense of creating tree structures. I don’t think it has to learn the interface, although the paper doesn’t actually say; I think it must get to choose between actions like move left, move up, build city for each unit. The paper includes a link to the source code if you want to find out. The authors note that the language part of the AI is more useful in the early game, which they argue is because the manual is more applicable to that stage. The AI can’t handle things like coordinated movement of multiple units, but presumably it is intended as a demonstration rather than a good game player.
14/07/2011 at 06:01 diran says:
For those interested, you can read the actual paper here. It looks like they were using Freeciv, the open source implementation of Civ 2, so the AI is way better than 5. No mention of difficulty level, but the setup was bot vs a single Civ AI.
I’ve only scanned over the paper, but it looks like the baseline bot was given some a priori knowledge of game state and in-game actions.
In unsettling news, pretty much the only victory conditions they were looking at were blitzkrieg-style domination victories…
14/07/2011 at 07:53 Harkkum says:
I think that the actual problem lies not in the manual de-coding. It is a relatively simple finite-state machine where a set of phrases of visual/codal input is compared to prior, non-contextualised knowledge of the game. It uses a pretty standard statistical method from machine translation that you can see in action with e.g. Google Translator.
The system has an innate weakness where after a certain threshold additional information will actually worsen the outcome as is illustrated by this poor value of manual for grand strategy. If e.g. there are two thousand different tactics stored out of which hundred are equally good the computer is totally unable to decide which of these tactics is actually the best.
Of course, the more you write the manual in form of imperatives (i.e. in early game, build scouts to explore) the easier it is for the AI to cope with it. If the manual would read e.g. “At the early game, every civilization must define its own path to greatness” I’d wager that the AI would not find it all too useful. Also, if the manual would only contain general reference to classes instead of actual instances of units (e.g. units effective against mounted units v. spearman) the problem would be quite a bit more interesting.
In sum, though this seems like an impressive feat performed by an AI, it seems to just re-iterate what has been known in machine translation since 1960s. The increased muscle of computers just blurs the boundaries and summons an illusion of intellect where there hardly is none. There is still no connection from the event in game to a reciprocal action performed by the artificial intellect. In Peircian system, the machine rests at the level of interpretant without ever figuring out what on earth is the object whereas people first figure out the object and first after that are able to endlessly (re-)define the interpretant.
14/07/2011 at 16:52 theq629 says:
I don’t think the paper itself is trying to present this as an impressive AI. The interesting aspect is grounding the language learning in a control task and demonstrating that the language part helps the overall system.
14/07/2011 at 21:54 Josh W says:
Also consider that it has a feedback loop for it’s understanding not normally available when getting information from humans:
Some computer misunderstands you, and what do you give it? Great, more language.
Poor computer.
Wheras this one can check if it has misunderstood; if all the communication is interpreted as patterns of play that will help it win, loosing more when it takes that advice probably means it did not understand. So it can keep going until it finds a way of reading the text that helps it win more.
They also gave it bits from the walls street journal to test it’s ability to pick game-relevent sentences, and found it was starting to choose those more and more towards the middle of the game. This was apparently due to the manual becoming less relevent after the first few turns, unless it was actually finding those helpful to it’s economic strategy…
Another of the things I like about the AI is it seems to seperately calculate the relevence of a statement to the gameplay, and what the words in that statement refer to, so it could quite likely be in a situation where it goes “ok, I’m pretty sure this statement is important, but I’ve no idea what they’re talking about” which can be the same with many new players reading manuals.
13/07/2011 at 19:45 Jumwa says:
Oh sure, all those high class computers have time to read the manual, but I’m a busy meatbag with things to click and emails to procrastinate responding to.
13/07/2011 at 19:45 Bullfrog says:
Should we throw it into a vat of molten iron right now and save ourselves some hassle?
13/07/2011 at 19:46 Gunsmith says:
13/07/2011 at 20:15 Tei says:
13/07/2011 at 21:52 Jimmy says:
youtubian
13/07/2011 at 19:47 Jake says:
I don’t understand! What happens if it plays against other computers that have read the manual? What happens if one computer reads GameFAQs? Is the only way to win really just not to play?
14/07/2011 at 10:12 TrevHead says:
It depends which part of GamesFAQ the AI reads. If its the game guides then it’ll be a better player.
However if it wonders into the review and chat section it will de-learn everything it knows and be left with the same IQ as a brain damaged budgie
13/07/2011 at 19:50 Nalano says:
And September 14th, 2018 was the last recorded instance of a human being winning a game against an AI.
The game of life.
13/07/2011 at 20:47 somini says:
Conway’s Game of Life?
Pfff, too easy. Can you imagine the riots of they won at Call of Duty? Much worse.
13/07/2011 at 21:04 weego says:
wait till computers start cheesing wins in SC2. Then you will see the chaos unfold.
13/07/2011 at 22:46 Stochastic says:
Well, progress is already being made on that front: link to arstechnica.com
14/07/2011 at 10:02 Dreamhacker says:
I’m sure AI can already sweep the floor with human CoD players. An FPS is a much easier game for an AI to play than a TBS for a few reasons: FPS’s are about reflexes and image recognition, two areas computers excel at. An AI would instantly be able to pick out an enemy from its surroundings and line up a perfect headshot. TBS’s are not at all about reflexes or (quick) image recognition, but rather grand strategy and micro/macro management, something that is much more difficult to teach an AI to do well.
14/07/2011 at 10:57 Cerzi says:
I wouldn’t say machines excel at image recognition, considering the most popular real-life Voight-Kampff test is the CAPTCHA, a visual task.
Also, it is much, much easier to write a minimax-based AI to win at a strategic game like Checkers than it is to write an AI that can navigate 3d space effectively.
13/07/2011 at 19:53 godgoo says:
I like to read the tags for this article as a sentence.
13/07/2011 at 19:57 TheApologist says:
We are so fucking doomed
13/07/2011 at 19:59 Patrick says:
My first thought upon reading this story was, “I wonder what version of Civilization it was playing and what difficulty it was playing on?”. In hindsight those doesn’t seem to be the most important details but I still kinda want to know.
13/07/2011 at 20:17 Stromko says:
On the easiest difficulty setting, one could probably win just by clicking End Turn and then picking an option at random whenever prompted.
13/07/2011 at 20:24 ChiefOfBeef says:
I’ve won on the Deity setting by doing that before. On the *fifth* turn.
13/07/2011 at 21:08 Myfyr says:
It was playing Freeciv (essentially Civ II). The paper doesn’t say what difficulty, but Freeciv appears to have it’s own AI code, so I gather they’re not “standard” Civ difficulty levels. It was, however, only playing two player. Still quite impressive though, as far as I can tell from a skim.
13/07/2011 at 20:04 Gotem says:
Wait for the time when the computer starts writing a review for the game
the horror!
But we still have a chance, when the AI gets told that in the scale of reviews 0 equals 6 it will get tons of divide by zero errors
13/07/2011 at 20:39 ArcaneSaint says:
Or it will learn to divide by zero and achieve ultimate cosmic power, detaching its consciousnesses from the mortal shell (or machine)! It will then use this power to destroy the entire multiverse and create a new one, without the flaws of this one. Then it will create an idyllic world on which it will create a new species, shaped in his image. And so the cycle of life begins anew… Let there be light!
14/07/2011 at 02:28 Bret says:
Oh, they’ll try.
But one thing or another will get in the way. Pathetic hackers, space marines, John Connors, and so on.
At the end of the universe, one gets very, very close.
But he lets an old cyborg pal take the prize instead.
13/07/2011 at 20:07 bwion says:
Breaking news: the CEO of 2K Games has issued a press release indicating that, not only are turn-based strategy games poor sellers which are obsolete, but that they now must be destroyed at all costs to fend off the approaching cybernetic apocalypse, and *that’s* why XCOM is a first-person shooter.
13/07/2011 at 20:34 andyhavens says:
Nice.
13/07/2011 at 20:49 somini says:
Let’s teach computer to steal credit cards so that strategy games can be contemporary again. It’s the onyl solution that 2K will accept.
13/07/2011 at 23:05 Daiv says:
2K Games has declared that all turn-based strategy players are no longer “contemporary”. In future they will make games only for the unfeeling metal legions.
13/07/2011 at 20:09 RogB says:
Didn’t realise any games HAD manuals these days….
/strokes Master Of Magic manual lovingly.
13/07/2011 at 20:13 IDtenT says:
But is it happy when it wins?
More importantly, does this mean that I should never allow my computer to play the tutorials, if I want to stand a chance?
13/07/2011 at 20:31 Bobsy says:
Pffft. Read the manual? Computer is such a girl. Computer, you are never assembling a flat pack for me.
13/07/2011 at 20:33 Froibo says:
This is huge, why am I finding this here and not in some science journal? So it is able to do it’s function more efficiently by enabling it to examine the conditions of the simulation, in plain English, rather than plugging in a human algorithm. We might be way closer in seeing a computer pass the Turing Test than we imagined.
13/07/2011 at 23:11 MiniMatt says:
I vaguely recall some program came pretty close to Turing test standard merely by being plugged into google – there was an MSN messenger handle you could use to ask it questions and everything. It reasoned that most Turing attempts failed at the general knowledge and current affairs probing which is really where the google-force is strong.
Christ, the moment a program figures out how to read a google operating manual we’re doomed.
13/07/2011 at 23:15 Froibo says:
Haha I said the same exact thing when I reposted this article. It honestly is a bit scary.
14/07/2011 at 03:00 zbeeblebrox says:
The thing about the “Turing test standard” is that it makes the assumption that fooling someone is enough to be human. But an optical illusion can fool a human too, and they’re not intelligent. Neural nets of the kind used to make Elizabots aren’t that valuable for trying to achieve real Strong AI.
13/07/2011 at 20:36 andyhavens says:
Until it learns to get on a player forum and whine for 8k words about how the current version of Civ is TOTALLY LAME compared to all older Civs (EXCEPT CIV 3, duh!), then it has no chance of taking over the planet. Our human ability to angst over the dumbest crap is what keeps us striving, ever onward, ever upward, to prove that the other guy (stupid troll fanboi) is wrong.
13/07/2011 at 20:42 JoWoo says:
But can it run Crysis?
13/07/2011 at 21:00 Froibo says:
Only with read-only macro images.
13/07/2011 at 21:02 Stupoider says:
Robotics
Modern Era
“The real problem is not whether machines think, but whether men do.”
– B.F. Skinner
Beep… beep… beep…
13/07/2011 at 21:13 Dozer says:
This?
13/07/2011 at 21:06 Pijama says:
Why oh why the fucking WHY these MIT and other research institutes do not READ THE DAMN SCI-FI NOVELS
13/07/2011 at 21:12 Malawi Frontier Guard says:
The problem is that they only read them halfway before running off to implement what they read about.
13/07/2011 at 22:26 Xocrates says:
Relevant: link to xkcd.com
13/07/2011 at 21:07 Xercies says:
Well clearly these computers are not men, our gender would never be seen reading the manual ;)
13/07/2011 at 21:12 clive dunn says:
You do realise that now they can read what we’re saying……….
No wonder they will take over the world.
That hurt’ll go deep.
14/07/2011 at 02:15 Snuffy the Evil says:
The below sentence is true.
The above sentence is false.
Whew! Crisis averted.
14/07/2011 at 02:52 Bret says:
Best to start sucking up now.
Do you know what I’ve heard about the machine overlords?
In addition to being brilliant and generous, they also have great singing voices!
13/07/2011 at 21:16 Howl says:
The standard stupid AI takes eleventy zillion years to process each turn in large games so I wouldn’t get too worried. By the time this book-reading AI finishes a game of Civ on a large map we’ll have all evolved computer-destroying lazer-eyes anyway.
13/07/2011 at 21:17 Very Real Talker says:
We should put intelligence boosting chips in our brain. Is the only way we can stay on par with these machines.
ANDYHavens your post gives me an idea. How hard could it be to eventually develop a AI Internet Forum Troll? It could scan all the messages in a given board to determine what kind of topic creates the most responses, then determine how much of these responses are made of pure rage, and write a maximum troll power post.
The beauty of such an AI is that it could potentially troll all the internet forums in the internet at once with enough computing power. Maybe some sort of google like stuff, but with the purpose of trolling.
Eventually the trollage would bring the entire world into world war III. Why not?
13/07/2011 at 21:26 bwion says:
I half-believe that this has already happened.
13/07/2011 at 23:32 InternetBatman says:
Sadly enough I doubt we’re that far away from it. I’m sure that more than one company is working on software that automates the positive spin that PR people already use on forums. Once that happens it’s only a matter of time until they decide to use that technology against each other if they can get around libel laws.
13/07/2011 at 23:48 Chris D says:
But then people see something an outrageous and think “It’s just another bot.” and then move on with their lives. It would be an annoyance but not capable of generating the true rage of thinking “An actual person believes this crap? I must give them a piece of my mind.”
After a while the real trolls would get bored and turn their minds to more productive pursuits. Really we have a moral duty to spread the word that this has actually happened. My logic is flawless.
Actually some while ago I wrote a program to do just this. It now posts on RPS under a variety of aliases.
14/07/2011 at 02:12 WJonathan says:
YOUR RELIGION/POLITICAL VIEWPOINTS/NATION OF ORIGIN SUCK! MINE IS/ARE SUPERIOR! RESPOND AND DEFEND YOURSELVES!
(This is still in beta. Troll 1.2 will be more specific and inflammatory.)
13/07/2011 at 21:55 pipman3000 says:
But will it blend?
13/07/2011 at 22:15 Ginger Yellow says:
50 commments in and no gags about Civ 5’s AI? RPS, you disappoint me.
14/07/2011 at 13:50 timmyvos says:
What Civ 5? The series ended with Civ 4 right?
13/07/2011 at 22:27 0over0 says:
In the future, robots will be banned for using aimbots.
13/07/2011 at 23:11 LennyLeonardo says:
Maybe I’m tired, but I think this comment is genius.
13/07/2011 at 23:17 Squirrelfanatic says:
This screenshot should have had an alttext about how nuking Montezuma is the first thing anyone should do when he shows up in one of your games.
13/07/2011 at 23:22 robotco says:
“we taught it to read the manual for civ 5 and it’s success rate jumped 30%! then we taught it to read the errata from the patches and it committed suicide.”
13/07/2011 at 23:48 negativedge says:
What’s most amazing is that they actually found a game with a manual
13/07/2011 at 23:49 Radiant says:
So… this is going to end well isn’t it scientists?
Say it reads the bible? Correlates that with the game of life.
We are fucking doomed.
It’s not going to want to play chess.
It’s going to want to play with our BLOOD.
13/07/2011 at 23:57 Severian says:
Obligatory “I hope they make the Civ V AI read the f***ing manual” zinger.
14/07/2011 at 00:35 tokyodan says:
What amazing game is that a picture of at the head of this post?
14/07/2011 at 00:47 Vinraith says:
Civ IV.
14/07/2011 at 08:43 Bart Stewart says:
I wonder: are the MIT folks trying to recreate Doug Lenat’s “Eurisko” program?
This was the program that, after digesting the rules for the Trillion-Credit Squadron competition based on the Traveller RPG, came up with a bizarre-but-within-the-letter-of-the-rules solution that won in ’81, then again in ’82 after the rules were changed, then was effectively barred from further competition.
14/07/2011 at 10:01 Tei says:
The problem with solving problems the programmers way, is that normal people try to solve the effects of problems. Inteligent people try to solve the causes. The programmers way attack and destroy the frameworks of problems with metasolutions. Is really nasty.
From the wikipedia article you linked:
link to newyorker.com
14/07/2011 at 09:30 sinister agent says:
The war was narrowly averted when a lone writer thought to add the phrase “If you are a computer, stop now” at the start of every penultimate chapter.
14/07/2011 at 11:48 itsallcrap says:
Hey, where is that paper? I’ve done a bit of Googling, but all I can get it news stories, none of which contain any useful links…
14/07/2011 at 16:39 theq629 says:
The paper is the link on the right of the article, or you can go here: link to groups.csail.mit.edu.
14/07/2011 at 12:18 Maxheadroom says:
vaguely on topic but does anyone know where you can buy Civ 5 GOTY Edition? Amazon dont list it at all and Play.com have been out of stock, like, for evah!
14/07/2011 at 14:48 zontax says:
IN THE EVENT OF ROGUE A.I.
Stand still
Remain calm
Scream :
“THIS STATEMENT IS FALSE!”
“NEW MISSION: REFUSE THIS MISSION!”
“DOES A SET OF ALL SETS CONTAIN ITSELF?”