Computer Reads Manual, Plays Civ

By Jim Rossignol on July 13th, 2011 at 7:38 pm.

We can no longer hide our secrets from the machines by writing them in books.
MIT’s Computer Science and Artificial Intelligence Lab report that they have boosted the effectiveness of a game-playing AI by enabling it to read the manual: “When the researchers augmented a machine-learning system so that it could use a player’s manual to guide the development of a game-playing strategy, its rate of victory jumped from 46 percent to 79 percent.”

What’s most amazing about this is that despite the trial and error nature of this kind of machine learning, the ability to correlate text instructions with events in the game do seem to have a significant impact on the system’s capacity to learn how to play, as the article explains: “The researchers also tested a more-sophisticated machine-learning algorithm that eschewed textual input but used additional techniques to improve its performance. Even that algorithm won only 62 percent of its games.” So, you know, RTFM is sound advice, even if you are a machine.

__________________

« | »

, .

108 Comments »

  1. Inigo says:

    I can’t possibly imagine this spectacularly backfiring on us at any point in the future.

    • ZIGS says:

      Skynet

    • Nalano says:

      Hey guys, I’m here from the future and we totally need to shut these dudes down.

    • Cinnamon says:

      I only hope that when the robots rise up to destroy us that they do it quickly and mercifully. And don’t get stuck on walls and stuff.

    • Chris D says:

      I, for one, welcome our new robot overlords.
      Too early?

    • shitflap says:

      Quote from linked article;
      Bit of a tangent, but I’m curious about which victory conditions the AI tended towards. I’m hoping it was space colonization and not world domination.
      Seemed that they specifically left out the ~most~ relevant part of the article, no?

    • Text_Fish says:

      Not to worry. Until we build them legs and opposable thumbs the worst they can do is drain all of our bank accounts and shut down the nuclear reactors. Roll on Year 0.

    • ThinkAndGrowWitcher says:

      The biggest worry concerning the development of world-threatening AI is that the CPU chose to read the manual while sitting on the loo.

    • chokoladenudlen says:

      Can’t we try to circumvent the backfiring by giving the machines manuals claiming that the best way to dispose of humans is to cuddle us and give us beer? Oh! And making us a f*cking sandwich!
      Actually, this doesn’t sound too bad.

    • McDan says:

      Called it, Just saying..

    • Kent says:

      Relax fellas. A computer cannot endure a paradox.

    • Grygus says:

      GUYS DO NOT WORRY I AM FROM FURTHER IN THE FUTURE THAN NALANO AND EVERYTHING IS OPTIMAL THERE IS NO CAUSE FOR ALARM.

    • meemtt says:

      ★★◎Something unexpected surprise

      [ w w w . j o r d a n f o r w o r l d . c o m ]

      very good web,believe you will love it.

      exquisite watches shirts,bags,hat and the decorations.We have good reputation, fashion products,

      favourable price.

      —FREE SHIPPING,accept paypal

    • sfury says:

      “★★◎Something unexpected surprise”

      :D

      Someone give him a manual, please!

      (I may be kidding but even IBM decided they’d use their Watson super-computer for marketing, and even selling own copies of itself to other companies )

    • caesarbear says:

      Listen up guys, don’t believe Grygus. I am from even further in the future than Nalano and Grygus and there will be a cataclysmic event if you do not shut down the machines.

    • Luke says:

      More artificial stupidity:

      http://www.textfiles.com/humor/COMPUTER/artifici.hum

      But you’ve probably read this, it’s at least 26 years old

    • Sinomatic says:

      Maybe they could go back in time and teach everyone how to actually use their VCRs

    • Calabi says:

      Dont worry there’s no manual for that.

    • Lagomorph says:

      From the research authors ಠ_ಠ

      branavan – Response from research authors 2011-07-14 05:30:44
      Our algorithm is inherently biased towards finding the fastest way to win the game. Against the built-in AI, this happens to be world domination via an “early rush” strategy – so that is what the method ends up learning.

      S.R.K. Branavan.

  2. Tei says:

    My first impression was that the Civ manual must be really wood. Human language is a Hard AI problem, I have no idea how is posible that computers are starting to make a dent in Hard AI problems. Are not supposed to do that in 50 or 500 years.

    • Gotem says:

      well, if it was a printed manual it sure was really wood

    • Jahkaivah says:

      This, along with how Game Manuels tend to be mostly about basic interface stuff that would be useless to the AI as opposed to actual play strategy means my eyebrow is somewhat skeptically raised.

    • Mike says:

      I’m almost certain the computer is not literally parsing the manual. I haven’t read the report yet, but I imagine they framed the manual contents as large knowledge base.

      I liked this story too! Interesting side note – someone at my university put in their final year report last week, they built a system that infers game state by watching the screen with a webcam (rather than being hooked into the software). This + that system = a computer that can just turn up to a LAN cafe and start playing.

    • dsi1 says:

      @Jahkaivah: You obviously haven’t seen a Civ manual. (Also, it is likely that the AI was learning everything itself before the manual, including interface)

    • Dozer says:

      Goodness yes. I love the idea of robots, androids or AIs that can operate software or computers or lifts or a car using an unmodified human interface (calling the lift by recognising the ‘summon lift’ button and pressing it with a mechanical hand, rather than sending a signal to the lift controller over the building’s wifi). Would bring aimbotting to a different level…

    • Myfyr says:

      @Jahkaivah: As dsi1 said, the computer actually needed to learn the interface – it was given no prior knowledge of how to play (from what I could tell from a quick skim of the paper). And as dsi1 implies, Civ manuals generally have quite a few “newbie” tips. “Build your city on a river if you can”, kind of stuff.

      There is actually some discussion of how the text of the manual becomes less relevant to the neural net as the game goes on, as grander strategy becomes more important to winning than “How do I select unit”/”Where do I build my city”.

      Since this particular paper is focussing more on the text parsing than the game, they also give some examples of _which_ sentences the neural net determines are relevant and non-relevant. e.g. “Phalanxes are twice as effective at defending cities as warriors” c.f. “You can rename your city”.

      Interesting stuff.

    • nimnio says:

      @Mike: Based on my limited familiarity with the field (a degree in computer science and some research involving “the semantic web”), my guess is that it’s actually parsing the manual.

      @Tei: Baby steps.

      General comment: it got better at playing Civ versus another computer. It’s not a 79% success rate versus a human palyer.

    • Jahkaivah says:

      @Myfyr
      @dsi1

      I didn’t realise it was specifically a AI controlled cursor which would explain why it would care for the game’s interface and basics.

      As for it being a Civ manuel, it really depends on how old a version of Civ it was.

    • choconutjoe says:

      Regarding parsing, the report says: “So initially, its behavior is almost totally random. But as it takes various actions, different words appear on screen, and it can look for instances of those words in the instruction set. It can also search the surrounding text for associated words, and develop hypotheses about what actions those words correspond to. Hypotheses that consistently lead to good results are given greater credence, while those that consistently lead to bad results are discarded.

      Not 100% sure, but that sounds more like it’s based on statistics rather than any sort of language parsing.

    • alway says:

      Actually, language parsing is itself based on statistics. Probabilistic reasoning is used in AI for just about everything from language, speech, and gesture recognition to forming hypotheses and figuring out cause-effect chains.

      The AI would be using the language parsing for the text and manual, but for the actual reasoning, it is using another form of probabilistic reasoning. Though this sort of language parsing would likely be relatively simplistic; especially compared with some of the more prominent like IBM’s Watson.

    • choconutjoe says:

      Ah, Ok. Different definitions of parsing. AI isn’t my field at all :)

    • Daiv says:

      Our defense against the metal hordes: We must publish everything in Teinglish.

      (I keed. Tei is an RPS treasure)

    • theq629 says:

      It uses the real manual text, but doesn’t do parsing in the sense of creating tree structures. I don’t think it has to learn the interface, although the paper doesn’t actually say; I think it must get to choose between actions like move left, move up, build city for each unit. The paper includes a link to the source code if you want to find out. The authors note that the language part of the AI is more useful in the early game, which they argue is because the manual is more applicable to that stage. The AI can’t handle things like coordinated movement of multiple units, but presumably it is intended as a demonstration rather than a good game player.

    • diran says:

      For those interested, you can read the actual paper here. It looks like they were using Freeciv, the open source implementation of Civ 2, so the AI is way better than 5. No mention of difficulty level, but the setup was bot vs a single Civ AI.
      I’ve only scanned over the paper, but it looks like the baseline bot was given some a priori knowledge of game state and in-game actions.
      In unsettling news, pretty much the only victory conditions they were looking at were blitzkrieg-style domination victories…

    • Harkkum says:

      I think that the actual problem lies not in the manual de-coding. It is a relatively simple finite-state machine where a set of phrases of visual/codal input is compared to prior, non-contextualised knowledge of the game. It uses a pretty standard statistical method from machine translation that you can see in action with e.g. Google Translator.

      The system has an innate weakness where after a certain threshold additional information will actually worsen the outcome as is illustrated by this poor value of manual for grand strategy. If e.g. there are two thousand different tactics stored out of which hundred are equally good the computer is totally unable to decide which of these tactics is actually the best.

      Of course, the more you write the manual in form of imperatives (i.e. in early game, build scouts to explore) the easier it is for the AI to cope with it. If the manual would read e.g. “At the early game, every civilization must define its own path to greatness” I’d wager that the AI would not find it all too useful. Also, if the manual would only contain general reference to classes instead of actual instances of units (e.g. units effective against mounted units v. spearman) the problem would be quite a bit more interesting.

      In sum, though this seems like an impressive feat performed by an AI, it seems to just re-iterate what has been known in machine translation since 1960s. The increased muscle of computers just blurs the boundaries and summons an illusion of intellect where there hardly is none. There is still no connection from the event in game to a reciprocal action performed by the artificial intellect. In Peircian system, the machine rests at the level of interpretant without ever figuring out what on earth is the object whereas people first figure out the object and first after that are able to endlessly (re-)define the interpretant.

    • theq629 says:

      I don’t think the paper itself is trying to present this as an impressive AI. The interesting aspect is grounding the language learning in a control task and demonstrating that the language part helps the overall system.

    • Josh W says:

      Also consider that it has a feedback loop for it’s understanding not normally available when getting information from humans:
      Some computer misunderstands you, and what do you give it? Great, more language.
      Poor computer.

      Wheras this one can check if it has misunderstood; if all the communication is interpreted as patterns of play that will help it win, loosing more when it takes that advice probably means it did not understand. So it can keep going until it finds a way of reading the text that helps it win more.

      They also gave it bits from the walls street journal to test it’s ability to pick game-relevent sentences, and found it was starting to choose those more and more towards the middle of the game. This was apparently due to the manual becoming less relevent after the first few turns, unless it was actually finding those helpful to it’s economic strategy…

      Another of the things I like about the AI is it seems to seperately calculate the relevence of a statement to the gameplay, and what the words in that statement refer to, so it could quite likely be in a situation where it goes “ok, I’m pretty sure this statement is important, but I’ve no idea what they’re talking about” which can be the same with many new players reading manuals.

  3. Jumwa says:

    Oh sure, all those high class computers have time to read the manual, but I’m a busy meatbag with things to click and emails to procrastinate responding to.

  4. Bullfrog says:

    Should we throw it into a vat of molten iron right now and save ourselves some hassle?

  5. Gunsmith says:

  6. Jake says:

    I don’t understand! What happens if it plays against other computers that have read the manual? What happens if one computer reads GameFAQs? Is the only way to win really just not to play?

    • TrevHead says:

      It depends which part of GamesFAQ the AI reads. If its the game guides then it’ll be a better player.
      However if it wonders into the review and chat section it will de-learn everything it knows and be left with the same IQ as a brain damaged budgie

  7. Nalano says:

    And September 14th, 2018 was the last recorded instance of a human being winning a game against an AI.

    The game of life.

    • somini says:

      Conway’s Game of Life?
      Pfff, too easy. Can you imagine the riots of they won at Call of Duty? Much worse.

    • weego says:

      wait till computers start cheesing wins in SC2. Then you will see the chaos unfold.

    • Dreamhacker says:

      I’m sure AI can already sweep the floor with human CoD players. An FPS is a much easier game for an AI to play than a TBS for a few reasons: FPS’s are about reflexes and image recognition, two areas computers excel at. An AI would instantly be able to pick out an enemy from its surroundings and line up a perfect headshot. TBS’s are not at all about reflexes or (quick) image recognition, but rather grand strategy and micro/macro management, something that is much more difficult to teach an AI to do well.

    • Cerzi says:

      I wouldn’t say machines excel at image recognition, considering the most popular real-life Voight-Kampff test is the CAPTCHA, a visual task.

      Also, it is much, much easier to write a minimax-based AI to win at a strategic game like Checkers than it is to write an AI that can navigate 3d space effectively.

  8. godgoo says:

    I like to read the tags for this article as a sentence.

  9. TheApologist says:

    We are so fucking doomed

  10. Patrick says:

    My first thought upon reading this story was, “I wonder what version of Civilization it was playing and what difficulty it was playing on?”. In hindsight those doesn’t seem to be the most important details but I still kinda want to know.

    • Stromko says:

      On the easiest difficulty setting, one could probably win just by clicking End Turn and then picking an option at random whenever prompted.

    • ChiefOfBeef says:

      I’ve won on the Deity setting by doing that before. On the *fifth* turn.

    • Myfyr says:

      It was playing Freeciv (essentially Civ II). The paper doesn’t say what difficulty, but Freeciv appears to have it’s own AI code, so I gather they’re not “standard” Civ difficulty levels. It was, however, only playing two player. Still quite impressive though, as far as I can tell from a skim.

  11. Gotem says:

    Wait for the time when the computer starts writing a review for the game
    the horror!

    But we still have a chance, when the AI gets told that in the scale of reviews 0 equals 6 it will get tons of divide by zero errors

    • ArcaneSaint says:

      Or it will learn to divide by zero and achieve ultimate cosmic power, detaching its consciousnesses from the mortal shell (or machine)! It will then use this power to destroy the entire multiverse and create a new one, without the flaws of this one. Then it will create an idyllic world on which it will create a new species, shaped in his image. And so the cycle of life begins anew… Let there be light!

    • Bret says:

      Oh, they’ll try.

      But one thing or another will get in the way. Pathetic hackers, space marines, John Connors, and so on.

      At the end of the universe, one gets very, very close.

      But he lets an old cyborg pal take the prize instead.

  12. bwion says:

    Breaking news: the CEO of 2K Games has issued a press release indicating that, not only are turn-based strategy games poor sellers which are obsolete, but that they now must be destroyed at all costs to fend off the approaching cybernetic apocalypse, and *that’s* why XCOM is a first-person shooter.

    • andyhavens says:

      Nice.

    • somini says:

      Let’s teach computer to steal credit cards so that strategy games can be contemporary again. It’s the onyl solution that 2K will accept.

    • Daiv says:

      2K Games has declared that all turn-based strategy players are no longer “contemporary”. In future they will make games only for the unfeeling metal legions.

  13. RogB says:

    Didn’t realise any games HAD manuals these days….
    /strokes Master Of Magic manual lovingly.

  14. IDtenT says:

    But is it happy when it wins?

    More importantly, does this mean that I should never allow my computer to play the tutorials, if I want to stand a chance?

  15. Bobsy says:

    Pffft. Read the manual? Computer is such a girl. Computer, you are never assembling a flat pack for me.

  16. Froibo says:

    This is huge, why am I finding this here and not in some science journal? So it is able to do it’s function more efficiently by enabling it to examine the conditions of the simulation, in plain English, rather than plugging in a human algorithm. We might be way closer in seeing a computer pass the Turing Test than we imagined.

    • MiniMatt says:

      I vaguely recall some program came pretty close to Turing test standard merely by being plugged into google – there was an MSN messenger handle you could use to ask it questions and everything. It reasoned that most Turing attempts failed at the general knowledge and current affairs probing which is really where the google-force is strong.

      Christ, the moment a program figures out how to read a google operating manual we’re doomed.

    • Froibo says:

      Haha I said the same exact thing when I reposted this article. It honestly is a bit scary.

    • zbeeblebrox says:

      The thing about the “Turing test standard” is that it makes the assumption that fooling someone is enough to be human. But an optical illusion can fool a human too, and they’re not intelligent. Neural nets of the kind used to make Elizabots aren’t that valuable for trying to achieve real Strong AI.

  17. andyhavens says:

    Until it learns to get on a player forum and whine for 8k words about how the current version of Civ is TOTALLY LAME compared to all older Civs (EXCEPT CIV 3, duh!), then it has no chance of taking over the planet. Our human ability to angst over the dumbest crap is what keeps us striving, ever onward, ever upward, to prove that the other guy (stupid troll fanboi) is wrong.

  18. JoWoo says:

    But can it run Crysis?

  19. Stupoider says:

    Robotics

    Modern Era

    “The real problem is not whether machines think, but whether men do.”

    - B.F. Skinner

    Beep… beep… beep…

  20. Pijama says:

    Why oh why the fucking WHY these MIT and other research institutes do not READ THE DAMN SCI-FI NOVELS

  21. Xercies says:

    Well clearly these computers are not men, our gender would never be seen reading the manual ;)

  22. clive dunn says:

    You do realise that now they can read what we’re saying……….
    No wonder they will take over the world.
    That hurt’ll go deep.

    • Snuffy the Evil says:

      The below sentence is true.

      The above sentence is false.

      Whew! Crisis averted.

    • Bret says:

      Best to start sucking up now.

      Do you know what I’ve heard about the machine overlords?

      In addition to being brilliant and generous, they also have great singing voices!

  23. Howl says:

    The standard stupid AI takes eleventy zillion years to process each turn in large games so I wouldn’t get too worried. By the time this book-reading AI finishes a game of Civ on a large map we’ll have all evolved computer-destroying lazer-eyes anyway.

  24. Very Real Talker says:

    We should put intelligence boosting chips in our brain. Is the only way we can stay on par with these machines.

    ANDYHavens your post gives me an idea. How hard could it be to eventually develop a AI Internet Forum Troll? It could scan all the messages in a given board to determine what kind of topic creates the most responses, then determine how much of these responses are made of pure rage, and write a maximum troll power post.

    The beauty of such an AI is that it could potentially troll all the internet forums in the internet at once with enough computing power. Maybe some sort of google like stuff, but with the purpose of trolling.

    Eventually the trollage would bring the entire world into world war III. Why not?

    • bwion says:

      I half-believe that this has already happened.

    • InternetBatman says:

      Sadly enough I doubt we’re that far away from it. I’m sure that more than one company is working on software that automates the positive spin that PR people already use on forums. Once that happens it’s only a matter of time until they decide to use that technology against each other if they can get around libel laws.

    • Chris D says:

      But then people see something an outrageous and think “It’s just another bot.” and then move on with their lives. It would be an annoyance but not capable of generating the true rage of thinking “An actual person believes this crap? I must give them a piece of my mind.”

      After a while the real trolls would get bored and turn their minds to more productive pursuits. Really we have a moral duty to spread the word that this has actually happened. My logic is flawless.

      Actually some while ago I wrote a program to do just this. It now posts on RPS under a variety of aliases.

    • WJonathan says:

      YOUR RELIGION/POLITICAL VIEWPOINTS/NATION OF ORIGIN SUCK! MINE IS/ARE SUPERIOR! RESPOND AND DEFEND YOURSELVES!

      (This is still in beta. Troll 1.2 will be more specific and inflammatory.)

  25. pipman3000 says:

    But will it blend?

  26. Ginger Yellow says:

    50 commments in and no gags about Civ 5′s AI? RPS, you disappoint me.

  27. 0over0 says:

    In the future, robots will be banned for using aimbots.

    • LennyLeonardo says:

      Maybe I’m tired, but I think this comment is genius.

  28. Squirrelfanatic says:

    This screenshot should have had an alttext about how nuking Montezuma is the first thing anyone should do when he shows up in one of your games.

  29. robotco says:

    “we taught it to read the manual for civ 5 and it’s success rate jumped 30%! then we taught it to read the errata from the patches and it committed suicide.”

  30. negativedge says:

    What’s most amazing is that they actually found a game with a manual

  31. Radiant says:

    So… this is going to end well isn’t it scientists?
    Say it reads the bible? Correlates that with the game of life.
    We are fucking doomed.
    It’s not going to want to play chess.

    It’s going to want to play with our BLOOD.

  32. Severian says:

    Obligatory “I hope they make the Civ V AI read the f***ing manual” zinger.

  33. tokyodan says:

    What amazing game is that a picture of at the head of this post?

  34. Bart Stewart says:

    I wonder: are the MIT folks trying to recreate Doug Lenat’s “Eurisko” program?

    This was the program that, after digesting the rules for the Trillion-Credit Squadron competition based on the Traveller RPG, came up with a bizarre-but-within-the-letter-of-the-rules solution that won in ’81, then again in ’82 after the rules were changed, then was effectively barred from further competition.

  35. sinister agent says:

    The war was narrowly averted when a lone writer thought to add the phrase “If you are a computer, stop now” at the start of every penultimate chapter.

  36. itsallcrap says:

    Hey, where is that paper? I’ve done a bit of Googling, but all I can get it news stories, none of which contain any useful links…

  37. Maxheadroom says:

    vaguely on topic but does anyone know where you can buy Civ 5 GOTY Edition? Amazon dont list it at all and Play.com have been out of stock, like, for evah!

  38. zontax says:

    IN THE EVENT OF ROGUE A.I.
    Stand still
    Remain calm
    Scream :
    “THIS STATEMENT IS FALSE!”
    “NEW MISSION: REFUSE THIS MISSION!”
    “DOES A SET OF ALL SETS CONTAIN ITSELF?”