Google's AI could probably beat you at Atari

A team of scientists at Google created an artificially intelligent computer program that can teach itself to play Atari 2600 video games.

Google DeepMind
Google's new AI program is capable of learning from experience, much like a human brain.

Computers have already beaten humans at chess and "Jeopardy!," and now they can add one more feather to their caps: the ability to best humans in several classic arcade games.

A team of scientists at Google created an artificially intelligent computer program that can teach itself to play Atari 2600 video games, using only minimal background information to learn how to play.

By mimicking some principles of the human brain, the program is able to play at the same level as a professional human gamer, or better, on most of the games, researchers reported today (Feb. 25) in the journal Nature. [Super-Intelligent Machines: 7 Robotic Futures]

This is the first time anyone has built an artificial intelligence (AI) system that can learn to excel at a wide range of tasks, study co-author Demis Hassabis, an AI researcher at Google DeepMind in London, said at a news conference yesterday.

Future versions of this AI program could be used in more general decision-making applications, from driverless cars to weather prediction, Hassabis said.

Learning by reinforcement

Humans and other animals learn by reinforcement — engaging in behaviors that maximize some reward. For example, pleasurable experiences cause the brain to release the chemical neurotransmitter dopamine. But in order to learn in a complex world, the brain has to interpret input from the senses and use these signals to generalize past experiences and apply them to new situations.

When IBM's Deep Blue computer defeated chess grandmaster Garry Kasparov in 1997, and the artificially intelligent Watson computer won the quiz show "Jeopardy!" in 2011, these were considered impressive technical feats, but they were mostly preprogrammed abilities, Hassabis said. In contrast, the new DeepMind AI is capable of learning on its own, using reinforcement.

To develop the new AI program, Hassabis and his colleagues created an artificial neural network based on "deep learning," a machine-learning algorithm that builds progressively more abstract representations of raw data. (Google famously used deep learning to train a network of computers to recognize cats based on millions of YouTube videos, but this type of algorithm is actually involved in many Google products, from search to translation.)

The new AI program is called the "deep Q-network," or DQN, and it runs on a regular desktop computer.

Playing games

The researchers tested DQN on 49 classic Atari 2600 games, such as "Pong" and "Space Invaders." The only pieces of information about the game that the program received were the pixels on the screen and the game score. [See video of Google AI playing video games]

"The system learns to play by essentially pressing keys randomly" in order to achieve a high score, study co-author Volodymyr Mnih, also a research scientist at Google DeepMind, said at the news conference.

After a couple weeks of training, DQN performed as well as professional human gamers on many of the games, which ranged from side-scrolling shooters to 3D car-racing games, the researchers said. The AI program scored 75 percent of the human score on more than half of the games, they added.

Sometimes, DQN discovered game strategies that the researchers hadn't even thought of — for example, in the game "Seaquest," the player controls a submarine and must avoid, collect or destroy objects at different depths. The AI program discovered it could stay alive by simply keeping the submarine just below the surface, the researchers said.

More complex tasks

DQN also made use of another feature of human brains: the ability to remember past experiences and replay them in order to guide actions (a process that occurs in a seahorse-shaped brain region called the hippocampus). Similarly, DQN stored "memories" from its experiences, and fed these back into its decision-making process during gameplay.

But human brains don't remember all experiences the same way. They're biased to remember more emotionally charged events, which are likely to be more important. Future versions of DQN should incorporate this kind of biased memory, the researchers said.

Now that their program has mastered Atari games, the scientists are starting to test it on more complex games from the '90s, such as 3D racing games. "Ultimately, if this algorithm can race a car in racing games, with a few extra tweaks, it should be able to drive a real car," Hassabis said.

In addition, future versions of the AI program might be able to do things such as plan a trip to Europe, booking all the flights and hotels. But "we're most excited about using AI to help us do science," Hassabis said.

Follow Tanya Lewis on Twitter. Follow us @livescienceFacebook &Google+. Original article on Live Science.

You've read  of  free articles. Subscribe to continue.

Dear Reader,

About a year ago, I happened upon this statement about the Monitor in the Harvard Business Review – under the charming heading of “do things that don’t interest you”:

“Many things that end up” being meaningful, writes social scientist Joseph Grenny, “have come from conference workshops, articles, or online videos that began as a chore and ended with an insight. My work in Kenya, for example, was heavily influenced by a Christian Science Monitor article I had forced myself to read 10 years earlier. Sometimes, we call things ‘boring’ simply because they lie outside the box we are currently in.”

If you were to come up with a punchline to a joke about the Monitor, that would probably be it. We’re seen as being global, fair, insightful, and perhaps a bit too earnest. We’re the bran muffin of journalism.

But you know what? We change lives. And I’m going to argue that we change lives precisely because we force open that too-small box that most human beings think they live in.

The Monitor is a peculiar little publication that’s hard for the world to figure out. We’re run by a church, but we’re not only for church members and we’re not about converting people. We’re known as being fair even as the world becomes as polarized as at any time since the newspaper’s founding in 1908.

We have a mission beyond circulation, we want to bridge divides. We’re about kicking down the door of thought everywhere and saying, “You are bigger and more capable than you realize. And we can prove it.”

If you’re looking for bran muffin journalism, you can subscribe to the Monitor for $15. You’ll get the Monitor Weekly magazine, the Monitor Daily email, and unlimited access to CSMonitor.com.