Emergent Tech

Artificial Intelligence

OpenAI bots smashed in their first clash against human Dota 2 pros

AI can react faster than humans, but don't play well enough to beat the masters yet

By Katyanna Quach


The International In the past hour, OpenAI's artificially intelligent bots lost their first match against professional players at smash-hit computer game Dota 2 at The International – the video game's annual championship tournament.

It's the first bout in a best-of-three competition between human professional players versus OpenAI's code, the other two rounds will take place over the next two days, each day a different human team.

Thousands of hardcore Dota fans lit up by glowing bracelets sat down at the Rogers Arena in Vancouver, Canada, to watch pros battle against a machine running OpenAI's software in this first round. The humans – dubbed Team paiN – were five players from Brazil, while OpenAI Five is made up five long-short-term memory neural-network-based agents.

Dota 2 is a popular battle strategy game played online. The goal is to gain take over bases and destroy the enemy’s “Ancient”, a sacred structure. Teams will often clash in sporadic bursts of battle, where the screen is ablaze with all players casting all sorts of glittery spells and powers. Sorta like Command and Conquer meets Tolkien.

It was a close match that lasted more than 45 minutes. The OpenAI bots seemed pretty disorientated right at the start. A group of three walked around in circles seemingly confused, while the human team explored the map. The software agents quickly snapped to their senses, and it was a pretty even match at the beginning. The first blood was drawn by Team paiN, and the kills were pretty tit-for-tat afterwards.

However, a third of the way in, the bots began to dominate in fights and collected more gold. Humans began to catch up a little after to take more towers, and the game seemed pretty neck and neck – despite OpenAI Five racking up more kills. The bots’ towers gradually fell, however, and in the final battle the AI players were wiped out and left their Ancient exposed. The human team quickly demolished it to win.

“We were definitely less confident about winning this time round, because the pros are a lot harder to face. It’s disappointing to lose, but I’m happy that we could stand against a top team in such a long match,” Filip Wolski, a member of technical staff at OpenAI, told The Register.

OpenAI Five showed some strange behavior such as lingering outside the lair of Roshan, a powerful enemy that requires teamwork to defeat. Slaying Roshan is a crucial step as it rewards players gold and an item that allows a hero to respawn quickly after death, however being killed by Roshan sucks up time, effort and resources. All the meanwhile, the map is open for the opposite team to obliterate towers. Any hesitation is therefore not good.

The bots also appeared to cast powerful spells during battles at odd times, sometimes when there were few or no enemies nearby.

Restricted play

Dota 2 is a complicated game. There are more than a hundred playable characters known as heroes each with their own strengths and weaknesses. Some have the ability to cast particularly powerful magical spells, some can travel quickly, some are very strong, and one even comes equipped with baby spiders. All of this, combined with the ability to collect gold to buy a plethora of items to strengthen heroes, makes the game incredibly complex.

The match of Team paiN against OpenAI Five, however, doesn’t capture all of this. There are simply too many combinations and possibilities for a computer to master at the moment. OpenAI has been steadily lifting restrictions on its software agents during gameplay as its bots improve – effectively taking off the training wheels. These changes include:

The decision to pre-select the heroes for both teams was made by OpenAI and the organizers of The International to save time, we're told.

Pro players practice for hours trying to improve reaction times, their teamwork, and their various strategies. Luckily, OpenAI Five has also had a lot of time to practice. In fact, it can play more games in a single day than someone can in their entire human lifetime. Each bot acquires a whopping 180 years of experience every day during the training process, so that’s 900 years per day for the entire team.

Unlike humans, bots can be cloned. OpenAI uses a technology called Rapid, which is a reinforcement-learning framework that allows multiple agents to play a large number of games in parallel. These agents accumulate knowledge in batches which is then used to train all five of the OpenAI Five bots using a proximal policy pptimization (PPO) algorithm, a general-purpose reinforcement learning algorithm. They’re always playing against themselves, too, a method known as self-play.

All of this training guzzles up a heck of a lot of GPUs and CPUs. OpenAI didn’t tell us exactly how many were used to gear up for The International after its practice match in August. However, the short answer is a lot – it slurped 128,000 CPU cores and 256 Nvidia P100 GPUs on Google Cloud when it was trained to play a series of informal matches earlier this year.

Two more matches to go

The bots also have other advantages besides more experience. They can see the whole map and state of the game at once. All the information – such as the health and positions of the heroes, and items in their inventory – is fed to the neural networks every four video frames. “It’s like playing with your eyes closed and opening them every four frames,” Greg Brockman, cofounder and OpenAI CTO, previously explained to The Register.

What does it take for an OpenAI bot to best Dota 2 heroes? 128,000 CPU cores, 256 Nvidia GPUs


OpenAI Five therefore gets to glimpse the whole map at once, whereas humans have to manually move their heroes around to see different parts of the map. So, both teams do have access to the same information but not at the same time.

Although the gameplay reaction time has increased from about 80 milliseconds to 200 milliseconds, so that the bots resemble similar reaction times for human players, the AI players still have the upper hand here. When it comes to menial tasks, clicking the right buttons is pretty much muscle memory, however, for critical moments, such as knowing when to strike with certain spells, a human will often pause to deliberate, and is likely to take about a second or two. The computer is still working in the order of hundreds of milliseconds.

And yet, the computer lost seemingly because it didn't have the long-term strategy honed by the humans.

"What we proved today is we're right there at the edge of human ability, and the test is whether we can beat a pro team," said Brockman in the past few minutes. "We're planning to play other pro teams at The International this week, and we'll have more to share tomorrow and Friday. The test for us is whether we can play at a high enough level to win against a professional team this week."

OpenAI have been steadily working on beating Dota 2 for a while. Last year, it quietly crushed Dendi, a pro player at The International in a restricted game of one-v-one in a mirror match, where each side plays with the same selection of heroes.

This year, it increased this to five-v-five matches, and these were initially mirror matches too until a public game against semi pros in a public match earlier this month.

OpenAI Five may have lost the first match, but it’s not over yet. It’s the best out of three matches, and they will face more pro teams on Thursday and Friday. Stay tuned for more. ®

Sign up to our NewsletterGet IT in your inbox daily


More from The Register

Two in five 'AI startups' essentially have no AI, mega-survey of nearly 3,000 upstarts finds

One pint of beer please, Mr Barman. I am 18 today, honest!

Facebook AI leader Sebastian Riedel to deliver keynote at MCubed 2019

Blind bird ticket offer ending soon

Standards group W3C wins support from all major players to get AI working in the browser

Google, Microsoft, Apple and Mozilla on board, says chair

Want to tell the world what you’re doing with AI? You’ve got 12 days...

Events MCubed call for papers extended

Just forget what Gartner said about AI in June 'cos CIOs are all over it now apparently

Are these the stats you were looking for? Reverse ferret by tech research giant

Big names hurl millions of pounds at scheme to hoist UK's AI knowhow

We're Europe's tech hub, crows minister, but investment weedy compared to the US and China

Register Lecture: Right to strike when your boss sells AI to the military?

Principles AND work for Google – it's been known to happen

Take my bits awaaaay: DARPA wants to develop AI fighter program to augment human pilots

I feel the need, the need for a general adversarial network

Can you tell real faces from fake AI-created ones? It's tough! Plus: Facebook's chief AI scientist talks hardware

Roundup Also, DeepMind published new code to help train agents play football

Photo 'memories' storage biz Ever uses family snaps to train facial recognition AI

You mean you didn't read the 2,566-word privacy policy?