Titanfall, shoot-'em-up gamers, cloudy contracts and cattle

Efficiency by numbers: It's wonderful, it's horrible and it is the future

Internet Security Threat Report 2014

Sysadmin blog A "servers are cattle, not pets", DevOps-style approach is the only feasible way for a small number of people to run modern cloud-scale data centres.

Small teams running large server farms are necessary to grind down costs so as to keep up with the Amazon-fearing race to the bottom. But even when cloud computing works exactly as designed, I am increasingly convinced these management styles are not always working as desired.

The real world impact of the "cattle, not pets" approach to server management struck me during a conversation with co-worker Josh Folland. He's a professional video gamer working on a review of Titanfall. When I asked him his opinion on the subject, I was surprised by his answer.

While he would easily rate the engine and gameplay as among the top 5 FPS games of all time, he spent a solid hour cursing Microsoft's Azure cloud, and the concepts behind cloud computing in general. His rationale hit surprisingly close to home.

Efficiency by the numbers

The core of Josh's complaint is thus: cloud servers are all about meeting quotas and SLAs; they don't provide the same quality of play that fleets of community-managed dedicated servers once did.

In the old days, community members would band together into "clans" and each "clan" would rent a dedicated physical server at a colocation facility to run the game server for their favourite FPS.

These servers might be restricted to clan members or opened to the public with a given amount of "slots" reserved. If you weren't a clan member and a clan member wanted on, you got bumped.

Clan servers had all the downsides of anything community-managed: petty infighting, lack of maintenance, absurd rules, mods, cheating, red tape and more. They were very much pets, not cattle; each server was babied by the clan administrator, tweaked and optimised until no two servers were alike.

Don't be kept prisoner by lag

Network lag is a great measure of how pets are better than cattle. In an FPS, lag is very, very bad. When you see the image of the enemy soldier on your screen, you aim at it and click on it. Doing so sends a packet to the server which then calculates whether or not you hit the fellow. The enemy troops, meanwhile, are moving around and probably shooting at you.

If your target has a ping of 35msec and you have a ping of 350msec then he can execute 10 actions for every one of yours. He will be able to quite literally dodge bullets and you won't see his bullets coming. In the modern FPS world when two gamers of more or less equal skill meet, the gamer with the lowest ping wins.

For all their faults, community-managed clan servers simply didn't have this problem. If you wanted to play with your buddies on a regular basis, you could all get a fast server with great connectivity located geographically near you. This would ensure that you all had low pings and experienced minimal lag when playing.

Titanfall doesn't work that way. Titanfall's multiplayer servers are a Microsoft Azure cloud-based affair. Players are assigned to a server based on an algorithm and the algorithm only really cares about meeting SLAs.

"Make sure that only X per cent of players per server have a ping higher than Y" or "make sure that CPU usage is below Z" – otherwise spawn new instances and load balance incoming players. It sounds good on paper, but in practice, it is less so.

Getting a sub-50 ping from here in Edmonton, Canada is rare, and the difference between 250msec pings and those below 100msec is entirely the luck of the draw. For hardcore gamers it turns what could have been one of the best games of skill yet created into a game of chance.

People on a string

The "cattle, not pets" thinking that goes into designing a modern cloud application is – to me at least – the issue that needs addressing. In setting a series of scripted limits, we are really saying "we are okay with annoying this many people".

If the people running Titanfall are anything like the hundreds of other SaaS companies I've interviewed then they are frustrated by discussions like the above. In my experience most SaaS developers believe strongly in the purported benefits of cloud computing and they simply don't understand why end users are frustrated.

From a DevOps point of view, the solution is simple: they can tighten their SLAs, set the limits differently and tweak the matchmaking algorithm. This would surely ensure a better experience for all. The infrastructure is dynamic and scalable, so once the perfect algorithms are found, then the thing will basically run itself and all these silly complaints will simply go away.

Unfortunately, this will never solve the real problem. People don't want to be equal. We don't like "the luck of the draw." We want the option to be in control, even if we never exercise it.

The casual gamer is a different animal from the fellow dumping four hours a night into this game with his buddies. Someone catching a round a week on his Intel Graphics-powered notebook doesn't have the same expectations of experience as the individual with the dual-Xeon rig sporting four video cards and a 240hz 56" LCD.

In much the same fashion, cloud storage is a wonderful solution for a certain category of customer, but a terrible plan for many others. Companies turning over terabytes of data every day but who can't get better than an overworked ADSL connection to work with certainly aren't going to be using it any time soon.

In this brave new cloudy world we are all homogenous. Just as servers are "cattle, not pets", we have become "numbers, not people". Perhaps the market opportunity for companies to differentiate themselves lies not in being more clever about the creation of automated scalable infrastructures based on cold numbers and unfeeling logic. It may just lie in those companies who swing the pendulum back towards catering to the individual customer once more. ®

Internet Security Threat Report 2014

More from The Register

next story
NSA SOURCE CODE LEAK: Information slurp tools to appear online
Now you can run your own intelligence agency
Fat fingered geo-block kept Aussies in the dark
NASA launches new climate model at SC14
75 days of supercomputing later ...
Yahoo! blames! MONSTER! email! OUTAGE! on! CUT! CABLE! bungle!
Weekend woe for BT as telco struggles to restore service
Cloud unicorns are extinct so DiData cloud mess was YOUR fault
Applications need to be built to handle TITSUP incidents
BOFH: WHERE did this 'fax-enabled' printer UPGRADE come from?
Don't worry about that cable, it's part of the config
Stop the IoT revolution! We need to figure out packet sizes first
Researchers test 802.15.4 and find we know nuh-think! about large scale sensor network ops
SanDisk vows: We'll have a 16TB SSD WHOPPER by 2016
Flash WORM has a serious use for archived photos and videos
Astro-boffins start opening universe simulation data
Got a supercomputer? Want to simulate a universe? Here you go
prev story


Go beyond APM with real-time IT operations analytics
How IT operations teams can harness the wealth of wire data already flowing through their environment for real-time operational intelligence.
10 threats to successful enterprise endpoint backup
10 threats to a successful backup including issues with BYOD, slow backups and ineffective security.
Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
Website security in corporate America
Find out how you rank among other IT managers testing your website's vulnerabilities.