Feeds

Akamai scoffs humble pie: Heartbleed defence crumbles, new SSL keys for customers

We got this covered ... er, maybe not

Beginner's guide to SSL certificates

Akamai has issued new SSL certificates to some of its customers after realising its customized OpenSSL was not immune to the Heartbleed bug as first thought.

Some time ago, the web distribution giant modified the code to the open-source OpenSSL library and rolled the tweaked version out to just its servers: that adjustment changed the way the library allocates memory so that any particularly sensitive data, such as private crypto-keys, is kept well away from general-purpose allocations that can be mined from afar using the Heartbleed bug.

Akamai thus claimed its customers' private SSL keys were never vulnerable to any Heartbleed attacks: the bug is exploited by sifting through a remote machine's memory for secret goodies like passwords and keys, but the replacement memory allocator, implemented before the bug's discovery, should have thwarted that.

Even so, Akamai still applied the Heartbleed fix to its flavour of OpenSSL just to be safe, as the people who found the bug warned the biz before going public on Monday, 7 April. Crucially, Akamai didn't feel the need to issue more than a very small number of new SSL private keys.

On Friday, 11 April, five days after the Heartbleed vulnerable was revealed, Akamai staff wrote on their blog:

Akamai patched the announced Heartbleed vulnerability prior to its public announcement. We, like all users of OpenSSL, could have exposed passwords or session cookies transiting our network from August 2012 through 4 April 2014. Our custom memory allocator protected against nearly every circumstance by which Heartbleed could have leaked SSL keys. There is one very narrow window through which 4 Akamai server clusters had a vulnerable release for 9 days in March 2013. For the small number of customers potentially affected, we are pro-actively rotating certificates.

Akamai then shared the source code to its OpenSSL modifications with the world, which, while appreciated, is where the wheels started to fall off.

Independent infosec bod Willem Pinckaers looked over the code and was able to poke holes in Akamai's "secure" memory allocator, derailing its defence against the Heartbleed flaw.

By Monday, 14 April, the firm had reversed its position: it admitted it will change-up customers' private cryptographic keys after all – because there's a chance they could have been compromised by anyone who knew of the OpenSSL bug:

Over the weekend, an independent security researcher contacted Akamai about some defects in the software we use for memory allocation around SSL keys. We discussed Friday how we believed this had provided our SSL keys with protection against Heartbleed and had contributed the code back to the community. The code that we had contributed back was, as we noted, not a full patch, but would be a starting point for improving the openssl codebase.

In short: we had a bug. An RSA key has 6 critical values; our code would only attempt to protect 3 parts of the secret key, but does not protect 3 others. In particular, we only try to protect d, p, and q, but not d mod (p-1), d mod (q-1), or q^{-1} mod p. These intermediate extra values (the Chinese Remainder Theorem, or CRT, values) are calculated at key-generation time as a performance improvement. As the CRT values were not stored in the secure memory area, the possibility exists that these critical values for the SSL keys could have been exposed to an adversary exploiting the Heartbleed vulnerability. Given any CRT value, it is possible to calculate all 6 critical values.

As a result, we have begun the process of rotating all customer SSL keys/certificates. Some of these certificates will quickly rotate; some require extra validation with the certificate authorities and may take longer.

Akamai charges its customers extra for serving media over HTTPS from its distribution network, so not that many Akamai users opted for SSL and thus not that many stored private keys on the company's servers. Therefore not that many were affected by the Heartbleed flaw, as some security watchers have already pointed out.

Akaami acted promptly when it realised there was a problem, but crypto-experts still faulted the biz for placing too much confidence in safeguards that turned out to be flawed.

"Nothing against Akamai, but seriously: they held off replacing certs because they thought they were secure? Ugh," said Matthew Green, a professor of computer science at Johns Hopkins University. "I'd really like to hear the case that Akamai didn't play Russian Roulette with their customers' security."

Akamai is not the only tech giant getting it in the neck for its handling of Heartbleed. Dropbox has been faulted for failing to warn customers it was affected by dangerous bug until Saturday, and even then only mentioning it in an easily overlooked forum post rather than through its corporate blog.

The Australian went big on the issue instead of reporting on the opening of a Sydney office by the cloudy sync and share outfit, already under attack earlier this week for its appointment of former US Secretary of State (and surveillance advocate) Condoleezza Rice to its board of director as a privacy adviser. ®

Protecting users from Firesheep and other Sidejacking attacks with SSL

More from The Register

next story
Spies would need SUPER POWERS to tap undersea cables
Why mess with armoured 10kV cables when land-based, and legal, snoop tools are easier?
Early result from Scots indyref vote? NAW, Jimmy - it's a SCAM
Anyone claiming to know before tomorrow is telling porkies
TOR users become FBI's No.1 hacking target after legal power grab
Be afeared, me hearties, these scoundrels be spying our signals
Jihadi terrorists DIDN'T encrypt their comms 'cos of Snowden leaks
Intel bods' analysis concludes 'no significant change' after whistle was blown
Home Depot: 56 million bank cards pwned by malware in our tills
That's about 50 per cent bigger than the Target tills mega-hack
Hackers pop Brazil newspaper to root home routers
Step One: try default passwords. Step Two: Repeat Step One until success
China hacked US Army transport orgs TWENTY TIMES in ONE YEAR
FBI et al knew of nine hacks - but didn't tell TRANSCOM
Microsoft to patch ASP.NET mess even if you don't
We know what's good for you, because we made the mess says Redmond
NORKS ban Wi-Fi and satellite internet at embassies
Crackdown on tardy diplomatic sysadmins providing accidental unfiltered internet access
prev story

Whitepapers

Providing a secure and efficient Helpdesk
A single remote control platform for user support is be key to providing an efficient helpdesk. Retain full control over the way in which screen and keystroke data is transmitted.
WIN a very cool portable ZX Spectrum
Win a one-off portable Spectrum built by legendary hardware hacker Ben Heck
Saudi Petroleum chooses Tegile storage solution
A storage solution that addresses company growth and performance for business-critical applications of caseware archive and search along with other key operational systems.
Protecting users from Firesheep and other Sidejacking attacks with SSL
Discussing the vulnerabilities inherent in Wi-Fi networks, and how using TLS/SSL for your entire site will assure security.
Security for virtualized datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.