This article is more than 1 year old

Akamai scoffs humble pie: Heartbleed defence crumbles, new SSL keys for customers

We got this covered ... er, maybe not

Akamai has issued new SSL certificates to some of its customers after realising its customized OpenSSL was not immune to the Heartbleed bug as first thought.

Some time ago, the web distribution giant modified the code to the open-source OpenSSL library and rolled the tweaked version out to just its servers: that adjustment changed the way the library allocates memory so that any particularly sensitive data, such as private crypto-keys, is kept well away from general-purpose allocations that can be mined from afar using the Heartbleed bug.

Akamai thus claimed its customers' private SSL keys were never vulnerable to any Heartbleed attacks: the bug is exploited by sifting through a remote machine's memory for secret goodies like passwords and keys, but the replacement memory allocator, implemented before the bug's discovery, should have thwarted that.

Even so, Akamai still applied the Heartbleed fix to its flavour of OpenSSL just to be safe, as the people who found the bug warned the biz before going public on Monday, 7 April. Crucially, Akamai didn't feel the need to issue more than a very small number of new SSL private keys.

On Friday, 11 April, five days after the Heartbleed vulnerable was revealed, Akamai staff wrote on their blog:

Akamai patched the announced Heartbleed vulnerability prior to its public announcement. We, like all users of OpenSSL, could have exposed passwords or session cookies transiting our network from August 2012 through 4 April 2014. Our custom memory allocator protected against nearly every circumstance by which Heartbleed could have leaked SSL keys. There is one very narrow window through which 4 Akamai server clusters had a vulnerable release for 9 days in March 2013. For the small number of customers potentially affected, we are pro-actively rotating certificates.

Akamai then shared the source code to its OpenSSL modifications with the world, which, while appreciated, is where the wheels started to fall off.

Independent infosec bod Willem Pinckaers looked over the code and was able to poke holes in Akamai's "secure" memory allocator, derailing its defence against the Heartbleed flaw.

By Monday, 14 April, the firm had reversed its position: it admitted it will change-up customers' private cryptographic keys after all – because there's a chance they could have been compromised by anyone who knew of the OpenSSL bug:

Over the weekend, an independent security researcher contacted Akamai about some defects in the software we use for memory allocation around SSL keys. We discussed Friday how we believed this had provided our SSL keys with protection against Heartbleed and had contributed the code back to the community. The code that we had contributed back was, as we noted, not a full patch, but would be a starting point for improving the openssl codebase.

In short: we had a bug. An RSA key has 6 critical values; our code would only attempt to protect 3 parts of the secret key, but does not protect 3 others. In particular, we only try to protect d, p, and q, but not d mod (p-1), d mod (q-1), or q^{-1} mod p. These intermediate extra values (the Chinese Remainder Theorem, or CRT, values) are calculated at key-generation time as a performance improvement. As the CRT values were not stored in the secure memory area, the possibility exists that these critical values for the SSL keys could have been exposed to an adversary exploiting the Heartbleed vulnerability. Given any CRT value, it is possible to calculate all 6 critical values.

As a result, we have begun the process of rotating all customer SSL keys/certificates. Some of these certificates will quickly rotate; some require extra validation with the certificate authorities and may take longer.

Akamai charges its customers extra for serving media over HTTPS from its distribution network, so not that many Akamai users opted for SSL and thus not that many stored private keys on the company's servers. Therefore not that many were affected by the Heartbleed flaw, as some security watchers have already pointed out.

Akaami acted promptly when it realised there was a problem, but crypto-experts still faulted the biz for placing too much confidence in safeguards that turned out to be flawed.

"Nothing against Akamai, but seriously: they held off replacing certs because they thought they were secure? Ugh," said Matthew Green, a professor of computer science at Johns Hopkins University. "I'd really like to hear the case that Akamai didn't play Russian Roulette with their customers' security."

Akamai is not the only tech giant getting it in the neck for its handling of Heartbleed. Dropbox has been faulted for failing to warn customers it was affected by dangerous bug until Saturday, and even then only mentioning it in an easily overlooked forum post rather than through its corporate blog.

The Australian went big on the issue instead of reporting on the opening of a Sydney office by the cloudy sync and share outfit, already under attack earlier this week for its appointment of former US Secretary of State (and surveillance advocate) Condoleezza Rice to its board of director as a privacy adviser. ®

More about

More about

More about

TIP US OFF

Send us news


Other stories you might like