I really was expecting the headline to be "Debian invents predictable random numbers".

Nuff said.

"Another has suggested the bug resides within OpenSSL itself and dates from May 2006,"

This is not true, this issue does not affect upstream OpenSSL or any vendors that are not derived from Debian.

The first of the two explanations is the correct one, Debian maintainers fixed a warning issued by the memory debugger Valgrind by commenting out the code instead of finding the route cause. Needless to say this was incredibly stupid of them. The code concerned allocated memory to the pool used when deriving the random number, commenting it out meant that the number of possible variations was reduced by a significant factor.

According to Ben Laurie of OpenSSL, the Debian maintainers never mentioned what they were doing to anyone at OpenSSL nor did they forward on the "fix". As a result no-one who actually knew the code was able to point out the mistake they were making.

Debian should revoke the permissions of the packager responsible and ensure that only qualified people are given such critical jobs. As Ben Laurie also notes, distributions should not be patching issues in their own copies without also sending patches upstream where they can be reviewed by people who know the code.

It's clearly a stupid mistake by someone in Debian and if they had followed procedures it should have been picked up a long time ago. However the error has now been detected and patched, and people know what to do to correct their systems.

I've already upgraded all my exposed systems and most of my non-exposed systems.

All software has bugs, even open source secure tools from a reputable security concious Linux distro. People using Debian or Mac OS X etc should not get smug and think just because Windows gets most of the attention of the bad people that their system is invulnerable, even though Unix/Linux systems are better designed and built NOTHING IS PERFECT!

At least unlike some vendors who post vague warnings and patches months after the event, with Linux and Debian, nothing is held back...

Adam,

"I've already upgraded all my exposed systems and most of my non-exposed systems."

Upgrading fixes the bug, but not the effects. You should also re-generate your keying material. SSH host keys. SSH user keys. SSL webserver/mailserver keys/certificates. OpenVPN keys. DNSSEC keys. (yeah, right :-) And what with all the key churn and the amount of fingerprint verification that usually happens, isn't this a great time for some MITM games too. (Might be a useful time to start publishing SSHFP DNS records and enable checking in your clients).

The vulnerability notice includes a URL to a scanner which detects keys with weak keying material, including for remote SSH systems. The astute reader will notice that this doesn't just help identify problematic keys on your own networks...

Uninitialised ram space is a lousy source of entropy anyway. It's far too predictable. So simply omitting stirring this buffer's-worth into the pot shouldn't have made any major difference.

If you look at the source, the next thing it goes and does is fills it with stuff from /dev/urandom or /dev/random. It even has a comment about trying for up to 10ms per file. So the uninitialised data is really only a last resort fallback in case there wasn't enough entropy in the system pool. And also no entropy-gathering daemon running.

Conceivably this could only be a problem because there's some other, as yet unacknowledged, problem with the kernel's entropy pool? That's a bit AFDBish, I guess.

Actually OpenSSL people were asked about this on their mailing list...

"What I currently see as best option is to actually comment out those 2 lines of code. But I have no idea what effect this really has on the RNG. The only effect I see is that the pool might receive less entropy. But on the other hand, I'm not even sure how much entropy some unitialised data has.

What do you people think about removing those 2 lines of code?"

This kind of problem happens a lot with Debian. The packagers alter software in order to make it fit some arbitrary "standards" which Debian invented, and they introduce new bugs. This case stands out because it's a security bug, but it's actually an extremely common event. What's worse is that it's been going on for a very long time.

I first installed Debian in 1996, and the exact same kinds of problems were occurring back then. Applications which were packaged by Debian would have all kinds of mysterious bugs in them, and when I removed the Debian-built application and installed the application from the original (not Debian) source code, the bugs went away. When reported the bugs I was told that it was my own fault by some Debian people, but then it turned out that other Debian users could reproduce the exact same bugs I'd described in the Debian packages.

I don't know the details of this OpenSSL bug, but in my opinion Debian would be well advised to make absolutely minimal or even no changes to source code and configurations of software.

The media wants sensations. This is one of those times. Consider a headline "A cryptographic service has a flaw!". It implies that the flaw is serious enough to consider newsworthy, but never mentions the exploitability of the said flaw; which in this case is practically non-existant.

The bug report reference is correct, but the patch in it is not the problematic change (read the patch in the context of the code it changes if you're unsure about this). The Debian maintainer ignored it and made their own change elsewhere.

It's also not in fact true that OpenSSL upstream were not consulted about it; indeed it seems Kurt got a go-ahead (http://marc.info/?l=openssl-dev&m=114652287210110&w=2) in a response from a member of the OpenSSL development team.

"The media wants sensations. This is one of those times. Consider a headline "A cryptographic service has a flaw!". It implies that the flaw is serious enough to consider newsworthy, but never mentions the exploitability of the said flaw; which in this case is practically non-existant." ..... By Gleb Posted Wednesday 14th May 2008 03:08 GMT

Which in this case, Gleb, is you crying wolf. And whether that is a question or a statement is dependent upon and relative to what you know about the Securing Code with Permissible NEUKlearer Links.

They are open about mucking up, and quick to patch it (unlike ms who say, yeah we'll patch it a month next tuesday)

I wasn't sure about the exploitability of this bug and it would be good if the The Reg could find out. I'm no crypto-expert, but it seems to me that the bug reduced the number of keys to somewhere around 250,000, in which case I would have thought the encryption would be trivially compromised. Whether I care or not is a different story. My neighbours are luddites, my ISP is lazy and I trust most of the internet gateways in-between. That said, I'm sure someone does care.

I think this bug is probably worth the media circus.

Read Ben Laurie's follow up for an explanation of why Kurt was given the thumbs up - http://www.links.org/?p=328

It also answers a lot of the questions asked here as well as filling in some of the background to the case.

You are correct, once the patch has been applied all keys need to be regerated, which is of course what I did - but did not say.

As of today all my systems, exposed or otherwise have been upgraded and keys have been regenrated.

The Debian Wiki contains a page which details the upgrade procedure: http://wiki.debian.org/SSLkeys

http://rafb.net/p/JFifHk85.html

Paris, cos she hates bugs too.

The keys are not only breakable by experienced crypto-analyst types. This bug has caused identical private SSH keys to be generated on totally unconnected hosts. As a result, logging into a remote server with my key a few days ago, I was logged in as somebody entirely different, who happened to have an identical private key, giving me access to their source code repo.

But, but... many eyes make all bugs shallow... ESR told me, so it must be true.

Meanwhile "At least unlike some vendors who post vague warnings and patches months after the event, with Linux and Debian, nothing is held back..."

Um... this was in the source tree for *two* *fucking* *years* - or 24 months after the event, in old money. And when it was finally patched in the public sources, it then took Debian five days to announce it to their customers. Ever wonder how many boxes you could compromise in five days given a clear run? Or do you simply assume that only the Good Guys read CVS commit logs?

zOMFG!!! I see what you mean now!

--- openssl-0.9.8c.orig/crypto/rand/md_rand.c

+++ openssl-0.9.8c/crypto/rand/md_rand.c

@@ -465,8 +465,10 @@

MD_Update(&m,local_md,MD_DIGEST_LENGTH);

MD_Update(&m,(unsigned char *)&(md_c[0]),sizeof(md_c));

#ifndef PURIFY

+#if 0 /* Don't add uninitialised data. */

MD_Update(&m,buf,j); /* purify complains */

#endif

+#endif

k=(st_idx+MD_DIGEST_LENGTH/2)-st_num;

Yes, that's a whole world of different! The 363516 patch was the way they should have done it, but the effect of doing it down here in the lower layers is that they just discarded all the bytes that RAND_poll so carefully read out from /dev/urandom and passed to them! It discards /all/ the entropy, from any source, kernel daemon and all! D'oh and double d'oh!

;-) Technically, of course, this could be fixed by a one-line follow-up patch:

--- openssl-0.9.8c.orig/crypto/rand/md_rand.c

+++ openssl-0.9.8c/crypto/rand/md_rand.c

@@ -466,7 +466,7 @@

MD_Update(&m,local_md,MD_DIGEST_LENGTH);

MD_Update(&m,(unsigned char *)&(md_c[0]),sizeof(md_c));

#ifndef PURIFY

-#if 0 /* Don't add uninitialised data. */

+#if 0 /* Don't add any data at all. */

MD_Update(&m,buf,j); /* purify complains */

#endif

#endif

There, that's correct now!

...sensational is when someone writes a worm that'll guess the keys, root your box, and start using it to relay "Ch33p v14gr4" spam to several hundred million of you best friends.

Spent many hours making new keys yesterday. Not pleased!

http://www.links.org/?p=328

http://advogato.org/person/branden/

Nice of the OpenSSL people to put their *real* "contact the devs" address somewhere easily found – right next to the bypass plans.

Umm...I use Umbongo...so I don't have several million best friends. At least they don't e-mail me much. Can you tell me where I can get so many best friends?

But on a more serious note, not being as much of a Linux geek as I'd like to be, what are the implications for desktop flavours of Debian/Ubuntu? I mean we're all using OpenSSL when we're ordering our pr0n and v!@gR4 are we not?

… because people have been mass-generating bad keys ‒ some with the intention of providing blacklists, and some being blackhats ‒ and these are in circulation.

http://blog.drinsama.de/erich/en/linux/2008051401-consequences-of-sslssh-weakness.html

'Instead of mixing in random data for the initial seed, the only "random" value that was used was the current process ID.'

- http://metasploit.com/users/hdm/tools/debian-openssl/

Ironically, the diff posted above (with #ifdef PURIFY) is the harmless half of change 141. It was the change to ssleay_rand_add() that was a disaster, not the change to ssleay_rand_bytes(), which is harmless.

Just check http://xkcd.com/424/

BTW, too bad for all these Debian tards, suits them well. Now, if you don't mind, I'll be awfk, have a few keys to re-generate...

there's a reason the xkcd was a bit late, check the mouseover text on the image...