Open-source 64-ish-bit serial number gen snafu sparks TLS security cert revoke runaround

A bunfight over a controversial UAE mobile security company led to the discovery that millions of TLS security certificates have been improperly issued – thanks to a dodgy default configuration in popular certificate authority (CA) management software.

During a discussion on the mozilla.dev.security.policy group about Darkmatter's application to become a fully fledged cert-issuing CA, netizens discovered that the company's supposedly 64-bit serial numbers in its certificates were in fact one bit short, the top bit being always zero to indicate a positive integer. After engineers at other organizations read the thread, they realised their own certificates were similarly affected.

Some 2,000,000-plus security certificates, issued by Google, GoDaddy, and others, may need to be replaced, as a result.

UAE-based Darkmatter is in the spotlight due to a January report by newswire Reuters alleging its involvement in state-backed hacking efforts, which it has denied. The report subsequently drew the attention of Firefox browser-maker Mozilla and prompted the Moz dev chatter.

Security researcher Adam Caudill summarised the problem in a blog post earlier this month: "During an analysis of certificates issued by DarkMatter, it was found that they all had a length of exactly 64 bits – not more, not less."

As he explained, RFC 5280, which, among other things, governs the format of public key certificates, states that cert serial numbers must be a "positive integer" of at least 64 bits and absolutely cannot be a negative number.

That may well explain why Darkmatter's certificate serial numbers have their top bits clear. The software used to generate the numbers may be using two's complement to represent the integers, which would require it to keep the top bit clear in each serial number to indicate it is a non-negative value. That would reduce the effective length to 63 bits.

Caudrill speculated that Darkmatter may have used a particular open-source certificate-issuing package, EJBCA, which defaults to outputting 64-bit certificate serial numbers from a random-number generator with the top bit clear to enforce the non-negative rule. This dramatically reduces the number of possible serial numbers, which are used to track issued certs and revoke them.

A cert's serial number should be unique among all other certificates issued by its CA. You really don't want any collisions between certs, as it will lead to further headaches and confusion, and one fewer bit increases the chance of this, for example.

It's not the end of the world. Traffic encrypted using the certs will not suddenly fall open to eavesdropping, for instance. It's pretty much mainly an embarrassing oversight in the strict, by-the-book world of cryptography.

There is nothing to stop CAs using longer serial numbers; it's just that the default in this particular toolkit is misleading.

D'OH-fault settings

A recent MDSP mailing list response by SSL.com's Fotis Loukos, its director of R&D, as reproduced by Mozilla CA program manager Wayne Thayer here, suggested that EJBCA's default settings may have been responsible for lulling CAs into a false sense of security. Ouch.

"EJBCA's method of generating serial numbers has led to a discrepancy between expected and actual behavior and output, such that any CA using EJBCA with the default settings will encounter this issue," he posted, noting that this would put those CAs into breach of Baseline Requirement 7.1, which is the CA rule (PDF, 65 pages) that states all non-sequential certificate serial numbers must be at least 64 bits wide, positive, and derived from a cryptographically secure pseudo-random number generator (CSPRNG).

Other responses from Apple, Google and others shed light on the practical impact.

Apple admitted it had issued a total of 878,000 non-compliant TLS certificates, of which 558,000 were still in use five days ago, as well as 2,000 S/MIME certs. In a timeline appended to its report yesterday, it said that in April 2017 it had "mistakenly suppressed alerts detecting serial numbers suspected to be insufficient in length," before starting to revoke the affected certificates last week.

Google Trust Services, the adtech monolith's certificates arm, did not spell out precisely how many non-compliant certificates it had issued but did say that it comprised all certificates that its Google Internet Authority G3 trust chain issued between 30 September 2016 and 28 February this year.

GoDaddy was similarly affected from 2016 onward, according to its response, having issued a total of 285,936 non-compliant certificates, of which 12,152 are still live. It scaled this down from its original estimate of 1.8 million non-compliant certificates, adding: "We are looking to scope and roadmap upgrading our certificate serial number to a minimum of 128-bit, or the max possible."

The controversy over Darkmatter continues. While the serial number security issue is largely theoretical – 63 bits leaves plenty of space to fend off collision attacks, even if it's not compliant with the spec – CAs will continue to have a minor headache as they identify, revoke, and reissue affected certs. ®