Original URL: http://www.theregister.co.uk/2008/11/04/secure_hash_competition/

US kicks off secure hash competition

Gentlemen, choose your algorithms

By Robert Lemos

Posted in Security, 4th November 2008 14:02 GMT

Dozens of amateur and professional cryptographers signed up last week for the United States' first open competition to create a secure algorithm for generating hashes - the digital fingerprints widely used in a variety of security functions.

The contest, run by the National Institute of Standards and Technology (NIST), seeks to find a strong replacement for the current family of hash functions, some of which have been shown to be cryptographically weaker than originally thought. The agency expected at least 40 proposals to be submitted by the Friday 31 October deadline.

Hash algorithms are very important functions in computer security. The algorithms can reduce a large data file - such as a Word document or e-mail message - to a simple, if sometimes long, number that can be used to identify the data, in the same way that fingerprints are used to identify humans. A good hash function gives a completely different result if the original file is changed even slightly. A variety of encryption and security functions use hashes, from integrity checks to digital signatures.

"They are probably the most widely used and least talked about operations in cryptography," said William Burr, manager of the security technology group for NIST. "We expect it to be an interesting and useful exercise, and we hope to find out a lot more about hash functions."

There are a number of hash functions in current use. Two early algorithms - MD4 and MD5 - are the basis for the current family of government-certified hashing algorithms, known as the secure hashing algorithms (SHA), and including SHA-0, SHA-1, and SHA-2, the latter which actually consists of four functions depending on the number of bits desired in the resulting hash.

Yet researchers have found practical attacks against MD4 and SHA-0, demonstrating the ability to generate "collisions," ways of creating two data files that result in the same hash. By forcing a collision, an attacker could, for example, create a modified version of a contract that appears to match - according to the hash - the original digitally-signed document. While SHA-1 can still not be practically attacked, the length of time it takes to find a collision has theoretically shrunk considerably. Cryptographers originally thought that a computer that could perform an attack calculation one million times every second would find a SHA-1 collision only once in 38 billion years, but in 2005, researchers found a way to produce a collision once every 19 million years and then shortened that to once every 300,000 years.

While no significant attacks have been found against SHA-2, NIST is not waiting. A year ago, the agency called for submissions for a proposed new hashing standard, SHA-3. By Friday, the deadline for entries, the agency expected to see nearly 40 proposals - many from teams of professional cryptographers. The submissions will be judged by NIST for mathematical soundness, the perceived randomness of the hash values, the computational and memory efficiency of the hash calculation and the flexibility of the algorithm.

The SHA-3 competition directly parallels the contest that determined the Advanced Encryption Standard (AES) mandated by the US government and used by many other nations and companies worldwide.

"Think of it as the encryption demolition derby," said Bruce Schneier, a well-known cryptographer and the chief technology officer for BT Counterpane. "We will be trying to destroy each other's algorithms. This will be a fun time for cryptographers."

Schneier and seven other cryptographers from industry and academia have submitted a proposal for their "Skein" family of hash functions. Cryptographer Ron Rivest - known famously as the 'R' in RSA encryption - and a team of more than a dozen cryptographers have proposed a hash algorithm that they dubbed MD6.

NIST plans to initially do a completeness check to make certain that every entry satisfies the requirements set by the government. The resulting list of submissions will be posted online by the end of the year, Burr said. In early 2009, the agency will hold a cryptography workshop to discuss the submissions. A second workshop will likely be scheduled in early 2010, by which time NIST aims to have the field whittled down to 15 algorithms, Burr said.

"If some of the proposals are pretty sketchy, then they are not going to make it compared with the proposals that explain why they did this or why they did that," Burr said. "It may be that we will decide that we want to do an earlier cut, so people can concentrate on the likely winners."

While the submissions have to gain check marks for NIST's requirements, the most significant gauntlet will be the scrutiny of other researchers.

"Over the next couple of years, a favorite occupation for grad students will likely be to break these algorithms," Burr said. "People are going to be writing papers about attacks on various hash functions. It's all fair game."

Burr believes that at least 40 teams will likely submit proposals, more than double the number of submissions to the contest to determine the Advanced Encryption Standard (AES). BT Counterpane's Schneier, a noted cryptographer, estimates that the field of submissions might rise to nearly 80, if interested amateur cryptographers make proposals as well.

More proposals is not necessarily better, Schneier said.

"The large number of entries is worrisome, because what it will do is dilute our (the community of cryptographers') analytical efforts," Schneier said. "The more there are, the less scrutiny that the proposals will get."

Yet, the cryptographers will have a lot of time to find the cracks in the hashing functions. The competition is a marathon, not a sprint, stressed Burr.

"SHA-3 is not going to be available, realistically, for another decade," he said. "If you look how long it is going to take us to pick it and how long it will take to make its way into products, well, we have been telling people not to wait on SHA-3."

If you have tips or insights on this topic, please contact SecurityFocus.

This article originally appeared in Security Focus.

Copyright © 2008, SecurityFocus