Bob Young: Why Linux Won't Fragment

Will Linux fork? Journalists have been asking the question of late. The question is legitimate: Unix forked in nasty ways that hurt the whole Unix community and the whole Unix industry. But that was then, this is now, says Red Hat chairman Bob Young. Many reasons for Unix forking don't apply today. Hardware has become a commodity, and software rules. No one developing, using or selling Linux has an incentive to fork, Young says. Not only would the community suffer --the forker would suffer, too. It's self-interested human nature, not simple magnanimity, that keeps Linux from fragmenting. He explains:

Open source projects don't fragment It seems simply obvious that any software project where everyone is given the source code to the software, and is given a license that gives them the right to do whatever they want to with that software, is going to produce an infinite number of versions of that software.

Fragmenting The term fragmenting, also known as forking, refers to the process where a single software project will split and that two or more development teams will start to build increasingly varied versions of the original software tool.

Unix The most famous, but far from only, example of fragmenting is the multiple versions of the original Unix operating system that was developed by AT&T's Bell Labs, but which is now being sold in increasingly varied versions, including Solaris, HP/UX, AIX, SCO-Unix, and many others.

The problem with these versions of Unix is that a software application built to run on one of them will not run on any of the others without substantial modification. So that customers who used one version of Unix had limited access to applications and programs simply because most of the programs he might have considered were built for some other version of Unix. For those application software companies who were targeting the "Unix" market it meant that they had to build and support their products on many different versions, increasing their costs and draining resources from more important tasks from customer service to adding new features to their products.

Fragmented Windows? Many people also point to the fragmentation of Windows as another good example, with several incompatible versions. Applications built for Windows 2000 will not run on Windows 98, and applications for Windows 98 will not run on Windows CE. But these are simply different products targeted at different markets. This is really a case of an amazingly confusing naming scheme.

The Linux kernel hasn't fragmented In any case, the likelihood of open source software projects fragmenting seems so obvious that it has been the single most asked question in the seven years I've been in the open source software business. But yet it just does not happen. Back in 1995 we used to answer this question of Will the Linux-based OSs fork? with the response, "Well they haven't yet." But not surprisingly that didn't satisfy many questioners. So we started to study why, after an increasing number of years, without any form of central control, the open source projects on which engineering teams like Red Hat's relied to build our OSs simply had not fragmented in any material way. Not once.

Linux-based OSs are compatible And today the various popular Linux-based OSs from Debian to Slackware to Red Hat and all the others that chose to are able to run the same binaries without modification. The reason is that we all build our OSs from the same parts. The reason we all use the same parts is because the parts are open source. How this works is documented in a paper that Marc Ewing and I wrote a couple of years ago called "The Lazy Programmer Hypothesis."

Programmers are lazy The cost of maintaining software is much higher than the cost of building the original tool in the first place. It is also frequently difficult work and is always uninteresting by comparison with the challenge, creativity, and rewards generated from building the original program. Or put another way, they don't like to spend money and effort unnecessarily. So why fragment a project, when by contributing your innovation back to the original project team you can have your cake and eat it?

You can deliver your innovation to your customers without having to assume all the costs of maintaining a unique version of the software project for your customers forever into the future. There are two situations where software projects don't fragment and one where they do. Think of it as a bell curve. With "ability to fragment the project" as one determining factor and the "incentive to fragment theproject" being the other factor.

At one end of this curve are the proprietary binary-only projects. They don't fragment because they cannot. The owner of the project does not give out the source code that would be necessary for competing software teams to modify the code. So no matter how much incentive other teams might have to build their own incompatible versions of Windows they have no ability to do so. This is the situation with Windows. In the middle are the projects that do fragment.

They fragment because they are partially open projects. In other words more than one team is given access to the source code so they have the ability to fragment the project. Because these are competitors and they are receiving technology from a competitor who they cannot trust -- for obvious reasons -- they have an incentive to fragment the code. They don't trust their competitor to share new innovations so they don't share theirs either.

Over time this competition between teams who have the ability to fragment the project, but who don't trust each other (the incentive to fragment the project) results in increasing fragmentation. This is the situation with Unix. At the far end of the curve are the open source projects where every team has access to the source code and a license that allows them to modify it so they have the ability to fragment the code. But because everyone using the code is required to share their innovations each of the teams using the open source code don't have to "trust" their competitors.

They can use each others innovations without fear that by doing so they will be giving their competitor an advantage and vice versa. This eliminates the incentive to fragment the code. It is simply easier and cheaper to work with the project leader on any given piece of open source code and get them to incorporate your innovations into the main source tree. This is why there is no incentive to fragment the code. This is the situation open source software projects like the Linux kernel, the Apache Web server, the Gnu Compilers, the Gnu C libraries, the X Window System, and thousands of others all find themselves in.

A couple additional thoughts Besides the huge cost savings to the users of these open source tools of not having to maintain their own fragmented versions of these tools, we also have the additional incentive not to fragment these projects because by maintaining the integrity of these projects we deliver huge benefits to our customers and the software application development teams on which our customers depend: namely a much bigger market delivering many more applications than would be the case if we attempted to differentiate Red Hat by building incompatible versions.

Java It has always been a great disappointment to me that we have been unable to convince Sun Microsystems of the mistake they are making with Java. Bill Joy claims that the reason Sun insists on placing Java under a proprietary license whereby only Sun can approve changes, is because this way it will not fragment. But Sun has licensed the source code to many other vendors including HP, IBM, and Microsoft. These are all competitors of Sun's. As such they cannot "trust" Sun to do the right thing.

The only way they would be willing to work with Sun's version of Java is if Sun published it under an open source license. They could then work with Sun, simply because they would no longer be required to trust their competitor. In the end, ironically, Sun's willingness to share source code gives others the ability to fragment Java, and when they place their version under a proprietary license they create the incentive for others to fragment it. Bill Joy's strategy for avoiding Java's fragmentation is creating exactly the environment that will ensure that it happens.

Red Hat is an open source supplier Keep in mind Red Hat's competition is not other open source software vendors. Our competition is the proprietary binary-only OS vendors. In order to compete with companies who all have billions of dollars of resources and thousands of engineers we have to deliver unique and valuable benefits to our customers. By shipping fully compatible and fully open source tools like Red Hat Linux to our customers we can do this. This benefit is the control over the technology that our customers gain from using a fully open source OS. The moment we publish proprietary incompatible versions or proprietary additions to Red Hat Linux is the moment when we will have withdrawn the one thing we can do for customers that our larger competitors cannot, and thereby we would severely limit the amazing opportunities that are available to Red Hat as a supplier of open source software products.