On the slow death of UNIX and the rise of Linux

Background:

Commercial UNIX variants arose from AT&T UNIX (first released in 1979). The market became dominated primarily by high-end RISC-based hardware manufacturers, such as DEC, IBM, SGI, and Sun. A number of companies did also release versions for the the Intell architecture including Microsoft for a while (they sold that division eventually to the Santa Clara Operation, or SCO, now Tarantella). Eventually, Novell gave the UNIX trademark to the Open Group which now certifies various operating environments to be "UNIX compliant."

Each UNIX vendor has pursued an aggressive strategy of differentiation and tied their operating systems to their hardware. This methodology, while perhaps necessary to remain competitive, has lead to much higher cost for UNIX-based systems. This is because the marginal cost for the operating system is nearly 0, while the research and development cost is quite high.

With the release of MS-DOS 1.0, Microsoft began what was to be a very successful coup in the operating system indistry. They established an operating system which for the first time could run on several different vendor's computer systems and the result was a collapse in prices both of software and of hardware. For a while, the more reliabel UNIX servers were immune and the real causalties were the Commodore Amiga, Apple, IBM, and others in the low-end markets. DOS lacked any capabilities to handle multi-process or multi-user environments so it was completely unsuitable for any server operations of any sort. DOS was, however, wildly successful because it lead to inexpensive and ubiquitous computers.

In 1991, Linus Torvalds began working on what was to be the Linux kernel. Initially, his motivation was exclusively leasure-oriented (as indicated by his mention that it would not be big and professional like GNU). Over time, the development of Linux has accellerated damatically, and now often has support for emerging hardware standards before Windows.

Microsoft Windows NT was designed to be Microsoft's answer to DEC VMS and UNIX, though Microsoft had earlier attempted to write their own UNIX varient called Xenix (later sold to SCO). Windows NT and Linux have become the engines which are causing the slow collapse of UNIX.

This discussion does not attempt to discuss the BSD's and their role in this process.

What makes UNIX vulnerable?

As mentioned before UNIX vendors have generally sold vertically integrated solutions, and there is a large issue of economy of scale in the software industry. Therefore, selling a lower-volume solution is not likely to be successful of a higher-volume solution exists which is good enough.

As Linux and Windows NT have matured and become both more stable and more scalable, they have become good enough for a range of applications which were previously the domain of UNIX, Netware, and VMS. These operating systems have suffered notably and have become much less profitable. Additionally, the cost has gone up. Price increases combined with margine decreases is not a sign of long-term viability. As can be expected, market shares have dropped substantially, and are continuing to drop of every year, according to the IDC.

UNIX Vendor Reaction

I think it is clear that most vendors are aware of the problem and are seeking solutions. For example, IBM has becoming increasingly vocal regarding their aim of replacing AIX with Linux in such a way as to avoid leaving their customers stranded. SGI seems to be pursuing a similar strategy.

Sun, on the other hand is trying to win Linux customers by offering interoperability but are trying to lock customers into their operating system. They are, however, conceding lower-end systems to thei rival and offer low-end servers running Linux. Of the major vendors, they are the only which does not seem to be pursuing a strategy of migrating customers to the commodity solution.

The SCO Group (formerly Caldera Linux) has actually moved the other direction. They have stopped selling Linux, sued various Linux contributors and users under a variety of claims (all with little success) and continued to sell OpenServer and other products. However, there is no indication that they will ever become profitable, and the IBM countersuit casts dark shadows over most of their continued operations.

Winners and Losers

I expect that Linux vendors and the hardware vendors that support them (particularly IBM) will be the large winners. I expect Sun and SCO to be the immediate losers. Perhaps the BSD varients will be winners as well. In the end, this struggle will lead to a more direct and heated struggle between Linux and Microsoft Windows (NT-based). Stay tuned!

Thoughts on an Upcoming Patent War

There seems to be growing speculation that Microsoft and others may be preparing a patent war against Linux. While I think that this is not nearly as imminant as some have argued, I think it will happen at some point. In the end, I think that open source will not only weather the storm but actually be stronger as a result. I am not a lawyer, of course, and nothing here should be construed as legal analysis or advice. However, this is a strategic document regarding the obstacles that using patents to stop open source and how we can be successful.

Market Considerations

Until now, Microsoft has had every incentive not to press for patent claims against competing products. First, they have been convicted of antitrust activity, and such action may lead to court-mandated licensing to competitors. But more importantly, such news could be taken by the industry as acknowledgement of the legitimacy of competition. By this thinking, suing over patent infringement in Linux would be good for Linux and bad for Microsoft. Such suits could therefore actually lead to marketshare losses for Microsoft.

It is true that things are changing. Such a suit today would not necessarily boost Linux's stature in the server market where it is already established, but it would require suing Microsoft customers for migrating away from Microsoft products. Additionally the court-mandated licensing concerns would still apply. So, it is still a lose-lose proposition for Microsoft as they would possibly be losing control over intellectual property and would certainly lose goodwill. So such a suit today would continue to boost Linux's market share at the expense of Windows.

The current Microsoft strategy appears to be that of providing moral and sometimes financial support to parties which litigate over Linux. Currently, the only party to do this, SCO, has had very little success in the courts, and my reading of their complaints has lead me to believe that they are not litigating at all over allegations of intellectual property in the Linux kernel itself, regardless of their press releases. Their lawsuit against Daimler-Crysler has been largely dismissed, their slander of title lawsuit over ownership of the relavent copyrights against Novell has been dismissed once and may be dismissed again. Their other suits notable in that they are not claiming ownership of any code in the Linux kernel in their actual complaints. If, as I expect, SCO loses the IBM counterclaim 10 (non-infringement of IBM's Linux activities), then I would expect Linux to benefit both from the focus that the suits have given and the vigorous defence of Linux by IBM, Novell, Red Hat, etc.

So copyright litigation does not seem to be a threat to Linux, at least not at the moment. What about patent litigation?

Patents and Licensing regarding Antitrust Law

Again, I am not a lawyer. If you really need analysis of this issue, hire an attourney.

In the past, the courts have generally held that patents cannot be used by a monopoly to prevent vital competition or to further activity prevented in antitrust activities. Thus one possible result of a suit by Microsoft is that it may be possible to force Microsoft to license such patents in terms compatible with the GNU General Public License (royalty-free, perpetual for all users of software under this license). This would be a more likely scenario in an interoperability project like Samba than in the Linux kernel.

For the Linux kernel, it may be possible to simply rewrite portions of the code to avoid the patents, and prevent Microsoft from being able to seek damages from users under antitrust laws. Also, prior art may be usable to weaken, narrow, or even overturn certain patents held by Microsoft.

Other Roads to Interoperability

Even if certain products, such as Samba, become problematic for patent reasons, there are other ways to enforce interoperability. In my opinion, such encumberances would be more likely in the event that Microsoft has already lost substantial marketshare on the desktop (or perhaps in countries which recognize software patents but are unwilling to force licensing in response to antitrust violations).

Interoperability can be achieved in any of three ways. The first is to provide network services on open source platforms which interoperate with windows-native services. This is the most dangerous approach, patent-wise, but it is often necessary because it allows one to add open source software to their network without any costly migrations. Samba embodies this approach.

The second approach can be the development of clients on Windows which utilize open standards (such as NFS). Microsoft Services for UNIX takes this approach. This is most useful when integrating Windows systens into existing UNIX or Linux networks.

A varient of these two strategies is the development of gateways which can allow clients using one method to use resources using a different one. For example, SFU provides a NFS gateway service and it is possible to do something similar using Samba.

The third strategy is fundamentally different. It is quite possible to develop methods for accessing network resources, such as files and printers, which are designed to be cross-platform from the beginning. OpenAFS, for example, provides distributed, fault-tolerant, secure file access across different platforms.

Assuming we don't see patent encumbrances killing open source interoperability products before Linux gains substantial marketshare on the desktop, such litigation in an attempt to force people to use Microsoft software may instead force people to consider larger migrations away from Microsoft software. Even if such a campaign were to be begun now, I think that it would still force companies to more seriously consider using open source software in larger roles.

How we should react as a community:

There are several things we can do to, in my opinion, to help protect ourselves from this upcoming patent war. Most of these are already underway.

The first is to attempt to actively challenge Microsoft patents which are important to open source software. Current work on this direction does include Pubpat's petition to get the USPTO to re-examine the patent on certain aspects of Microsoft's FAT filesystem. We continue to review such patents and challenge them where appropriate.

Secondly, we need to develop an infrastructure for a communal response to litigation in general. Groklaw has already served to demonstrate that such an infrastructure is not only possible but effective. Such a community resource can be used to facilitate distributed searches for prior art, a forum for legal discussion (where lawyers and non-lawyers can converse), and many other such things. Although Groklaw is extremely valuable, it is run by one person (Pamela Jones). Ideally, we need a foundation-based service which will allow such an infrastructure to be owned by the community in general. Private sites, such as Groklaw would become the extremities of the distributed system rather than the core, as it is today in the SCO battles.

In the end, I do not think that Microsoft or anyone else can win a patent war against Linux (think of the result of Unisys's statements regarding patents and GIF format). In the end, we will end up with something more powerful and better engineered, and with better marketshare.

How to get ahead with open source

What will the new economy be like?

The basic name of the game in capitalism is control of capitalism. The basic name of the game in feudalism is the control of land. The unit of control in feudalism is the state, while the unit of control in capitalism is the person (natural person or corporation). So the state attempts to control land while the person attempts to control capital. As societies progress, these units and goals do not go away. So in capitalist societies, we still have states which control land, but this becomes less important than it did for a feudal society. I.e. capitalist societies are probably less likely to go to war to annex land than their feudal counterparts. States still require land, but expansion and control are less important goals than they were in the past.

Similarly, corporations and individuals in the new economy will still require capital just as states still require land. However the control of capital of the market as we know it today will be lower priority. Expansion will still be important but may take on a different tone. For example, today, the United States is not interested in exanding its borders, but the expansion has taken on a new dimension where the sphere of influence both culturally and militarily are the current replacements for annexing land.

Today, the state exists primarily as a support mechanism for the economy and personal wealth (including corporate wealth), and to provide basic services such as emergency response, law enforcement, etc. These roles will of course remain unchanged.

However, corporations will become support mechanisms for the communities. Today we already see this with major successful open source projects, such as PostgreSQL, Mozilla, Linux, and Apache. Such corporations provide support, further development, and other services for the other members of the community. This is not that different from the current situation where corporations offer services for support of social communities today, except that the community is actually the entity which engineers the goods. Thus actual control over the market must be foregone in favor of control over the direction of development.

How to get ahead:

When controlling the direction of development, a company can attempt to expand the services they offer to existing customers, cut their costs, etc. To understand how this is done, lets look at the structure of the community (as applicable to old economy as new economy thinking).

The community exists as a collection of persons (including corporations) with a single person or small group in the center. Mere users exist on the fringes of the community (think of users being the perimiter of a circle with the core group or person in the middle). The various service providers exist in relation to others within the circle. Service providers which rely on a vendor's services or products exist between the users and those who they rely on.

In this framework, influence is defined by the arc length around the perimeter which exists in the shadow or influence of the entity.

Another way to look at this is to imagine a somewhat conical hill, with the core at the top, and the users encircling the hill. The other suppliers forming a chain between them. These conceptualizations only provide a way of looking at a single moment in the community and neither the circle nor hill metaphores are static.

Gaining influence requres offering services which build a sub-community around one's services. These can include support, feature development, etc. The object of the game is to influence as many users in the community by getting them to use your features. Such influence leads to demand. Demand leads to revenue. The more influence one can have, in general, the more demand there will be for one's services.

What is different about this new economy is that one has the opportunity to influence the development of products to maximize one's revenues in ways which simply are not possible with proprietary products.

Business Value of Open Source

There is an interesting article on business value and open source software on Linuxworld.

The article is an interesting look into why one should consider participation in open source communities. There are a few things I would add though.

The first is that businesses which even use open source software, and whose sysadmins participate on the email lists, are influencing the development of the software in ways which are not possible with closed source software. Indeed the wall which exists in closed source software between support and research and development does not exist in open source projects. The core developers help with technical support, and such feedback allows them to make future versions better. This work also largely replaces market research in determining which features need to be included in the product. I.e. discussions on the email lists allow people to determine what the current users or would-be users need. This also allows successful open source projects to better meet the needs of their users.

Open source promotes a sort of community economy which is fundamentally different from the consumer economy it replaces. Personally I disagree with Richard Stallman regarding the dangers that closed source poses to open source. Indeed I cannot think of any instance where a closed source product has successfully replaced a dominant open source product in an established market. Yet there are plenty of examples where open source products have successfully challenged proprietary applications in certain markets.

Take for example Apache. The combined market share of Apache and the open source NCSA web server (which it replaced) has always been high, but not in all web server markets. Indeed, until recently, Apache was not considered a strong contender for web application servers (this market was held by Sun, Netscape, and Microsoft web application servers). However, in recent years, companies have begun to use Apache in this roll in increasing numbers. Apache is the clearest rebuttal to the concerns that open source is threatened by closed source appropriation of its source code because the license explicitly allows this. Yet none of the commercial versions are able to touch the marketshare of the original project.

A second, weaker, example is that of open source UNIX workalikes. This includes the BSD's as well as Linux, and have always been prevailent in environments where technical skills have been more prevailent than financial resources (such as servers for ISP's, and niche uses in such agencies as NASA). However, in recent years, Linux has moved into the mainstream server market, helping to marginalize Netware and proprietary UNIX. BSD varients are continuing to grow, though more slowly. Linux has also moved into embedded devices to threaten such operating systems as QNX, Embedded Windows XP, and possibly even Windows CE in the smartphone market.

Thus, the shift from closed source to open source, like the shift from feudalism or communism to capitalism, is a one-way process and cannot likely be reversed once the new system is established.

Open Source and the New Economy

What is Open Source and Free Software?

Open Source and Free Software is released under licenses which allow others to redistribute modified and unmodified copies of the software as well as derivative works. Some licenses require that these copies and works be distributed under the same or similar licenses, while others will allow the licenses to be changed.

The most successful open source software is developed and maintained by large communities or self-regulating networks of developers. This process decentralizes research, development, and engineering. Thus it creates an environment of collaborative development where many corporations and individuals participate.

Why does this challenge our notions of capitalism?

In traditional capitalism, natural persons and corporations get ahead by controlling as much capital as possible. Therefore one tries to maximize intellectual property, human and other resources, and working capital in order to obtain more leverage in a given market. In capitalism, the large corporations are the powerhouses of production, and nobody else can compete with them on their scale.

Decetralizing software development doesn't get rid of corporations, rich people, or other complaints that a few people have regarding capitalism, but it fundamentally changes the rules of the game. Indeed, the community gains much more market leverage than any one corporation due to the fact that even if it has no formal organization (such as a non-profit corporation), it still posesses vast intellectual property and human resource reserves which can marginalize competitors. This phenominon can exist regardless of whether the open source software can be resold under standard commercial licenses, as the Apache Web Server has shown. Therefore this is not a phenominon of one license, but rather a phenominon of the processes.

If open source methodologies spread to other fields of engineering, this may radically reshape our technology firms. For example communities of engineers could be designing computer chips, kitchen appliances, perhaps even jumbo jets. Such development practices would undermine much of the dominance that large corporations have in the markets today. For example, although a computer chip or a jumbo jet would probably require a large corporation to manufacture it, the actual design would be less dominated by the corporations and more by customers and users.

If Open Source is post-Capitalist then is it Communist?

In Soviet communism, the State owned and ran everything. Therefore I have often referred to it as "Feudalism backed by Marxist Propaganda." It was never a danger to societies where democracy and capitalism had become established. Indeed in the end, the Soviet Union fell and moved on to capitalism, and such market reforms are also underway in China (which seems to be in a transition into a capitalist economy).

The move from communism or feudalism to capitalism is the move from state control to corporate control. Rather than reversing this trend, open source and free software continue to the move towards more abstract and agile organizational methods. The communities which maintain the most successful open source software projects are continuously evolving in an organic way. The core developers may remain constant and continue to have final say regarding what goes into the project, but the community which uses and provides feedback and contributions to the project is dynamic and so the ability of community-driven open source projects to innovate massively outpaces even that of the largest software vendors, such as Microsoft.

This is an economic shift towards something new.