Softpanorama (slightly skeptical) Open Source Software Educational Society

May the source be with you, but remember the KISS principle ;-) Softpanorama Search

The Historical Dimension of "Solaris vs. Linux" Debate

Linux is just a part (and far from being the most important part) of open source movement, which became prominent with the advent of Internet. Open source movement is not something new. People in local communities, political parties, religious organizations, public television audiences, and ethnic groups do in fact often succeed in getting themselves organized and mobilized in pursuit of a public good for the group. Often the level of productivity is below the level that would be possible in for-profit enterprise; often there is some level of free-riding involved; but history alike are replete with examples of long-term and successful voluntary collective action. Programming is no exception.

It is important to understand that Linux is the second free Unix-like OS with the first being BSD. The Orwellian level of rewriting of history that exists in pro-linux camp and that also poisoned much of mainstream press (not without gentle IBM help) can be nicely illustrated by the fact that Linus Torvalds often considered to be a revolutionary despite the fact that Linux kernel is a straightforward (even somewhat backward) clone of Unix kernel, a 30 years old OS. Often this misinformation is coupled with another: that Linux was the first open source OS. Generally programming is not a place for blind devotion. As John Stuart Mill aptly put it long before computers were available:

There must be discussion, to show how experience is to be interpreted. Wrong opinions and practices gradually yield to fact and argument: but facts and arguments, to produce any effect on the mind, must be brought before it. Very few facts are able to tell their own story, without comments to bring out their meaning. The whole strength and value, then, of human judgment, depending on the one property, that it can be set right when it is wrong, reliance can be placed on it only when the means of setting it right are kept constantly at hand. In the case of any person whose judgment is really deserving of confidence, how has it become so? Because he has kept his mind open to criticism of his opinions and conduct. Because it has been his practice to listen to all that could be said against him; to profit by as much of it as was just, and expound to himself, and upon occasion to others, the fallacy of what was fallacious. Because he has felt, that the only way in which a human being can make some approach to knowing the whole of a subject, is by hearing what can be said about it by persons of every variety of opinion, and studying all modes in which it can be looked at by every character of mind.

It is very important to understand that before 1991, the year when Linus started his project, several books with in-depth coverage of Unix internals were published. Source code was also widely available:

• 4.3BSD NET/1 tape was released in November 1988
• NET/2 tape was released in June 1991, just three months before Linux 0.01. NET/2 tape was almost complete well debugged open source implementation of Unix (just six files were missing for copyright reasons).
• Six months later Bill Jolitz had written replacements for the six missing files and released a fully compiled and bootable system for the 386-based PC architecture which he called 386/BSD. He also published source code in Dr. Dobbs Journal in a series article starting from early 1991. For more information about the first ten years of Linux history see Bezroukov2005a.

Sun shipped its first UNIX-based Sun-1 workstation in 1982: almost ten years before Linus Torvalds decided to replicate Unix kernel as a "hobby/vanity fair" project. In 1986 Sun (for a short period of time) produced Intel 386 based Unix workstation [Sparcproductdirectory2004]:

If, like me, you were looking for a well integrated Unix platform on Intel Architecture in 1986 you could do a lot worse than choose Sun's 386i. It ran SunOS and some DOS applications in a of Intel development. compatible window, and it had slots for PC-AT disks. There the similarity to a PC ended. Instead of low resolution PC graphics it had high resolution Sun graphics. Instead of low performance disk drives it came with high performance SCSI disks - and a lot more RAM than the PCs of that vintage. Although the 386i was rubbish as a graphics workstation (the graphics was too slow) and despite the fact that the DOS emulation was too slow and useless to run most DOS applications - the 386i was a low cost entry level database server which could outperform Sun-3's and low end VAXes for a fraction of the cost. When I first signed up as a Sun VAR in the 1980s - it was the 386i which I used as the Trojan horse to replace our aged Zilog boxes and eventually displace the HP and DEC minis. We were using Motorola's VMEbus SBCs as our main real-time platform - but needed a Unix with a future to coexist with our multiprocessor applications. Although the first SPARC systems had come out - the original Sun-4's didn't inspire any more confidence in me than the myriad of other RISC based Unix servers which were hitting the market at the time. Sun was supporting 3 hardware platforms with its SunOS at the time:- Motorola, SPARC and Intel. My guess was that Sun's long term strategy would be to run with Intel. I was right - but there was to be an 18 year discontinuity before that became a firm reality.

When we compare Solaris with linux it is important to understand that historically Sun has a very impressive record of making contributions to the community. Among the most notable contributions we can mention:

• The idea "the network is the computer" - many pieces of hardware, some of them mutually incompatible, accessible as one system via common protocol (TCP/IP). The acronym "SUN" means "Stanford University Network". Please remember that this idea was introduced in 1981 with SunOS, long before WWW revolution that from 1994 gradually make TCP/IP available to everybody.
  

• Virtual filesystem interface. Sun OS 2.0 offered a filesystem independent interface to concurrently support several different filesystems within an OS instance. This interface, today know as the vnode/VFS interface was created first of all to accommodate NFS. UFS was modified to accommodate this new vnode/VFS interface framework.
  

• /proc virtual filesystem (which originated in Plan 9 but was introduced into Unix by Solaris). The idea of using filesystem interface to expose and manipulate system blocks now is standard part of any Unix. So called "proc" utilities introduced by Solaris also proved to be incredibly useful. Both pgrep and pkill are real time savers for system administrators. Less known but still interesting utilities include pstack to show you a process' stack (for each thread), psig to see the signal handlers for a process, ptree to draw an ASCII chart of a process' parents and children. The latter you need only occasionally, but when you need them, there is no substitute.
  

• Loadable kernel modules (SunOS 4.x was the first Unix to offer them). This was one of the reasons that Solaris typically was shipped without a C compiler.
  

• RPC -- which is a revolutionary way to distribute computation in Unix, the infrastructure that made such important Unix achievements as rsh, NSF, NIS, ssh and many other protocols possible.

  • NFS -- we do not need to discuss the importance of NFS. It's self-evident. I would just mention that concept of "floating" home directory pioneered by Solaris: in Solaris home directories are in /export/home not in /home like in linux. If this is an NFS-exported filesystem, you can mount it on /home directory and work with the same home directory on different servers -- a pretty advanced for early 90th concept introduced by Sun, the part of their "network is a computer" approach.
    

  • NIS -- the first and pretty successful attempt to merge administration of multiple Unix server into some kind of "superhost" environment. Was widely used in university environment and essentially pioneered ESM (enterprise system management) tools which are specifically designed to make Unix more manageable in large enterprise environments. Now by-and-large replaced with LDAP.
    

• PAM -- this is standard way to implement complex authentication schemes in all modern Unixes and linux. PAM was first proposed by Sun Microsystems in an RFC 86.0 dated October 1995 (see The original PAM RFC). As authors of PAM - InstallationWiki state:

  The history of PAM goes back to 1995 when developers from Sun Microsystems implemented a generic framework for Solaris. When Solaris 2.6 was released in August 1997, PAM was an integrated component of the operating system. Ever since then, Solaris has been using PAM for authentication. In February 1997, the Linux-PAM project began, and most GNU/Linux distributions today are using PAM.

• Packages for binary software distribution. In 1992 Solaris adopted a SVR4 mechanism for "standardizing" the installation of optional software called SVR4 packages. Very quickly all Sun software was packaged with this format. Most vendors switched to this format in Solaris 2.3. Packages can be installed/deinstalled with pkgadd/pkgrm utilities. Note that the "pkg" system introduced a lot of new concepts which later were adopted in linux packaging schemes, for example the concept of packages database (in /var/sadm/install), the concept of checking of installed package for integrity and many others. A database was also used for clean removal of packages. Here how the "pkg" concept was described in early version of Solaris FAQ dated 16 Jun 93:

  • What are "packages"?

    A SVR4 mechanism for "standardizing" the installation of optional software. Most vendors are expected to use this format for distributing add-on software for Solaris 2.x.

    Packages can be installed/deinstalled with pkgadd/pkgrm which are standard SVR4 items, or with swm (CRT) or swmtool (GUI-based) which are provided only in Solaris 2.

    Note that the "pkg" system keeps lots of files in /var/sadm/install, and in particular the file "contents", which is hundreds of KB, and that there are two copies of it while pkgadd is running, so you needs lots of free space where /var is, typically the root. This file must be kept around if you want, for example, to use pkgrm to remove a package, or pkgchk to verify months later that all of a package's files are still intact.

  RPM packaging format was created in 1995 by Marc Ewing and Erik Troan for Red Hat distribution. RPM was in turn based on several earlier linux packaging systems but soon it became the most common way of packaging software for linux and until apt was created dominated among linux distributions. It replicated the basic functionality of SVR4 packages with some extensions Its rapid adoption in linux distributions as well as the creation of alternative formats like apt can be considered to be the testament of success of Solaris-pioneered approach to binary software packaging for Unix. Some information about history of RPM is available from Maximum RPM
  

• Open Office (if this can be considered an innovation). But in a way it was, despite the fact that the product was bought by Sun: due to the fact that Sun released the codebase under an open source license Open Office instantly become the only credible free Office environment available for both Unix and linux and the only environment that in some respects is competitive with Microsoft Office.
  

• The ides of a uniform scripting language for Unix environment. While this idea of universal scripting language for Unix was killed by Sun brass which, unfortunately fell in love with Java, the company supported the development of TCL and the funded the conversion of TCL from a university-based project into full-fledged production strength scripting environment. This was a very sad story of innovation killed by a corporation in a way similar to the history of treatment of VM/CMS by IBM brass.
  
  It's really unfortunate that both Solaris and linux remains extremely archaic in this respect and that no Unix flavor provides standard, universal for most applications scripting language like REXX in OS/2 or VBA in Windows. In this respect it is honest to state that both Solaris and linux should be considered backward, not revolutionary environments (actually linux can be considered a backward, outdated OS in more then one respect as we will discover later).
  

• The first successful commercial implementation of light-weight virtual machines pioneered by FreeBSD. Sun significantly extended this concept in Solaris in comparison with FreeBSD implementation and due to this Solaris 10 be called the first XXI century Unix. Solaris zone cleanly solve the very important for large enterprise environment problem of "root hell" when root access is shared between multiple "application owners" and system administration group (which in turn can consist of a dozen of people). As a result left hand often does not know what right hand is doing. The main problem in UNIX security is not that root is too powerful, but that a regular user account as not powerful enough and that for many tasks application owners need to be able to switch to root. Giving application owners a separate zone with full root privileges creates much nicer, development friendly environment, the environment currently which gradually is replicated in other Unix flavors much like NFS and /proc subsystem (AIX 6 was the second OS after Solaris to implement this approach; Red Hat was the third).

• Coming from Stanford experience Sun was the first vendor committed to shipping servers with "no proprietary network" - all networking was TCP/IP-based networking and was available in Solaris from day one.  Actually Solaris was the first OS in which detaching and re-attaching networking cable on a working server was a trivial exercise. Sun servers have high interoperability and due to this they have been all over the Web in 2000 just before dot-com crash. Linux TCP-networking was added as an after-thought.

This list definitely can be continued but the statement that I want to make is that rumors about linux being an innovative OS are greatly exaggerated :-). Moreover Linux track record in innovation looks pale even in comparison with other major branches of commercial Unixes each of which also introduced important parts of modern Unix infrastructure that linux reimplemented. For example AIX introduced concept of volume manager (licensed from Veritas but actually pretty native for all IBM Operative system products with idiosyncratic distinction between physical and logical drives which comes from the days of OS/360) and one of the first really successful in production implementations on Unix of paravirtualization (Lpars in AIX).

I put this example to show that the level of distortions of history typical in many publications devoted to linux is such that for a reasonable computer science literate reader it should immediately produce an allergic reaction similar to the reaction many native English speakers get reading a paper with just too many grammar and spelling errors (this one is a nice example of the genre ;-).

For those who got their education of Unix history from various attempts to rewrite it in best Bolsheviks traditions it is important to remember that linux actually belongs to the dozen of Unixes created for Intel with previous major implementation being Microsoft Xenix, which actually created all the necessary for Unix on PC logical infrastructure including partitioning schemes, boot managers for dual boot and console switching (idea of multiple virtual consoles via Alt-F1, Alt-F2 and so on belongs to Xenix). Linux also just one of several free Unixes for Intel with the other major flavors being FreeBSD, OpenBSD and NetBSD.

I would like to stress it again that it was Microsoft XENIX not linux, which created most of the infrastructure for Unix on Intel including the critical mass of books. Whether we like it on not linux owes much of its success to Microsoft: it was XENIX which provided all (yes, all) the major technical solutions and infrastructure used by each and every subsequent Intel Unix implementation (for example the use of Alt-F1,F2, etc for console switching was a feature introduced by XENIX ). Moreover even later after abandoning Unix in favor of OS/2 Microsoft indirectly subsidized all Unixes on Intel as the de-facto owner of PC standard: hardware that any Intel based Unix is running on is created by OEMs using the standards that Microsoft license for free to all PC manufactures and the cost of this hardware is mainly determined by the size of the market created by Microsoft OSes. Plug and play hardware specification is a nice example of Microsoft contribution to linux success here. Whether we like Microsoft or not, the simplest and reasonably precise definition of PC always was "Microsoft compatible computer". As the most recent example it was actually Microsoft who politely and firmly explained to Intel that it should provide hardware compatibility with Opteron and not to reinvent the 64-bit extensions wheel. For Intel breach of relations with Microsoft was too serious threat to ignore. That's just one example of how Microsoft provides and defends unification and standardization of hardware platform often mistakenly called Intel-platform, but which properly should be called Microsoft-Intel platform, the platform that linux uses for free without any investments. In this respect one can state that linux is just a side effect of Microsoft success in hardware. The undeniable fact is that linux is critically dependent on Microsoft-subsidized hardware and without Microsoft success there would never be any linux as mass, supported by such companies as IBM and HP, operating system. I think that Linux Towards should seriously consider adding Microsoft to the list of kernel contributors ;-).

Sun fought Microsoft-Intel alliance on his own UltraSparc ground. That was a heavy uphill battle that left many scars and Sun did not succeed, but it fought Microsoft-Intel dominance straight on and until recently, if we allow somewhat politically incorrect word, did not tried to co-habilitate Microsoft subsidized hardware. That was a completely different game, much more complex and dangerous but if we like to use high words it was real fight for the freedom (unlike Intel CPUs UltraSparc was/is an open sourced CPU). In a way, typical linux advocates rumblings (including some Linux Torvalds and Richard Stallman's interviews) are completely hypocritical -- it you love freedom so much why on the Earth you use closed proprietary CPU when a free open-sourced CPU is readily available and even does not cost that much.

Please also do not forget that linux is now fifteen years old OS that has a lot of baggage and the development is no longer as nimble and fast as it used to be. Most core developers became tired and disillusioned "old guard". By all CS standards, fifteen year is an old age for any OS, so developers who participated in it from day one approach the age when some sclerosis and arthritis should be expected :-). And change of the guard in free/open source development is a very complex and painful process as BSD experience (which is an even older OS which remains the trailblazer for all newcomers into free OS arena and as patriarch of the field experienced signs of sclerosis both in architecture and key developers minds before it hit linux) can attest. Days when Linus Torvalds will be just another retired multi-millionaire with his own (smaller then Larry Edison's :-) yacht might be closer then many linux fans think...

Historically Sun's assault of IBM's mainframe business was one of the major reasons why IBM became so interested in Linux and tried to save his mainframe franchise from extinction by implementing VM/Linux project. For IBM Linux was a brilliant counterplay of Sun encroaching into its mainframe turf. Idea of running Linux via tried and true VM instantly gave a new lease of life to the dying, but still very good technology. And that's why the major push in IBM adoption of Linux was initially directed toward VM/Linux -- a bold and reasonably successful attempt to stop sliding of the market share of the oldest IBM platform into oblivion. That essentially saved such an amazing (and underappreciated by almost everybody, including old IBM brass) OS as VM. Later VM virtualization technology was also ported to AIX. Currently IBM tries to emphasize Linux on Power ...

The last but not least historical distortion is the linux work is often presented as a work of volunteers. This is hypothesis open to review for all but the earliest years of linux development. A recent report from the Linux Foundation reveals the extent to which linux development looks like a corporate cooperative initiative:

• 73.2% changes to the kernel came from employees working on behalf of their companies ( Red Hat, Novell, and IBM - accounted for 28.4% of all the changes),
• 13.9% came from volunteers without a corporate affiliation,
• the remaining 12.9% of changes came from developers whose affiliation is unknown.

Report states: "It is worth noting that, even if one assumes that all of the 'unknown' contributors were working on their own time, over 70% of all kernel development is demonstrably done by developers who are being paid for their work."

That brings us to the ideological dimension of Solaris vs. linux debate.