Softpanorama (slightly skeptical) Open Source Software Educational Society

May the source be with you, but remember the KISS principle ;-) Softpanorama Search

Solaris vs. Linux: Framework for the Comparison

by Dr Nikolai Bezroukov

5. Hardware: SPARC vs. X86

The strongest selling point of linux for enterprise customers is actually not the OS per se, but the Intel-based hardware on which linux is running. Sun is looking good here. Sun's response to dot com bubble burst was to invest in R&D while the market settled. Part of this research (and acquisitions) was to get the best Opteron based technology - and today they are very visible in the high end Opteron market ( Dell only recently (mid May, 2006) declared that they will sell Opteron systems too). 

But let's look at the situation from a computer science standpoint. In uniprocessor mode Intel (or Opteron CPUs) are much faster and much cheaper. Any look at SPECINT benchmarks leaves no doubt about the current situation.

But UltraSparc CPUs are more fault-tolerant and on Sun servers are better integrated with the rest of hardware. while really expensive in comparison with Intel Duo Sun midrange servers as usually handle well  the most taxing enterprise loads. They are usually reasonably scalable and the necessity to move to the high-end servers is slightly less.

In the past Sun used to have better positioning in its Sparc line and this positioning was greatly eroded by recent lines of Intel and AMD CPUS (the same face waits IBM Power line and Itanium line). As John C. Shoemaker (Sun Microsystems's Executive Vice President and General Manager of Computer Systems from 1995 to 2002) recollected on his article A personal view of Sun Microsystems:

SPARC was never competitive. Its RISC architecture was an advantage in technical applications, and it made certain tradeoffs in input/output and cache sizes to achieve this targeted application success; however, it was not a competitive general purpose uni-processor, ever!

It was always about the system for Sun, not the chip. The system targeted technical application. When we pushed to commercial markets, it was through the implementation of a highly scalable vertical architecture, taking full advantage of a well-aligned strategy with the software developers at Sun. They were, and still are, brilliant at creating an amazingly robust and highly scalable OS (Solaris). No competitor could come close to Sun. We were able to attack aggressively and take the commercial market with system price/performance leadership on an open platform, running basically all (well over 12,000) major commercial and technical industry applications.

At the point in time when we began to achieve significant wins in commercial account applications, we acquired Cray Business Systems Division from Silicon Graphics (SGI) in 1995. Cray had previously chosen SPARC/Solaris, on a level playing field basis, as their best bet for entry into the commercial market. Cray BSD brought a world-class technical team to Sun, one with specific expertise in data center class systems design and quality requirements. This acquisition brought Sun a breakthrough 64-processor product with dramatic industry leading reliability, availability, scalability, and cost performance. Intel, IBM, and others had better uni-processor performance but, because they did not have Solaris, no one could scale above three or four processor systems. Microsoft made many promises to customers about future scalability, but those promises turned into nothing but delays.

Linux is essentially an OS that flourishes only on Intel. It never achieved any significant success on RISC architecture although IBM now is trying to change this situation pushing Linux on Power5 and potentially cannibalizing its own AIX sales (at least on low level servers).  Actually running several competing with each other hardware and software offerings is nothing new for IBM.

If you look on the general quality of the servers, Sun is a solid engineering company and Sun hardware engineers has a small but devoted camp of followers even on Intel platform (with Opteron line of servers), to say nothing about UltraSparc enthusiasts. Reliability record of Sun servers is excellent.  Many enterprise Sun servers work without any issues for 10 or more years.  For non-critical systems generally you can run them without hardware maintenance contract with Sun if you have one spare model on the shelf. In this case you just replace the server if something happens. This way serious money can be saved in maintenance. Even small Ultra 5 and 10 desktops are sill running as personal workstations in many organizations. And nobody is buying any hardware support for them.  

5.1.  UltraSparc is an expensive, slow,  but pretty cool CPU :-)

While an excellent CPU with advanced architecture, UltraSparc has higher price/performance ratio then either Intel or AMD Opteron CPU offerings. This situation slightly improved with the introduction of T1 CPU[Wikipedia] but newer Intel offerings does not leave Sun much chance.   They are really open processors: on March 21, 2006, Sun made the UltraSPARC T1 source code available under the GPL.

The main advantages of UltraSparc are:

1. Energy efficiency: it consumes less energy then either Intel or Opteron and much less then IBM's Power5 CPUs.  This is probably not enough to make them noticeably more attractive for server farms in hosting environment where the margins are extremely thin and you need  many cheap units as well as the ability to upgrade them without incurring huge capital costs, but it is incentive for some large companies datacenters that run into air-conditioning or electricity supply limits to their growth as the move to a new building in such cases is a very expensive proposition.

2. Fault tolerance. Sun server hardware can do amazing things with faulty components calculating the number of faults pr 24 hours.  Almost any duplicate component can be switched off. Even low level Sun server like V240 can automatically disable bad memory chips, one faulty CPU or one burned power supply. In some sense it is close to a cheap cluster, but without cluster software complexity. And that's alone (long with the reduced complexity) worth quite a bit of money (I hate cluster software ;-)
   
   Sun servers also behave reasonably well after overheating: they shut down gracefully and usually are able to restart after cooling-off without any problems. That's important as central aid conditioning can go south in large datacenters in summer (sometimes due to electricity supply problems) and even with reserve generators might represent the weakest link as practice show that those reserve generators are not necessary switch on after the event :-).
     

3. Cleaner architecture. Being big Endean CPU with RISC instruction set provides some complier level advantages in comparison with convoluted instruction set of X86 line. From CS point of view (and complier writers point of view) X86 CPUs architecture and instruction set are a horrible mess.
    

4. Better hardware-based stack protection. From the security standpoint UltraSparc permit implementation of pretty tight stack overflow protection. Solaris 10 on Opteron also has this advantage  (in 64 bit edition. ).
    

5. Partitioning. Similar to high-end Sun SMP systems, the UltraSPARC T1 can be partitioned under Solaris 10.  Thus, several cores can be partitioned for running a single or group of processes and/or threads, whilst the other cores deal with the rest of the processes on the system.

You can also use Solaris 10 on Opteron or Intel, but you need to be careful with the compatibility list and need to buy certified hardware from sun, dell or HP: currently the level of support of  PC hardware in Solaris for Intel is far more limited that in Linux. Not that it is great on Linux as "barely working" driver layer is not acceptable for enterprise severs).  Also quality of some drivers for Linux leaves much to be desired. Sun while supporting less hardware is more strict with quality of the drivers: when Sun does provide drivers it provides reliable drivers. 

Also there were periods when Sparc hardware were price-competitive with Intel offerings and such periods might repeat in the future. Paul Murphy  stated this consideration the best in his March 2005 article "Face-off: SPARC/Solaris vs. Intel/Linux" [Murphy&Golden] :

Suppose, for example, that back in mid-2000, you installed a pair of Sun 450s running Oracle, each with two 327 GB T3 workgroup arrays, 4 CPUs at 450 MHz, and 8 GB of RAM. Taken together that might have cost you around $700,000 initially, and something like $2,900 a month for hardware support after the first year. Now, five years later, users are complaining about those servers' performance. Also, some of your colleagues are saying you'd get a six-month payout just on support costs by replacing this stuff with a couple of Dell boxes running Linux. They add that you'd get an enormous performance boost into the bargain.

Your colleagues are about half right on the money, absolutely right about the performance opportunity and dead wrong about the implicit exclusion of SPARC/Solaris from the operation of Moore's Law.

That dual 450/T3 combination gave you a good bang for the buck in its day. Do a detailed review of the options available to replace it today, and you'll find that vendor pricing relationships haven't changed much on, say, Dell's 6600 quad Xeons, Sun's V40z quad Opteron, IBM's newest PowerRISC 720, or Sun's 440 UltraSPARC.

Back in 2000, both Compaq and IBM had gear that could compete on performance, but not on price, with the Sun 450/T3 combination. Today that range is narrower with Sun, Dell and IBM all clustered around $40,000 for a single system capable of handling the entire load. Even so, the cheapest and fastest options are still from Sun.

The fastest option, built around Sun's four-way Opteron V40z, is also the cheapest from a hardware perspective. At about $38,000 inclusive, it should handle the load formerly shared by the two 450s while cutting response time in half. The most expensive option is probably the Dell 6600 (I could not get fully configured pricing on the IBM 720 from IBM's Web site) at about $45,300 inclusive.

The most reasonable option, the Sun 440, is neither the cheapest nor the fastest at about $41,000 and including a 3310 external disk array. It has, however, the benefit that you can put it in place without procedural, technical or software change.

If we talk about openness (the pet theme of Linux lovers), then SPARC architecture in Sun's systems is more entitled to be called an open architecture than is anything that comes from Intel's fabs. And it's a pure hypocrisy (similar to Linus Torvalds despicable from hardware design openness standpoint stance in Transmeta) to ignore this fact for any open source evangelist. 

Intel's microprocessors are more of an industry commodity than a standard. The term standard should be reserved for specifications that have received the imprimatur of an officially recognized standards body such as ANSI, the IEEE, the ISO, the IETF, or the W3C. When a technical specification for a protocol, programming language or other technology receives the endorsement of one of these organizations, that specification is considered to be ratified as a standard.  From this point of view SPARC is the industry standard, Intel X86 is not and will not be in the foreseeable future.

Moreover while slower then recent Intel offerings, architecturally UltraSparc is a very interesting microprocessor. And as we mentioned before it is a big Endean microprocessor, which is actually the only right way to build microprocessors :-). And as the amount of memory increases beyond 4G it has some advantages due to Alpha-style organization of memory bus. Old, 32 bit Intel servers scale very weakly beyond 4G of memory. UltraSparc scales to 64G without major problems. Interesting benchmarks for T1 can be found at  http://www.rz.rwth-aachen.de/computing/hpc/hw/niagara.php  and related slides http://www.rz.rwth-aachen.de/computing/hpc/hw/niagara/Niagara_ZKI_2006-03-31_anMey.pdf

And the ability to access huge amount of memory is more important then raw speed in many typical enterprise applications. If we are taking about memory intensive applications like databases doubling the amount of RAM always wins doubling of the number of CPUs.  Most organizations usually overpay CPUs and underpay for memory when ordering new servers. There are just too many enterprise servers that are completely unbalanced CPU/memory wise and/or over-specified to the extent that they run with CPU utilization of less then 10%. Generally it usually does not make much sense to order less then 4G per CPU on a typical transaction oriented servers. For example to order server with "top of the line" (at the time of writing) two 3.6GHz Intel X86-64 CPU and just 2G of memory while similar priced server with 3 GHz CPUs (which are twice cheaper then 3.6GHz CPU at the time of writing) and 8G of memory (that can be bought for a difference in CPU prices)  has much higher price performance ratio on all major transaction-oriented enterprise applications. If we are talking about Oracle and sizable databases, 16G of memory typically is more optimal then 4G and produce the speed-up that easily compensate much slower speed of CPU (or less number of CPUs).

Also if you think about it Intel 8086 design was extended for way too long...  AMD has done an excellent job of extending it to 64 bit but still old warts of initial 16 bit design are still present.

In is not accidental that in 2001 Moreover UltraSparc III  got prestigious Analysts Choice Award for Best Server/Workstation Processor by Microprocessor Reports):

PALO ALTO, CA -- January 22, 2001 -- Sun Microsystems, Inc. (Nasdaq: SUNW) today announced that its 64-bit UltraSPARC[tm] III processor received MicroDesign Resources (MDR) Microprocessor Report 2001 Analysts Choice Award in the Best Server/Workstation processor category. Sun Microsystems accepted the award at MDRs second annual awards ceremony, Thursday, Jan. 18 at the Hyatt Saint Claire in San Jose, Calif.

The MDR awards confirm the top technology picks from the Microprocessor Reports analyst team, recognizing excellence in technology innovation, design and implementation. The Sun Microsystems, Inc. UltraSparc III processor won first place for Best Server/Workstation Processor in a competitive field of nominees which also included Intel's Itanium, IBM's Freeway for eServer z900, and Intel's Pentium III Xeon processors.

The prestigious MDR awards celebrate the companies and products that shaped the electronics industry in 2000. Microprocessors for servers and workstations do the heavy lifting in this industry. The immense processing requirements placed on this class of processors require them to incorporate many advanced features, said Max Baron, editor-in-chief and principal analyst, Microprocessor Report. Because of its advanced multiprocessing architecture, we gave the Sun UltraSparc III processor the Microprocessor Report 2001 Analysts Choice Award for Best Server/Workstation Processor.

Partially due to UltraSparc CPU architecture Solaris on UltraSparc scales very well on multi-CPU machines, compared to Linux. After 8 CPUs the difference in quality of SMP implementation became really noticeable. Also you can add the advantages of Solaris 10 being a 64 bit OS, with large max file sizes and the ability to access huge amount of RAM without any special hacks, then the advantage of Solaris for mid-range Opteron servers became more prominent. Again I am not talking here about running workstation loads or Web servers, I am talking about typical mid-range servers loads in large enterprises, typically SAP and database related loads.

On a negative side Solaris 10 on UltraSparc suffers from lower performance of classic UltraSparc CPUs in comparison with Intel and AMD CPUs. Only new T1 chips from Sun are competitive on transactional benchmarks (as they have 6 or 8 cores). For all major practically important enterprise applications T1 based computers and high end Fujitsu UltraSparc compatible CPUs are the only one  competi tive with Opteron. On SPECint2000/SPECfp2000 benchmarks only Fujitsu UltraSparc compatible CPUs are competitive (T1 consists of several relatively slow cores  - 1GHz each -- so without multithreading it provides less advantages).  That puts Sun in some disadvantage on the low and midrange servers (up to 4 CPUs).

The code density is lower  on Sparc compared to X86 family of CPUs (32-bit or 64-bit), but there is no need to exaggerate: it is no longer critical and much depends on the application and compiler. GCC in its current form (versions 3 line)  makes the situation much worse.
 

5.2. Solaris on Opteron

Now let's briefly disuses Sun Opteron line of servers. Here the price advantage of Dell is much less that on UltraSparc and  for two CPUs servers Sun competes with Dell (and HP) neck-to-neck both in the price and support costs. Moreover you can buy "bare-bone" Opteron server (for example bare bone X2100 with one Opteron 146 2.0GHz CPU and 512M of memory costs less then $800) and install additional memory and hardrives yourself saving some percentage of the cost (sometimes one third).  With Dell supporting Solaris on its key Power Edge line the difference in price is zero although reliability of drivers can still be an issue (that makes Sun servers safer bet for large enterprise)

HP probably represents middle ground.  HP track record as for drivers is better then Dell and many Opteron-based servers from HP are Solaris 10 compatible. HP's support matrix for 64-bit Solaris 10 can be viewed here.  In Feb 2, 2006 article Register published an interesting article HP confirms plan to attack Sun via Solaris  which stated the following:

HP has long "officially" supported various versions of Solaris on its Xeon- and Opteron-based servers. Now, however, it's kind of ready to talk about this support. The company this week "announced" support of Sun's version of Unix in a statement to staff, according to insiders.

... ... ...

"Enabling 64-bit Solaris 10 on Opteron-based ProLiant servers is an extension of HP's Sun Attack program - a way to provide a solution for customers who are interested in moving from Sun/Solaris to an Industry-Standard HP solution."

... ... ...

Since reviving Solaris x86, Sun has now managed to push HP, IBM and Dell into admitting that they will sell the OS if customers ask for it.

HP has long been the most able Solaris x86 supporter. This is largely a result of a decent sized Solaris x86 business at Compaq. IBM follows with its recent agreement to let Sun service Solaris running on Big Blue's blades. Dell only ships Solaris when customers order a ton of servers and demand the OS.

The impressive Solaris x86 download figures quoted by Sun and buzz around Open Solaris seem to center on academic users and hobbyists. This makes it difficult to tell how many large customers have decided to run Solaris x86 in production. If you are such a customer, please let us know.