Mistakes made because of the differences between various Unix/Linux flavors or configurations

News Sysadmin Horror Stories Recommended Links Creative uses of rm Mistakes made because of the differences between various Unix/Linux flavors Missing backup horror stories Lack of testing complex, potentially destructive, commands before execution of production box Pure stupidity
Locking yourself out Premature or misguided optimization Reboot Blunders  Performing the operation on a wrong server Executing command in a wrong directory Side effects of performing operations on home or application directories Typos in the commands with disastrous consequences Side effects of patching
Multiple sysadmin working on the same box Side effects of patching of the customized server Ownership changing blunders Dot-star-errors and regular expressions blunders Excessive zeal in improving security of the system Unintended consequences of automatic system maintenance scripts LVM mishaps Abuse of privileges
Safe-rm Workagolism and Burnout Coping with the toxic stress in IT environment The Unix Hater’s Handbook Tips Horror stories History Humor Etc

Introduction

In most case differences between flavors of Unix are just annoying but in some case then can became source of serious errors

Differences between flavors of Linux such as Red Hat and Suse can also lead to unintended consequences. And some of them are really devious because the working assumption for sysadmin is those two flavors of Linux are "almost identical".  For example, if  in one of those flavors a particular daemon  is started via RC scripts and in other via xinetd. Often people even do not understand that different methods are deployed and start manually daemon which should be started via   xinetd.

Similarity differences in /etc/profile and /etc/profile.d  as well as customized /root/.bash_profile and /root/.bashrc  can play nasty tricks. For example by default in RHEL command rm is aliased to rm -i. If on a new system that you are working this alias was deleted and you rely on it issuing command rm resolv.* intending to delete only old versions of the file interactively,  you are up to a nasty surprise: you will not see any prompts and all files will be deleted immediately. That's why many qualified sysadmin think that it is a bad idea to use aliases with  the same name  as the command itself.

Many qualified sysadmin think that it is a bad idea to use aliases with  the same name  as the command itself.

Also RHEL and SUSE has different security mechanisms in place. The SUSE administrative tools seem to be  superior to Redhat (WebYast,  Yast2), while authoyast is vastly inferiors to kickstart (and was way too buggy until SLES 12, when Yast was rewritten in Ruby).

But real fun starts when scripts designed and tested on one flavor of Linux or Unix are moved to the other. Here even innocent differences in location of files such as /var/log/messages on Suse and Red Hat  vs. /var/adm/messages on Solaris can led to interesting side effects.  Another good example is that on HP-UX df command produces completely different output then on all other flavors of Unix.  You need to use bdf to get similar output.

In RHEL 7  directories /bin, /lib, /lib64, and /sbin, were converted to symbolic links that point to corresponding directories in /usr hierarchy.

lrw|rw|rw|.   1 root root    7 Sep 25  2017 bin -> usr/bin
dr-xr-xr-x.   4 root root 4096 Sep 25  2017 boot
drw|r-xr-x.  18 root root 3120 Sep 24 22:48 dev
drw|r-xr-x.  87 root root 8192 Sep 26 03:49 etc
drw|r-xr-x.   3 root root   22 Sep 24 22:48 home
lrw|rw|rw|.   1 root root    7 Sep 25  2017 lib -> usr/lib
lrw|rw|rw|.   1 root root    9 Sep 25  2017 lib64 -> usr/lib64
drw|r-xr-x.   2 root root    6 Nov  5  2016 media
drw|r-xr-x.   4 root root   35 Sep 24 22:48 mnt
drw|r-xr-x.   3 root root   16 Sep 25  2017 opt
dr-xr-xr-x. 124 root root    0 Sep 24 22:47 proc
dr-xr-x---.   5 root root  201 Sep 25 05:51 root
drw|r-xr-x.  26 root root  860 Sep 26 03:49 run
lrw|rw|rw|.   1 root root    8 Sep 25  2017 sbin -> usr/sbin
drw|r-xr-x.   2 root root    6 Nov  5  2016 srv
dr-xr-xr-x.  13 root root    0 Sep 24 22:48 sys
drw|rw|rwt.   8 root root  108 Sep 28 03:40 tmp
drw|r-xr-x.  13 root root  155 Sep 25  2017 usr
drw|r-xr-x.  20 root root  282 Sep 24 22:48 var

That means that check for those directories using  -d  test condition in bash will fail as they now are symlinks, not directories.

Default locations of many classic Unix utilities differs form one Linux  flavor to another,  and from Linux  to Solaris or HP-UX.  If the script uses absolute path some commands might never be executed. Just because  the particular script works one flavor of *nix does not mean, that it will work for another.

That also means that you should never ever assume that some prepackaged script that you downloaded does anything right. Running such script without testing can be quite suicidal as they do not always check if particular cd operation is successful.

Sometimes scalability cause serious problem. It's one thing to run a particular script on the server with 10 users and the other with 10000 users:

...Management told us to email a security notice to every user on the our system (at that time, around 3000 users). A certain novice administrator on our system wanted to do it, so I instructed them to extract a list of users from /etc/passwd, write a simple shell loop to do the job, and throw it in the background.

Here's what they wrote (bourne shell)...

for USER in `cat user.list`;
   do mail $USER <message.text &
done

Have you ever seen a load average of over 300 ???

Subtle differences between bash and ksh

Note: You might be better off forgetting all this silly noise about compatibility and using bash. Bash is now standard de-facto and is available of almost all enterprise flavors of Unix and commercial Linuxes.

It is still not available on HP-UX 11 by default, but you can install it from a depo. Anyway even they provide bash with binaries available for all HP-UX versions.  It is actually much easier to install bash on all systems that struggle with shell compatibility questions.  Solaris, AIX and linuxes have bash installed out of the box and now bash in the natural least common denominator of shells.  Moreover bash 3.xx is pretty close to ksh93 and if you tested you scripts with it, in most cases the script should work OK.  That main difference is treatment of the last stage of pipeline, but now there is bash option via which you "enforce" ksh behaviour (which is actually the only rational behaviour ;-).

Originally sh was Born shell that has a separate implementation. Those days sh is often implemented as a special compilation of existing ksh version (or bash)

The second problem in writing portable script is that there are also many subtle differences in utilities provided and their location for various flavors of Unix. There are several ways to deal with this problem. One is to provide symbolic link to the "most reasonable location", the second is to provide wrappers for all command which are set by which command at the  start of the script or in some other way (such wrapper can also provide debugging mode substituting the command for echo which is a very useful feature)  and the third that probably the most popular is to encode path and the name of the utility in shell variables.

Both the Bourne shell, and the Korn shell, can use the semicolon and the carriage return interchangeably in their syntax of the if, for, and while built-in commands.  When using the brackets ( [ ], [[ ]], (( )) ) within if commands, you must separate both inside ends of the brackets from the inside characters with a space. The only exception is the semicolon which can be used directly after the closing  bracket, for example

if (( $i -- 0 )); then 
   ... ... ...
fi

Shell Script Porting Guidelines

Here is a semi-useless table from the FAQ that compares shells.  It is old and from purely academic point of view is incomplete as it does not include ksh93 which is in many respects was the pinnacle in the development of the traditional Unix shells (Bash 4.x now emulates most of those features too) .

But again you need to forget about academic discussion here: in most case it is simpler to switch to bash (by pre-installing it on all your systems) then struggle with all this level of complexity. This si Syssifis labour, no more no less. Bash is not perfect (and in some areas is crappy like default treatment of the last state of the pipe).  but it works well enough on all Unix flavors to suit your needs. Sometime the best way to conquer obstacle is to go around it :-)

                                    sh   csh  ksh  bash tcsh zsh  rc   es
Job control                          N    Y    Y    Y    Y    Y    N    N
Aliases                              N    Y    Y    Y    Y    Y    N    N
Shell functions                      Y(1) N    Y    Y    N    Y    Y    Y
"Sensible" Input/Output redirection  Y    N    Y    Y    N    Y    Y    Y
Directory stack                      N    Y    Y    Y    Y    Y    F    F
Command history                      N    Y    Y    Y    Y    Y    L    L
Command line editing                 N    N    Y    Y    Y    Y    L    L
Vi Command line editing              N    N    Y    Y    Y(3) Y    L    L
Emacs Command line editing           N    N    Y    Y    Y    Y    L    L
Rebindable Command line editing      N    N    N    Y    Y    Y    L    L
User name look up                    N    Y    Y    Y    Y    Y    L    L
Login/Logout watching                N    N    N    N    Y    Y    F    F
Filename completion                  N    Y(1) Y    Y    Y    Y    L    L
Username completion                  N    Y(2) Y    Y    Y    Y    L    L
Hostname completion                  N    Y(2) Y    Y    Y    Y    L    L
History completion                   N    N    N    Y    Y    Y    L    L
Fully programmable Completion        N    N    N    N    Y    Y    N    N
Mh Mailbox completion                N    N    N    N(4) N(6) N(6) N    N
Co Processes                         N    N    Y    N    N    Y    N    N
Builtin artithmetic evaluation       N    Y    Y    Y    Y    Y    N    N
Can follow symbolic links invisibly  N    N    Y    Y    Y    Y    N    N
Periodic command execution           N    N    N    N    Y    Y    N    N
Custom Prompt (easily)               N    N    Y    Y    Y    Y    Y    Y
Sun Keyboard Hack                    N    N    N    N    N    Y    N    N
Spelling Correction                  N    N    N    N    Y    Y    N    N
Process Substitution                 N    N    N    Y(2) N    Y    Y    Y
Underlying Syntax                    sh   csh  sh   sh   csh  sh   rc   rc
Freely Available                     N    N    N(5) Y    Y    Y    Y    Y
Checks Mailbox                       N    Y    Y    Y    Y    Y    F    F
Tty Sanity Checking                  N    N    N    N    Y    Y    N    N
Can cope with large argument lists   Y    N    Y    Y    Y    Y    Y    Y
Has non-interactive startup file     N    Y    Y(7) Y(7) Y    Y    N    N
Has non-login startup file           N    Y    Y(7) Y    Y    Y    N    N
Can avoid user startup files         N    Y    N    Y    N    Y    Y    Y
Can specify startup file             N    N    Y    Y    N    N    N    N
Low level command redefinition       N    N    N    N    N    N    N    Y
Has anonymous functions              N    N    N    N    N    N    Y    Y
List Variables                       N    Y    Y    N    Y    Y    Y    Y
Full signal trap handling            Y    N    Y    Y    N    Y    Y    Y
File no clobber ability              N    Y    Y    Y    Y    Y    N    F
Local variables                      N    N    Y    Y    N    Y    Y    Y
Lexically scoped variables           N    N    N    N    N    N    N    Y
Exceptions                           N    N    N    N    N    N    N    Y

Key to the table above.

   Y      Feature can be done using this shell.
          
   N      Feature is not present in the shell.
          
   F      Feature can only be done by using the shells function
          mechanism.
          
   L      The readline library must be linked into the shell to enable
          this Feature.


Old News ;-)

Comparison of SLES (SUSE) and RHEL (Red Hat) on IBM System p

18 November 2008 | ibm.com/developerworks

... ... ...

What you won't find with RHEL is an integrated GUI-like YaST2, from which you can launch any command that your heart desires. With Red Hat, you have GUIs, but you need to remember the names of the commands that launch them; there is not one command from which it all flows as there is with SLES. To add logical volumes, you'll use the system-config-lvm command. The first thing you'll need to do prior to setting up our volume group is initialize your unpartitioned space (see Figure 8), which, here, is 20 GB: # system-config-lvm.
 

[Mar 23, 2006] Microsoft today published an IDC report which looked at issues related to Unix migration

Hat tip to Solaris Migration resources in the SPARC Product Directory

IDC interviewed 400 Unix/RISC customers about their attitudes to migration. The average interview time was 30 minutes. It makes very interesting reading. The report says Sun is the main target for migration and the clock is ticking. Here are some extracts...

"Sun Solaris continues to be the most popular Unix variant for Unix servers running Web infrastructure. While this is not surprising given the dot-com success that Sun enjoyed in the late 1990s and into 2000, it does continue to represent a challenge for Sun because these workloads have been frequent targets for migration over the past five years."

"IDC believes that the server life-cycle issues are driving much of this change, because the systems in the Unix installed base have aged in recent years, and this is compounded by the fact that many dot-com era installations with Web enablement of business workloads, are coming of age and will require replacement. In addition, many Unix/RISC servers were deployed in preparation for Y2K and many of these systems are now nearing the end of their useful life cycle." ...read the article (pdf), ...IDC profile, Solaris Migration Resources

Recommended Links

ORACLE-BASE - UNIX Commands for DBAs

AIX to Solaris

AIX to Solaris Operating System Migration Solutions

Linux to Solaris

Linux Migration Guide - OpenSolaris

Migrating from Linux to Solaris or OpenSolaris - Solaris Developer ...

Solaris to Linux

Solaris to AIX

Solaris to HP-UX

HP Solaris-to-HP-UX Porting Kit

Porting Solaris code onto Itanium HP-UX 11i and Red Hat.

HP to AIX

HP to AIX migration - Network Tuning parameters - Toolbox for IT ...

HP Simplifies Migration for Sun Solaris Customers to More ...

Migrating from Solaris to HP-UX—components

Computerworld > HP's blue light special- 85% off HP-UX with ...

HP-UX to Solaris

HP-UX to Solaris Migration


Last modified: October 20, 2019