Lack of testing of complex, potentially destructive, commands
before execution on the production box

An educational and horrifying tales of SGS (Spontaneous Gross Stupidity)

Amount of testing of complex commands, especially those that involve recursive operations should generally be proportional to potential damage.

Backups in Unix can be one of the most dangerous commands used, and they are used to prevent rather than cause a problem.  If any Unix utility were a candidate for a warning message, or error checking, this would be it.

One common mishap is running as root complex command like  find  ... -exec rm {} \; without any testing or even rereading the command several times before hitting Enter.   This is covered in more details in

• Creative uses of rm with unintended consequences
• Typical Errors In Using Find

Never ever assume that some prepackaged script that you are running does anything right. Beware anything recursive when logged in as root!

Such errors are often made under time pressure. As for find example above instead of rm other commands can be equally destructive, for example chown or, especially, chmod. 

• From: [email protected] (Rick Furniss)
  Organization: WINCO

     Horror stories:

     Did this myself many years ago, and have come close to it since.

     Murphy's law #?? , preventive maintenance doesn't.

     try this one:   /etc/dump /dev/rmt/0m /dev/dsk/0s1
               Or:   tar cvf /dev/root /dev/rmt0

  Backups on unix can be one of the most dangerous commands used, and they are used to prevent rather than cause a problem.  If any Unix utility were a candidate for a warning message, or error checking, this would be it.

  Just in case you didn't catch the HORROR above,  the parameters are specified backwards causing a TOTAL wipe out of the root file systems.

  More systems have been wiped out by admins, than any hacker could do in a life time.

Among most typical cases

1. Running complex rm command without testing it using ls . You should always use ls -Rl command to test complex rm  -R  commands (  -R, --recursive means process  subdirectories recursively). Think - a "Think it over first" (TIOT) type of utility can help here as they provide you a pause and a chice to correct possible error.
    
2. Mistyping complex path or file name in rm commands. It's always better to use copy and paste operation for directories and files used in rm command as it helps to avoid various typos. If you do recursive deletion is is useful to have a "rty run" using las of find to see the files.  Here saferm utility and other utilities of this class can be helpful.

   Here are to example of typos:

   There are some exotic typos that can lean you to troubles, especially with -r  option. One of them is unintended space:

    rm /homejoeuser/old *

   instead of

   rm /homejoeuser/old*

   Or, similarly:

    rm * _old

   instead of

   rm *_old

   In all cases when you are deleting the directory directly copying its name from the listing obtained via ls or find command is much safer that typing it by yourself.

   Such a mistake is more damaging  is you use -r option. For example:

   rm -rf /tmp/foo/bar/ *

   instead of

   rm -rf /tmp/foo/bar/*

   So it is prudent to block execution of rm commands with multiple argument, if option -r is used in your saferm script.

   In some situation results can be devastating. For example, in the example above you would erase all your files and subdirectories in the /etc directory. Modern flavors of Unix usually prevent erasing / but not /etc.

   Again, it's much safer to list the directory and copy it using cut and paste.

   You can also hit Enter by mistake before finishing typing the line containing rm command. 

Here is a couple of examples:

From: [email protected] (Marc Fraioli)

Organization: Grebyn Timesharing

Well, here's a good one for you:

I was happily churning along developing something on a Sun workstation, and was getting a number of annoying permission denied from trying to write into a directory hierarchy that I didn't own. 

Getting tired of that, I decided to set the permissions on that subtree to 777 while I was working, so I wouldn't have to worry about it.  Someone had recently told me that rather than using plain "su", it was good to use "su -", but the implications had not yet sunk in.  (You can probably see where this is going already, but I'll go to the bitter end.) 

Anyway, I cd'd to where I wanted to be, the top of my subtree, and did su -.  Then I did chmod -R 777.  I then started to wonder why it was taking so damn long when there were only about 45 files in 20 directories under where I (thought) I was. 

Well, needless to say, su - simulates a real login, and had put me into root's home directory, /, so I was proceeding to set file permissions for the whole system to wide open. I aborted it before it finished, realizing that something was wrong, but this took quite a while to straighten out.

Marc Fraioli

Oh you bastards. I was hoping that a thread like this would never appear, because if it did, I knew I would have to confess. Oh well...

About a year back, I was looking through /etc and found that a few system files had world write permission.  Gasping with horror, I went to put it right with something like

dipshit# chmod -r 664 /etc/*

(I know, I know, goddamnit!.. now)

Everything was OK for about two to three weeks, then the machine went down for some reason (other than the obvious).  Well, I expect that you can imagine the result.  The booting procedure was unable to run fsck, so barfed and mounted the file systems read-only, and bunged me into single-user mode.

Dumb expression..gradual realisation..cold sweat.

Of course, now I can't do a frigging chmod +x on anything because it's all read-only. In fact I can't run anything that isn't part of sh.

Wedgerama. Hysteria time. Consider reformatting disks. All sorts of crap ideas. Headless chicken scene. Confession.

"You did WHAT??!!"

Much forehead slapping, solemn oaths and floor pacing.

Luckily, we have a local MegaUnixGenius who, having sat puzzled for an hour or more, decided to boot from a cdrom and take things from there. He fixed it.

My boss, totally amazed at the fix I'd got the system into, luckily saw the funny side of it.  I didn't.  Even though at that stage, I didn't know much about unix/suns/booting/admin, I did actually know enough to NOT use a command like the one above. Don't ask. Must be the drugs.

BTW, if my future employer _is_ reading this (like they say he/she might), then I have certainly learned tonnes of stuff in the last year, especially having had to set up a complete Sun system, fix local problems, etc :-)

Anyone else got a tale of SGS (Spontaneous Gross Stupidity) ?

-dave "I'm much better now, honest.. no, really.. hey what's this button doooooooooOOOOOO..."

Recommended Links

Softpanorama Recommended

Top articles

Sites

• Typical Errors In Using Find
• xargs Command Tutorial
• Bash test and comparison functions – IBM Developer by Ian Shields February 20, 2007

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: December 30, 2020