Administration of Remote Servers

Imagine that you're in charge of several dozen of Linux servers, mix of SLES and Red Hat. Servers are located in several remote datacenters with no IT staff.

An interesting part starts when one of the servers at remote location stops responding.  We are assuming that network connection to the site is in working condition and the servers has remote administration device installed. Typically those are Dell or HP so either DRAC or ILO are available. Those two are the key and learning them well in not just important, its a critical life saver skill. That includes usage of virtual media, as there is nobody at the remote some to put a CD/DVD in the slot.  See

• iLO 3 -- HP engineering fiasco
• Dell DRAC

Here is an interesting snippet from an Internet discussion on serverfault.com remote access - How do you remotely administer your Linux boxes - Server Fault

My toolset for these operations is painfully sparse (SSH into the box, edit files in VIM, WGET remote files that I need), and I suspect there is a much better way to do it. I'm curious to hear what other people in my position are doing.

Sparse? What on earth do you mean? Excuse me for ranting, but dismissing ssh, vim and wget as painful is almost insulting. From your question I deduce you are mainly a programmer for your daytime job, so I kinda understand the question. But honestly, I would not hire a Linux admin who is not comfortable with any of the three tools you mentioned.

Are you using some form of Windowing system and remote-desktop equivalent to access the box, or is it all command line? Managing remote Windows boxes is trivial, since you can simply remote desktop in and transfer files over the network. Is there an equivalent to this in the Linux world?

For administrator tasks I never, ever use an X environment. You do not need one, it'll only take up system resources and, for the most of the time, they're a hindrance instead of a help. Most GUI configuration tools (well, practically all, really) only offer a subset of the configuration option you can set in a configuration file with vim.

Managing a Linux box is no less trivial than managing a Windows box. It just takes some time to gain a decent skill set.

And a network file transfer equivalent? Plenty. scp, sftp, ftp, nfs, cifs / smb (Windows file sharing protocols), and then some.

Are you doing your config file changes/script tweaks directly on the machine? Or do you have something set up on your local box to edit these files remotely? Or are you simply editing them remotely then transferring them at each save?

Depends on what I am doing. Most of the things I do directly in the config files on the machine (for development and testing boxes) and then I push the file into a configuration channel on our Satellite server, after which I deploy the file to all servers directly (for production boxes). Really, vim is a treasure. That is, when you find out how to use it properly.

How are you moving files back and forth between the server and your local environment? FTP? Some sort of Mapped Drive via VPN?

scp all the way and maybe some sftp, and I suggest you do too. Never, ever use FTP to move sensitive files (e.g. config files) over a public network. I do not use a mapped network because again, all I need is on the server. If you mean c files and not configuration files here, I usually use something like svn or git and then push my changes to the box.

I'd really need to get some best practices in place for administering these boxes. Any suggestions to remove some of the pain would be most welcome!

You are already using them: ssh, scp, wget and vim. Those are not pain. There might be some teething pains, while you figure out how powerful they are. But, to bring the Windows analogy back, I feel seriously hampered when I have to use a Windows box. For you it's the other way around. It's just what you are used to. So, give it some time and it'll come to you.

You mentioned already ssh, vim and wget which is essential and perfect. Some additional tools that can make life easier:

1. GNU Screen

"GNU Screen is a free terminal multiplexer that allows a user to access multiple separate terminal sessions inside a single terminal window or remote terminal session. It is useful for dealing with multiple programs from the command line, and for separating programs from the shell that started the program." (From the GNU_Screen page on wikipedia)

A main advantage is that you can have one or several virtual terminals that are in the exactly same state as you left them when you come back (i.e. relogin via ssh). This is also good when your connection is broken for some reason.

Screen works indepently from the software you use to connect to the box (it lives on the server), so it combines well with putty or most other terminal software.

This article shows some nice things you can do with it: http://www.pastacode.de/extending-gnu-screen-adding-a-taskbar/en/

2. Midnight Commander

A console based graphical-like browsing tool for viewing and manipulation of files and directories. Can also do secure remote transfers.

3. rsync

For fast, secure and reliable file transfer and synchronisation between different locations

4. VCS

Use of a distributed version control system like bazaar, mercurial or git to update code. Github or Bitbucket offer commercial code hosting, but it is not necessary, you can also use it efficiently on your own machines.

Joseph Kern: can you elaborate how you exactly use git for remote config organization?

5. Terminal Clients

On unix-like-systems they are already on board, on Windows you can use Putty, Tera Term, Mind Term or Pandora. Or make a cygwin installation and ssh from the cygwin terminal windows to the remote boxes (which has more advantages but this is a question of what you prefer).

6. Tunneling and Port Forwarding

It can be helpful to forward certain ports securely to your local machine. For example you could forward the mysql port TCP 3306 or postgres TCP 5432 and install some database administration tool locally.

You can build tunnels from Windwos machines with putty (or command line based with it's little brother plink), with cygwin and Mindterm also can do port forwarding. If you are locally on a unix-like machine you can use ssh odr plink to create such tunneling.

To create some more stable and permanent tunneling for various ports I recommend OpenVPN. The "pre-shared-key" tunneling method from point to point is not so hard to install.

7. Have a local unix-like system

When your local machine is a Mac you have this already, you can open a local shell. When your workstation is windows-based it could be helpful to create a local Unix-like server, which is in the same local network. This can be a different machine in a different room connected to the same router or switch. Or if you want only one machine, you can install the free VMware server and make a virtual machine, preferably the same operating system as your remote machine. Install a samba server on it and you can "net use" the samba shares from your desktop.

If you an ssh server on the local server and open port 22 on your router for it you can ssh into your local system when you are outside.

You can build tunnels to remote machines or transfer and synchronize files and whole file trees with rsync. You can use it for testing, for VSC, for local development, as a local webserver, for training purposes.

You can pull backups from remote machines. You can create local cron jobs that do the backups automatically (e.g. databases you want to save locally regularly)

NEWS CONTENTS

• 20191108 : Multiple Linux sysadmins working as root ( May 21, 2012 , serverfault.com )
• 20191108 : MAT Monitoring and Administration Tool ( MAT Monitoring and Administration Tool, )
• 20191108 : Unix SysAdm Resources Automated Unix SysMgmt Software ( Unix SysAdm Resources Automated Unix SysMgmt Software , )

Old News ;-)

[Nov 08, 2019] Multiple Linux sysadmins working as root

No new interesting ideas for such an important topic whatsoever. One of the main problems here is documenting actions of each administrator in such a way that the set of actions was visible to everybody in a convenient and transparent matter. With multiple terminal opened Unix history is not the file from which you can deduct each sysadmin actions as parts of the history from additional terminals are missing. , not smooch access. Actually Solaris has some ideas implemented in Solaris 10, but they never made it to Linux

May 21, 2012 | serverfault.com

In our team we have three seasoned Linux sysadmins having to administer a few dozen Debian servers. Previously we have all worked as root using SSH public key authentication. But we had a discussion on what is the best practice for that scenario and couldn't agree on anything.

Everybody's SSH public key is put into ~root/.ssh/authorized_keys2

• Advantage: easy to use, SSH agent forwarding works easily, little overhead
• Disadvantage: missing auditing (you never know which "root" made a change), accidents are more likely

Using personalized accounts and sudo

That way we would login with personalized accounts using SSH public keys and use sudo to do single tasks with root permissions. In addition we could give ourselves the "adm" group that allows us to view log files.

• Advantage: good auditing, sudo prevents us from doing idiotic things too easily
• Disadvantage: SSH agent forwarding breaks, it's a hassle because barely anything can be done as non-root

Using multiple UID 0 users

This is a very unique proposal from one of the sysadmins. He suggest to create three users in /etc/passwd all having UID 0 but different login names. He claims that this is not actually forbidden and allow everyone to be UID 0 but still being able to audit.

• Advantage: SSH agent forwarding works, auditing might work (untested), no sudo hassle
• Disadvantage: feels pretty dirty - couldn't find it documented anywhere as an allowed way

Comments:

• About your "couldn't find it documented anywhere as an allowed way" statement: Have a look to the -o flag in the useradd manual page. This flag is there to allow multiple users sharing the same uid. – jlliagre May 21 '12 at 11:38
• Can you explain what you mean by "SSH agent forwarding breaks" in the second option? We use this at my work and ssh agent forwarding works just fine. – Patrick May 21 '12 at 12:01
• You should ssh out from your non-root account rather than from within sudo. – Random832 May 21 '12 at 12:33
• Another consequence of sudo method: You can no longer SCP/FTP as root. Any file transfers will first need to be moved into the person's home directory and then copied over in the terminal. This is an advantage and a disadvantage depending on perspective. – user606723 May 21 '12 at 14:43
• Why aren't puppet/chef/ansible-type systems being considered? – Alex Holst May 21 '12 at 20:29

The second option is the best one IMHO. Personal accounts, sudo access. Disable root access via SSH completely. We have a few hundred servers and half a dozen system admins, this is how we do it.

How does agent forwarding break exactly?

Also, if it's such a hassle using sudo in front of every task you can invoke a sudo shell with sudo -s or switch to a root shell with sudo su -

thepearson thepearson 775 8 8 silver badges 18 18 bronze badges

• 10 Rather than disable root access by SSH completely, I recommend making root access by SSH require keys, creating one key with a very strong keyphrase and keeping it locked away for emergency use only. If you have permanent console access this is less useful, but if you don't, it can be very handy. – EightBitTony May 21 '12 at 13:02
• 17 I recommend disabling root login over SSH for security purposes. If you really need to be logged in as root, log in as a non-root user and su. – taz May 21 '12 at 14:29
• +1.. I'd go further than saying "The second option is the best". I'd it's the only reasonable option. Options one and three vastly decrease the security of the system from both outside attacks and mistakes. Besides, #2 is how the system was designed to be primarily used. – Ben Lee May 23 '12 at 18:28
• 2 Please, elaborate on sudo -s . Am I correct to understand that sudo -i is no difference to using su - or basically logging in as root apart from additional log entry compared to plain root login? If that's true, how and why is it better than plain root login? – PF4Public Jun 3 '16 at 20:32

add a comment | 9 With regard to the 3rd suggested strategy, other than perusal of the useradd -o -u userXXX options as recommended by @jlliagre, I am not familiar with running multiple users as the same uid. (hence if you do go ahead with that, I would be interested if you could update the post with any issues (or sucesses) that arise...)

I guess my first observation regarding the first option "Everybody's SSH public key is put into ~root/.ssh/authorized_keys2", is that unless you absolutely are never going to work on any other systems;

1. then at least some of the time, you are going to have to work with user accounts and sudo

The second observation would be, that if you work on systems that aspire to HIPAA, PCI-DSS compliance, or stuff like CAPP and EAL, then you are going to have to work around the issues of sudo because;

1. It an industry standard to provide non-root individual user accounts, that can be audited, disabled, expired, etc, typically using some centralized user database.

So; Using personalized accounts and sudo

It is unfortunate that as a sysadmin, almost everything you will need to do on a remote machine is going to require some elevated permissions, however it is annoying that most of the SSH based tools and utilities are busted while you are in sudo

Hence I can pass on some tricks that I use to work-around the annoyances of sudo that you mention. The first problem is that if root login is blocked using PermitRootLogin=no or that you do not have the root using ssh key, then it makes SCP files something of a PITA.

Problem 1 : You want to scp files from the remote side, but they require root access, however you cannot login to the remote box as root directly.

Boring Solution : copy the files to home directory, chown, and scp down.

ssh userXXX@remotesystem , sudo su - etc, cp /etc/somefiles to /home/userXXX/somefiles , chown -R userXXX /home/userXXX/somefiles , use scp to retrieve files from remote.

Less Boring Solution : sftp supports the -s sftp_server flag, hence you can do something like the following (if you have configured password-less sudo in /etc/sudoers );

sftp  -s '/usr/bin/sudo /usr/libexec/openssh/sftp-server' \
userXXX@remotehost:/etc/resolv.conf

(you can also use this hack-around with sshfs, but I am not sure its recommended... ;-)

If you don't have password-less sudo rights, or for some configured reason that method above is broken, I can suggest one more less boring file transfer method, to access remote root files.

Port Forward Ninja Method :

Login to the remote host, but specify that the remote port 3022 (can be anything free, and non-reserved for admins, ie >1024) is to be forwarded back to port 22 on the local side.

 [localuser@localmachine ~]$ ssh userXXX@remotehost -R 3022:localhost:22
Last login: Mon May 21 05:46:07 2012 from 123.123.123.123
------------------------------------------------------------------------
This is a private system; blah blah blah
------------------------------------------------------------------------

Get root in the normal fashion...

-bash-3.2$ sudo su -
[root@remotehost ~]#

Now you can scp the files in the other direction avoiding the boring boring step of making a intermediate copy of the files;

[root@remotehost ~]#  scp -o NoHostAuthenticationForLocalhost=yes \
 -P3022 /etc/resolv.conf localuser@localhost:~
localuser@localhost's password: 
resolv.conf                                 100%  
[root@remotehost ~]#

Problem 2: SSH agent forwarding : If you load the root profile, e.g. by specifying a login shell, the necessary environment variables for SSH agent forwarding such as SSH_AUTH_SOCK are reset, hence SSH agent forwarding is "broken" under sudo su - .

Half baked answer :

Anything that properly loads a root shell, is going to rightfully reset the environment, however there is a slight work-around your can use when you need BOTH root permission AND the ability to use the SSH Agent, AT THE SAME TIME

This achieves a kind of chimera profile, that should really not be used, because it is a nasty hack , but is useful when you need to SCP files from the remote host as root, to some other remote host.

Anyway, you can enable that your user can preserve their ENV variables, by setting the following in sudoers;

 Defaults:userXXX    !env_reset

this allows you to create nasty hybrid login environments like so;

login as normal;

[localuser@localmachine ~]$ ssh userXXX@remotehost 
Last login: Mon May 21 12:33:12 2012 from 123.123.123.123
------------------------------------------------------------------------
This is a private system; blah blah blah
------------------------------------------------------------------------
-bash-3.2$ env | grep SSH_AUTH
SSH_AUTH_SOCK=/tmp/ssh-qwO715/agent.1971

create a bash shell, that runs /root/.profile and /root/.bashrc . but preserves SSH_AUTH_SOCK

-bash-3.2$ sudo -E bash -l

So this shell has root permissions, and root $PATH (but a borked home directory...)

bash-3.2# id
uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel) context=user_u:system_r:unconfined_t
bash-3.2# echo $PATH
/usr/kerberos/sbin:/usr/local/sbin:/usr/sbin:/sbin:/home/xtrabm/xtrabackup-manager:/usr/kerberos/bin:/opt/admin/bin:/usr/local/bin:/bin:/usr/bin:/opt/mx/bin

But you can use that invocation to do things that require remote sudo root, but also the SSH agent access like so;

bash-3.2# scp /root/.ssh/authorized_keys ssh-agent-user@some-other-remote-host:~
/root/.ssh/authorized_keys              100%  126     0.1KB/s   00:00    
bash-3.2#

Tom H Tom H 8,793 3 3 gold badges 34 34 silver badges 57 57 bronze badges

• 1 I like the hacks. – sjbotha May 21 '12 at 20:16

add a comment | 2 The 3rd option looks ideal - but have you actually tried it out to see what's happenning? While you might see the additional usernames in the authentication step, any reverse lookup is going to return the same value.

Allowing root direct ssh access is a bad idea, even if your machines are not connected to the internet / use strong passwords.

Usually I use 'su' rather than sudo for root access.

symcbean symcbean 18.8k 1 1 gold badge 24 24 silver badges 40 40 bronze badges

• 4 Adding multiple users with the same UID adds problems. When applications go to look up the username for a UID number, they can look up the wrong username. Applications which run under root can think they're running as the wrong user, and lots of weird errors will start popping up (I tried it once). – Patrick May 21 '12 at 11:59
• 8 The third option is just a bloody bad idea . You're essentially breaking the 1:1 relation between UIDs and usernames, and literally everything in unix expects that relation to hold. Just because there's no explicit rule not to do it doesn't mean it's a good idea. – Shadur May 21 '12 at 14:28
• Sorry, but the third option is a horrible idea. Having multiple UID 0 people logging in is just asking for problems to be multiplied. Option number 2 is the only sane one. – Doug May 21 '12 at 20:01
• The third option doesn't deserve that many downvotes. There is no command in Unix I'm aware of that is confused by this trick, people might be but commands should not care. It is just a different login name but as soon as you are logged in, the first name found matching the uid in the password database is used so just make sure the real username (here root) appears first there. – jlliagre May 21 '12 at 21:38
• @Patrick Have you seen this in practice? As much as I tested, then applications pick root user if root user is the first one in /etc/passwd with UID 0. I tend to agree with jlliagre. The only downside that I see, is that each user is a root user and sometimes it might be confusing to understand who did what. – Martin Sep 4 '16 at 18:31

add a comment | 2 I use (1), but I happened to type

rm -rf / tmp *

on one ill-fated day.I can see to be bad enough if you have more than a handful admins.

(2) Is probably more engineered - and you can become full-fledged root through sudo su -. Accidents are still possible though.

(3) I would not touch with a barge pole. I used it on Suns, in order to have a non-barebone-sh root account (if I remember correctly) but it was never robust - plus I doubt it would be very auditable.

add a comment | 2 Definitely answer 2.

1. Means that you're allowing SSH access as root . If this machine is in any way public facing, this is just a terrible idea; back when I ran SSH on port 22, my VPS got multiple attempts hourly to authenticate as root. I had a basic IDS set up to log and ban IPs that made multiple failed attempts, but they kept coming. Thankfully, I'd disabled SSH access as the root user as soon as I had my own account and sudo configured. Additionally, you have virtually no audit trail doing this.
2. Provides root access as and when it is needed. Yes, you barely have any privileges as a standard user, but this is pretty much exactly what you want; if an account does get compromised, you want it to be limited in its abilities. You want any super user access to require a password re-entry. Additionally, sudo access can be controlled through user groups, and restricted to particular commands if you like, giving you more control over who has access to what. Additionally, commands run as sudo can be logged, so it provides a much better audit trail if things go wrong. Oh, and don't just run "sudo su -" as soon as you log in. That's terrible, terrible practice.
3. Your sysadmin's idea is bad. And he should feel bad. No, *nix machines probably won't stop you from doing this, but both your file system, and virtually every application out there expects each user to have a unique UID. If you start going down this road, I can guarantee that you'll run into problems. Maybe not immediately, but eventually. For example, despite displaying nice friendly names, files and directories use UID numbers to designate their owners; if you run into a program that has a problem with duplicate UIDs down the line, you can't just change a UID in your passwd file later on without having to do some serious manual file system cleanup.

sudo is the way forward. It may cause additional hassle with running commands as root, but it provides you with a more secure box, both in terms of access and auditing.

Rohaq Rohaq 121 3 3 bronze badges

Definitely option 2, but use groups to give each user as much control as possible without needing to use sudo. sudo in front of every command loses half the benefit because you are always in the danger zone. If you make the relevant directories writable by the sysadmins without sudo you return sudo to the exception which makes everyone feel safer.

Julian Julian 121 4 4 bronze badges

In the old days, sudo did not exist. As a consequence, having multiple UID 0 users was the only available alternative. But it's still not that good, notably with logging based on the UID to obtain the username. Nowadays, sudo is the only appropriate solution. Forget anything else.

It is documented permissible by fact. BSD unices have had their toor account for a long time, and bashroot users tend to be accepted practice on systems where csh is standard (accepted malpractice ;)

add a comment | 0 Perhaps I'm weird, but method (3) is what popped into my mind first as well. Pros: you'd have every users name in logs and would know who did what as root. Cons: they'd each be root all the time, so mistakes can be catastrophic.

I'd like to question why you need all admins to have root access. All 3 methods you propose have one distinct disadvantage: once an admin runs a sudo bash -l or sudo su - or such, you lose your ability to track who does what and after that, a mistake can be catastrophic. Moreover, in case of possible misbehaviour, this even might end up a lot worse.

Instead you might want to consider going another way:

• Create your admin users as regular users
• Decide who needs to do what job (apache management / postfix management etc)
• Add users to related groups (such as add "martin" to "postfix" and "mail", "amavis" if you use it, etc.)
• Fix permissions (chmod -R g+w postfix:postfix /etc/postfix)
• give only relative sudo powers: (visudo -> let martin use /etc/init.d/postfix , /usr/bin/postsuper etc.)

This way, martin would be able to safely handle postfix, and in case of mistake or misbehaviour, you'd only lose your postfix system, not entire server.

Same logic can be applied to any other subsystem, such as apache, mysql, etc.

Of course, this is purely theoretical at this point, and might be hard to set up. It does look like a better way to go tho. At least to me. If anyone tries this, please let me know how it went.

Tuncay Göncüoğlu Tuncay Göncüoğlu 561 3 3 silver badges 9 9 bronze badges

• I should add handling SSH connections is pretty basic in this context. Whatever method you use, do not permit root logins over SSH, let the individual users ssh with their own credentials, and handle sudo/nosudo/etc from there. – Tuncay Göncüoğlu Nov 30 '12 at 17:53

MAT Monitoring and Administration Tool

MAT is an easy to use network enabled UNIX configuration and monitoring tool. It provides an integrated tool for many common system administration tasks, including Backups., and Replication It includes a warning system for potential system problems, and graphing of many common system parameters. Click here for more.

Coming soon in version 0.24 will be an embedded interpreter with it you will be able to monitor any parameter you can write a script to capture. It also will create the ability to have OS specific configuration tools.

Unix SysAdm Resources Automated Unix SysMgmt Software

• Master System is a public-domain Unix systems configuration tool written in Perl. The system is architecture and operating system independent, but it can handle architecture and operating system dependent configuration. It is designed to control the configuration of large groups of systems that are configured in the same style, but are not necessarily identical. From a group at Rutgers University.
  
• Webmin is a free web-based admin interface for Unix systems. Via a web browser, you can configure DNS, Apache, Samba, filesystems, startup scripts, inetd, crontabs and more. Written in Perl5 and easily extendable. Supports several Linux versions and Solaris.

Recommended Links

Softpanorama Recommended

Top articles

Sites

Conveniently Administering Remote Servers

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: February 19, 2020