Softpanorama

May the source be with you, but remember the KISS principle ;-)
Home Switchboard Unix Administration Red Hat TCP/IP Networks Neoliberalism Toxic Managers
(slightly skeptical) Educational society promoting "Back to basics" movement against IT overcomplexity and  bastardization of classic Unix

Log Rotation in Solaris Using Logadm

News

See also

 Articles Recommended Links Reference
Perl Log Rotation tools Log monitoring HTTP Log tools Humor Etc

Starting from Solaris 9, there is a standard tool, called logadm, to rotate logs. Logadm is an independent Utility, unlike integrated in syslog daemon capability that can be found in AIX. The logadm command is a preconfigured entry in the default crontab file supplied with Solaris 9 and Solaris 10.  The  /usr/lib/newsyslog script is no longer used.

Note: Before Solaris 9 there was FreeBSD-style tool newsyslog located at /usr/lib/  and Perl script rotatelog written by  Shaun Rowland. It  meant to be run from cron, with a configuration file and documentation available. For Solaris 8 and earlier you can download the tar.gz file and untar it. Then go in the new directory, and execute make install. It will install everything by default under /usr/local/ (mainly /usr/local/sbin/rotatelog and /usr/local/etc/rotatelog.conf).

logadm  uses a configuration file logadm.conf. The latter specifies the schedule for log rotation and options with which rotation will be performed. In the default configuration of logadm.conf  the /var/adm/messages  file is already configured to be rotated:

The syntax of the logadm.conf  file is as follows:

logname  <space> options

A very simple configuration to rotate these files. Insert the following line: 

/var/adm/sshdlog -C 7 -P 'Sat Jan  21 08:10:00 2006' -a 'pkill sshd2'

As you can see from the line above, we specified /var/adm/sshdlog  as our log to rotate.

The options (see Reference for the full list).:

Like with cron, the -w option to logadm(1M) is the preferred way to modify the configuration file, but if you do edit it by hand. In this case use "logadm -V" to check it for errors and to activate changes.

For example, one entry in the /etc/logadm.conf file specifies that the /var/log/syslog file is rotated weekly unless the file is empty. The most recent syslog file becomes syslog.0, the next most recent becomes syslog.1, and so on. Eight previous syslog log files are kept. The /etc/logadm.conf file also contains time stamps of when the last log rotation occurred. It is written in the field defined by -P flag.

You can use the logadm command to customize system logging and to add additional logging in the /etc/logadm.conf file as needed.

For example, to rotate the Apache access and error logs, use the following commands:

# logadm -w /var/apache/logs/access_log -s 100m
# logadm -w /var/apache/logs/error_log -s 10m

In this example, the Apache access_log file is rotated when it reaches 100 Mbytes in size, with a .0, .1, (and so on) suffix, keeping 10 copies of the old access_log file. The error_log is rotated when it reaches 10 Mbytes in size with the same suffixes and number of copies as the access_log file.

The /etc/logadm.conf entries for the preceding Apache log rotation examples look similar to the following:

# cat /etc/logadm.conf
...
/var/apache/logs/error_log -s 10m
/var/apache/logs/access_log -s 100m

For more information, see logadm(1M).

You can use the logadm command as superuser or by assuming an equivalent role (with Log Management rights). With role-based access control (RBAC), you can grant non-root users the privilege of maintaining log files by providing access to the logadm command. For example, add the following entry to the /etc/user_attr file to grant user andy the ability to use the logadm command:

andy::::profiles=Log Management

Or, you can set up a role for log management by using the Solaris Management Console. For more information about setting up a role, see "Role-Based Access Control (Overview)" in System Administration Guide: Security Services.

The log rotation problem exists on any server but it is the most acute on log consolidation servers (LOGHOST servers) that collect logs from other servers.  The volume of logs on a busy Solaris loghost system grow over 50G in a week. On a typical loghost just log for /var/adm/messages  can easily be large then 1G in one day. 

Among log files which should be rotated on a typical Solaris box: :

Top updates
Bulletin Latest Past week Past month
Google Search

NEWS CONTENTS

Old News ;-)

BigAdmin - Submitted Tech Tip System Log Rotation Utilizing logadm (for Solaris 9, Solaris 10) by Kristopher March

Today, most system administrators are looking for ways to maximize their time at work and are always finding new and efficient ways of carrying out their daily tasks: The built-in Solaris tool, logadm, is one way to do so. New in the Solaris 9 Operating System, logadm is found in /usr/sbin.

Monitoring logs of all types found on a Solaris OS-based machine is a role most system administrators assume on a daily basis. The most typical system log of interest would be /var/adm/messages. On a busy system, this log file tends to grow rather quickly, thus making log examination a burdensome task. Numerous other log files could exist on a Sun server running the Solaris OS, all depending on the configuration. Other applications send information to a log file. Whether it is informational or critical, the text notices contained within these log files often provide pertinent data that should be viewed frequently.

Other log files to note:

In this Tech Tip, we will discuss the appropriate steps needed to set up and utilize logadm. In the default configuration, the /var/adm/messages file is already configured to be rotated. Therefore, we will use the sshdlog as our example. For most environments, running SSH is a critical component to secure machines. With the combination of Secure FTP and many users logging in, there is the potential for this file to grow rather quickly over a 24-hour period. Also, the policies of some computer centers require these files to be archived for a certain amount of time.

Configuration

logadm is an ideal tool to be run from cron. logadm uses a configuration file found in the /etc directory. It is called logadm.conf, and there are a couple of ways to customize this file to best suit your needs.

In this example, we will edit the file directly using the VI editor. Following the completion of any modification to the /etc/logadm.conf file, we run the logadm -V file to check for any errors and to validate the file before going further in our process.

Follow these steps to get started.

*must be completed as uid: root

1. Verify you are running the correct version of the Solaris OS. It should be version 9.

% /usr/bin/uname -r      # this will show the version number 5.9

2. Determine which files need to be rotated on a daily basis. Again, we use the sshdlog found under /var/adm/ in this example. Most other files will work.

3. Make a copy of the logadm in case you need to revert back to the default file.

% cp /etc/logadm.conf  /etc/logadm.conf.orig

4. Edit the file using VI.

% vi /etc/logadm.conf

5. The syntax for log rotating is as follows:

logname  <space> options

Here we use a very simple configuration to rotate these files. Insert the following line: 

/var/adm/sshdlog -C 8 -P 'Sat Dec  6 08:10:00 2003' -a 'kill -HUP `cat \
	/var/run/sshd2_22.pid`'

As you can see from the line above, we specified /var/adm/sshdlog as our log to rotate.

The options I choose to use are explained here:

-C -- This option specifies the maximum number of log files to keep. If exceeded, the oldest file will be deleted to keep the number at 8. If you need to keep these files longer, change the number to a larger value.

-P -- This option is used to specify a timestamp for the log?s last rotation.

-a -- This option is used to specify a post command that is run once after all logs have been rotated. Although not entirely necessary, I send a kill signal to the SSH parent PID to reread any configuration files and send a message to the sshdlog.

6. Once finished editing your file, save it and validate the sshdloglogadm.conf as specified above.

Many other useful options can be used with the logadm tool. The best reference available is the man pages on this subject, which are available in the standard man files location: /usr/bin/man logadm.

Setting Up logadm to Run from cron

Now that you have your logadm configuration file updated and validated, it's time to set up cron to automatically run logadm so that your log file will be rotated.

As root, open the crontab file and start a new line. Note: I chose to run logadm every night at 11:58 pm. 
/usr/bin/crontab -e    - Insert the following line. 
58 23 * * * /usr/sbin/logadm

Specified in cron without any options, logadm will use the default configuration file found in /etc/.

Congratulations! You have successfully set up log rotating and are on your way to adding other log files that you see fit. Any logs now specified will automatically rotate without any user intervention.

Blog O' Matty " Compressing Solaris logfiles

If you run busy Solaris servers, you may have realized that the logfiles in /var/log and /var/adm can grow pretty large. These files are periodically rotated with the Solaris logadm utility, but they are not compress with the default logadm configuration.

To change this behavior, you can add use the "-z 0″ option to compress logfiles after they are rotated:

$ logadm -w /var/log/maillog -C 8 -P 'Wed Dec 21 19:56:59 2005′ -a 'kill -HUP `cat /var/run/syslog.pid`' -s 1m -z 0

Once the "-z" option is added, you can check the configuration by running logadm with the "-V" option:

$ logadm -V | grep maillog
/var/log/maillog -C 8 -P 'Wed Dec 21 19:56:59 2005′ -a 'kill -HUP `cat /var/run/syslog.pid`' -s 1m -z 0

If you would like to compress existing files and force a rotation, you can invoke logadm with the "-p now" option:

$ logadm -p now /var/log/maillog

Hopefully this will save folks some disk space (not that it's really required with the size of today's disks).

[users@httpd] Piped Log Rotation

From: [email protected]
Subject: [users@httpd] Piped Log Rotation
Date: Mon, 19 May 2003 20:14:27 GMT
Prev Next Prev by Thread Next by Thread

I've written a simple PERL program that I'm using to pipe the CustomLog=
s
that will split our logs into 24hour chunks:

CustomLog "| /usr/local/apache/bin/siphon.pl --file /tmp/access --time
12:00AM" common

=A0It functions very similar to the rotatelogs program that comes with =
Apache
but writes to our standard files names that we use for logs here
(access.15May-12AM). Couple of questions about the piped CustomLog. One=
, is
there anyway to have it pipe to my PERL script without spawning and
intermediate shell? Here's what the process table looks like right now,=
 it
spawns an sh process to run the perl process:

# ps -fj -g 25982
=A0 =A0 =A0UID =A0 PID =A0PPID =A0PGID =A0 SID =A0C =A0 =A0STIME TTY =A0=
 =A0 =A0TIME CMD
=A0 nobody 25985 25982 25982 25982 =A00 16:02:10 ? =A0 =A0 =A0 =A00:00
/usr/local/apache/bin/httpd
=A0 =A0 root 25983 25982 25982 25982 =A00 16:02:10 ? =A0 =A0 =A0 =A00:0=
0 /bin/sh
-c =A0/usr/local/apache/bin/siphon.pl --file /tmp/norm --time 11:41
=A0 nobody 25988 25982 25982 25982 =A00 16:02:10 ? =A0 =A0 =A0 =A00:00
/usr/local/apache/bin/httpd
=A0 =A0 root 25984 25983 25982 25982 =A01 16:02:10 ? =A0 =A0 =A0 =A00:0=
0 /usr/bin/perl
/usr/local/apache/bin/siphon.pl --file /tmp/norm --time 11:41
=A0 nobody 25987 25982 25982 25982 =A00 16:02:10 ? =A0 =A0 =A0 =A00:00
/usr/local/apache/bin/httpd
=A0 nobody 25989 25982 25982 25982 =A00 16:02:11 ? =A0 =A0 =A0 =A00:00
/usr/local/apache/bin/httpd
=A0 nobody 25986 25982 25982 25982 =A00 16:02:10 ? =A0 =A0 =A0 =A00:00
/usr/local/apache/bin/httpd
=A0 =A0 root 25982 =A0 =A0 1 25982 25982 =A00 16:02:10 ? =A0 =A0 =A0 =A0=
0:00
/usr/local/apache/bin/httpd



Second question is this. If the web admin botches the args for this PER=
L
program, I'd like to kill the HTTPD process so that they know right awa=
y
that there's a problem (rather then just sending a message to STDERR wh=
ich
is going to go unnoticed into the error log unless they check right awa=
y).
Since there is an intermediate shell process in between my program and =
the
parent httpd process, I can't simply kill my parent to kill the web ser=
ver.
Is it safe instead to get the Process Group ID and send a kill to it?
(getpgrp()) This is written specifically for UNIX environments (solaris=
 and
aix), not Wintel.

Thanks for the help.

-Norm Ackroyd=

Re Solaris log files

On Sat, 20 Feb 1999, Todd Williams wrote:
> 
> I've always considered writing some tool that would allow you to manage
> and age all your log files from one config file.  Maybe the config file
> would be a table that lists the base logfile name, the interval at which
> it gets aged, the number of logs or amount of space to keep online before
> deleting them, etc.
> 
> Anybody know of any such program?  It might be too much work for
> too little gain.

I'm sure there are others but you might check out RedHat's logrotate.
You can get it from them in RPM or SRPM (source) or you can find it in
several places as a tar ball.  Here's one:

   ftp://gatekeeper.dec.com/pub/linux/redhat/code/logrotate/

It's GNU'ed and is quite flexible.  A quote from the man page:

       logrotate  is  designed  to ease administration of systems
       that generate large numbers of log files.  It allows auto-
       matic  rotation,  compression, removal, and mailing of log
       files.  Each  log  file  may  be  handled  daily,  weekly,
       monthly, or when it grows too large.

And an example conf from the man page:

       # sample logrotate configuration file
       errors [email protected]
       compress

       /var/log/messages {
           rotate 5
           weekly
           postrotate
                                     /sbin/killall -HUP syslogd
           endscript
       }

       /var/log/httpd/access.log {
           rotate 5
           mail [email protected]
           errors [email protected]
           size=100k
           postrotate
                                     /sbin/killall -HUP httpd
           endscript
       }

       /var/log/news/* {
           monthly
           rotate 2
           errors [email protected]
           postrotate
                                     kill -HUP `cat /var/run/inn.pid`
           endscript
           nocompress
       }

I haven't tried compiling it for any of the other UNIX flavors I
administer but it does a nice job on my linux systems.

Recommended Links

Google matched content

Softpanorama Recommended

Top articles

Sites

BigAdmin - Submitted Tech Tip- System Log Rotation Utilizing ...

Blog O' Matty " Compressing Solaris logfiles

docs.sun.com System Administration Guide Advanced Administration

In Solaris 9 release, system log files are now rotated by the logadm command from an entry in the root crontab file. The /usr/lib/newsyslog script is no longer used.

The new system log rotation is defined in the /etc/logadm.conf file. This file includes log rotation entries for processes such as syslogd. For example, one entry in the /etc/logadm.conf file specifies that the /var/log/syslog file is rotated weekly unless the file is empty. The most recent syslog file becomes syslog.0, the next most recent becomes syslog.1, and so on. Eight previous syslog log files are kept.

The /etc/logadm.conf file also contains time stamps of when the last log rotation occurred.

You can use the logadm command to customize system logging and to add additional logging in the /etc/logadm.conf file as needed.

For example, to rotate the Apache access and error logs, use the following commands:

# logadm -w /var/apache/logs/access_log -s 100m
# logadm -w /var/apache/logs/error_log -s 10m


In this example, the Apache access_log file is rotated when it reaches 100 Mbytes in size, with a .0, .1, (and so on) suffix, keeping 10 copies of the old access_log file. The error_log is rotated when it reaches 10 Mbytes in size with the same suffixes and number of copies as the access_log file.

The /etc/logadm.conf entries for the preceding Apache log rotation examples look similar to the following:

# cat /etc/logadm.conf
.
.
.
/var/apache/logs/error_log -s 10m
/var/apache/logs/access_log -s 100m

For more information, see logadm(1M).

You can use the logadm command as superuser or by assuming an equivalent role (with Log Management rights). With role-based access control (RBAC), you can grant non-root users the privilege of maintaining log files by providing access to the logadm command.

For example, add the following entry to the /etc/user_attr file to grant user andy the ability to use the logadm command:

andy::::profiles=Log Management

Or, you can set up a role for log management by using the Solaris Management Console. For more information about setting up a role, see "Role-Based Access Control (Overview)" in System Administration Guide: Security Services.

Articles

BigAdmin - Submitted Tech Tip System Log Rotation Utilizing logadm (for Solaris 9 OS Only) by Kristopher March

Today, most system administrators are looking for ways to maximize their time at work and are always finding new and efficient ways of carrying out their daily tasks: The built-in Solaris tool, logadm, is one way to do so. New in the Solaris 9 Operating System, logadm is found in /usr/sbin.

Monitoring logs of all types found on a Solaris OS-based machine is a role most system administrators assume on a daily basis. The most typical system log of interest would be /var/adm/messages. On a busy system, this log file tends to grow rather quickly, thus making log examination a burdensome task. Numerous other log files could exist on a Sun server running the Solaris OS, all depending on the configuration. Other applications send information to a log file. Whether it is informational or critical, the text notices contained within these log files often provide pertinent data that should be viewed frequently.

Other log files to note:

In this Tech Tip, we will discuss the appropriate steps needed to set up and utilize logadm. In the default configuration, the /var/adm/messages file is already configured to be rotated. Therefore, we will use the sshdlog as our example. For most environments, running SSH is a critical component to secure machines. With the combination of Secure FTP and many users logging in, there is the potential for this file to grow rather quickly over a 24-hour period. Also, the policies of some computer centers require these files to be archived for a certain amount of time.

Configuration

logadm is an ideal tool to be run from cron. logadm uses a configuration file found in the /etc directory. It is called logadm.conf, and there are a couple of ways to customize this file to best suit your needs.

In this example, we will edit the file directly using the VI editor. Following the completion of any modification to the /etc/logadm.conf file, we run the logadm -V file to check for any errors and to validate the file before going further in our process.

Follow these steps to get started.

*must be completed as uid: root

1. Verify you are running the correct version of the Solaris OS. It should be version 9.

% /usr/bin/uname -r      # this will show the version number 5.9

2. Determine which files need to be rotated on a daily basis. Again, we use the sshdlog found under /var/adm/ in this example. Most other files will work.

3. Make a copy of the logadm in case you need to revert back to the default file.

% cp /etc/logadm.conf  /etc/logadm.conf.orig

4. Edit the file using VI.

% vi /etc/logadm.conf

5. The syntax for log rotating is as follows:

logname  <space> options

Here we use a very simple configuration to rotate these files. Insert the following line: 

/var/adm/sshdlog -C 8 -P 'Sat Dec  6 08:10:00 2003' -a 'kill -HUP `cat \
	/var/run/sshd2_22.pid`'

As you can see from the line above, we specified /var/adm/sshdlog as our log to rotate.

The options I choose to use are explained here:

-C -- This option specifies the maximum number of log files to keep. If exceeded, the oldest file will be deleted to keep the number at 8. If you need to keep these files longer, change the number to a larger value.

-P -- This option is used to specify a timestamp for the log?s last rotation.

-a -- This option is used to specify a post command that is run once after all logs have been rotated. Although not entirely necessary, I send a kill signal to the SSH parent PID to reread any configuration files and send a message to the sshdlog.

6. Once finished editing your file, save it and validate the sshdloglogadm.conf as specified above.

Many other useful options can be used with the logadm tool. The best reference available is the man pages on this subject, which are available in the standard man files location: /usr/bin/man logadm.

Setting Up logadm to Run from cron

Now that you have your logadm configuration file updated and validated, it's time to set up cron to automatically run logadm so that your log file will be rotated.

As root, open the crontab file and start a new line. Note: I chose to run logadm every night at 11:58 pm. 

/usr/bin/crontab -e    - Insert the following line. 
58 23 * * * /usr/sbin/logadm

Specified in cron without any options, logadm will use the default configuration file found in /etc/.

Congratulations! You have successfully set up log rotating and are on your way to adding other log files that you see fit. Any logs now specified will automatically rotate without any user intervention.

Reference

docs.sun.com man pages section 1M System Administration Commands

logadm(1M)– manage endlessly growing log files

OPTIONS

OPERANDS

EXAMPLES

FILES

ATTRIBUTES

SEE ALSO

NOTES

SunOS 5.9 Last Revised 6 Dec 2001

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

Copyright © 1996-2021 by Softpanorama Society. www.softpanorama.org was initially created as a service to the (now defunct) UN Sustainable Development Networking Programme (SDNP) without any remuneration. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License. Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.

This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...

 You can use PayPal to to buy a cup of coffee for authors of this site

Disclaimer:

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: January 09, 2020