Linux Installation in Large Enterprise Environment

There are multiple ways to install and run  Linux. Each distribution contain documentation that covers this process. This page contains mostly tips, that are important enough to be more widely known. There are separate guidelines that cover installation in more details:

1. Configure RAID and Create Dell Service Partition
2. Boot from Suse SP2 Installation DVD or Other Medium
3. Partition the Harddrives
4. Select set of packages to be installed
5. Configure root and Perform network configuration
6. Reboot the system and perform post-install configuration
7. Harden the server

There are two versions of Linux 32 bit and 64 bit. 64-bit version requires CPUs with extended instruction set (AMD64 and Intel 64 CPU). For servers 64-bit version is better as it allow to access more then 16G of RAM and on configurations from 4G to 16G works faster.

The only exception is virtual machine environment: 32-bit kernel is better for VMware environment.

Availability of Linux for 64-bit Intel and AMD hardware platforms does not necessarily mean that all the applications included have already been ported to 64-bit platforms. Often this is not true and a lot of application even commercial one are distributed only for 32-bit mode. 

To be executed correctly, every application requires a range of libraries. Unfortunately, the names for the 32-bit and 64-bit versions of these libraries are identical. They must be differentiated from each other by placement:

• To retain compatibility with the 32-bit version, the libraries are stored at the same place in the system as in the 32-bit environment. The 32-bit version of libc.so.6 is located under /lib/libc.so.6 in both the 32-bit and 64-bit environments.
• All 64-bit libraries and object files are located in directories called lib64. The 64-bit object files you would normally expect to find under /lib, and /usr/lib are now found under /lib64, and /usr/lib64. This means that there is space for the 32-bit libraries under /lib and /usr/lib, so the filename for both versions can remain unchanged.

Subdirectories of 32-bit /lib directories whose data content does not depend on the word size are not moved.

• The 64-bit kernels for x86_64 offer both a 64-bit and a 32-bit kernel ABI (application binary interface). The latter is identical with the ABI for the corresponding 32-bit kernel. This means that the 32-bit application can communicate with the 64-bit kernel in the same way as with the 32-bit kernel.
• The 32-bit emulation of system calls for a 64-bit kernel does not support all the APIs used by system programs. This depends on the platform. For this reason, a small number of applications, like lspci, must be compiled
• A 64-bit kernel can only load 64-bit kernel modules that have been specially compiled for this kernel. It is not possible to use 32-bit kernel modules.

In the case of manually updated libraries, you need to resolve these issues yourself.

The Linux Boot Process

The Linux boot process consists of several stages each represented by another component. The following list briefly summarizes the boot process and features all the major components involved.

1. BIOS After the computer has been turned on, the BIOS initializes the screen and keyboard and tests the main memory. Up to this stage, the machine does not access any mass storage media. Subsequently, the information about the current date, time, and the most important peripherals are loaded from the CMOS values. When the first hard disk and its geometry are recognized, the system control passes from the BIOS to the boot loader.
    
2. MBR boot Loader The first physical 512-byte data sector of the first device defined in BIOS load chain is loaded into the main memory and the program that resides at the beginning of this sector takes over. It usually loads another loader but it can do so only if the latter is located at fixed disk addresses as no file system is present. That's why it is often referred to as Master Boot Record (MBR). The boot loader then passes control to the actual operating system, in this case, the Linux kernel at /boot partition. 
    
3. Kernel and initramfs MBR loads both the kernel and an initial RAM–based file system (initramfs) into memory. initramfs is a small cpio archive It contains a small executable that uses BIOS calls called init that handles the mounting of the real root file system.  It provides a minimal Linux environment that enables the execution of programs before the actual root file system is created. This minimal Linux environment is loaded into memory by BIOS routines and does not have specific hardware requirements other than sufficient memory.
   • init on initramfs This program performs all actions needed to mount the proper root file system on the installation media, like providing kernel functionality for the needed file system and device drivers for typical mass storage controllers. After the root file system has been found, it is checked for errors and mounted. If this has been successful, the initramfs is cleaned and the init program on the root file system is executed.
      
   • Before the root file system can be mounted and the operating system can be started, the kernel needs the corresponding drivers to access the device on which the root file system is located. These drivers may include special drivers for certain kinds of hard drives or even network drivers to access a network file system. The needed modules for the root file system may be loaded by init on initramfs. After the modules are loaded, udev provides the initramfs with the needed devices. Later in the boot process, after changing the root file system, it is necessary to regenerate the devices. This is done by boot.udev with the command udevtrigger.

The main purpose of init on initramfs is to prepare the mounting of and access to the real root file system. Depending on your system configuration, init is responsible for the following tasks.

• Loading Kernel Modules. Depending on your hardware configuration, special drivers may be needed to access the hardware components of your computer (the most important component being your hard drive). To access the final root file system, the kernel needs to load the proper file system drivers.
• Providing Device Special Files. For each loaded module, the kernel generates device events. udev handles these events and generates the required device special files on a RAM file system in /dev. Without those special files, the file system and other devices would not be accessible.
• Managing RAID and LVM Setups. If you configured your system to hold the root file system under RAID or LVM, init sets up LVM or RAID to enable access to the root file system later.
• Managing Network Configuration. If you configured your system to use a network-mounted root file system (mounted via NFS), init must make sure that the proper network drivers are loaded and that they are set up to allow access to the root file system.

When init is called during the initial boot as part of the installation process, its tasks differ from those mentioned earlier:

• Finding the Installation Medium. As you start the installation process, your machine loads an installation kernel and a special initrd with the YaST installer from the installation medium. The YaST installer, which is run in a RAM file system, needs to have information about the location of the installation medium to access it and install the operating system.
   
• Initiating Hardware Recognition and Loading Appropriate Kernel Modules. The boot process starts with a minimum set of drivers that can be used with most hardware configurations. init starts an initial hardware scanning process that determines the set of drivers suitable for your hardware configuration. The names of the modules needed for the boot process are written to INITRD_MODULES in /etc/sysconfig/kernel. These names are used to generate a custom initramfs that is needed to boot the system. If the modules are not needed for boot but for coldplug, the modules are written to /etc/sysconfig/hardware/hwconfig-*. All devices that are described with configuration files in this directory are initialized in the boot process.
   
• Loading the Installation System or Rescue System. As soon as the hardware has been properly recognized, the appropriate drivers have been loaded, and udev has created the device special files, init starts the installation system, which contains the actual YaST installer, or the rescue system.
   
• Starting YaST. Finally, init starts YaST, which handles package installation and system configuration.

Installation from CD/DVD

CD/DVD are default installation medium for YAST. Local CD/DVD installations are very much the same for Solaris and Linux. The installation program prompts for information when it is required. A menu of options is presented in some parts so that you can select the appropriate choice or choices. As with a Solaris installation, Linux will perform the following operations.

1. Probe the hardware for devices and load the appropriate device drivers.
2. Select language
3. Accept license agreement
4. Select a time zone.
5. Select GUI (Gnome of KDE). Gnome will be used in lectures.
6. Assign a host name.
7. Set up a network setup type (DHCP or static IP)
   1. If static IP, you will be asked to enter the IP address, subnet mask, default gateway, and DNS servers to use.
   2. You can also use DHCP.
       
8. Select installation type (the initial not upgrade ).
    
9. Repartition hard disk:
   • Desktop has 150G drive
   • Laptops have 100G drive
      
10. Prompt for a automatic or manual disk layout for the file systems. Select manual and allocate 4G for swap and 10G for root and 8G for /home partitions (if you have a separate /home partition you can upgrade system without need to restore your data)
     
11.  Select filesystem for partitions. You should use ext4 as your filesystem
     
12. Prompt for the software bundle or packages to install.  You need carefully browse and add your own choices of software packages to the default selection proposed by the installer, so click on the Change button and select Software from the menu, which brings you to the main Software Selection screen
    • suse installs vim and screen automatically
    • Add several packages that I will use in lectures:
      • vino -- VNC for Gnome
      • mc (Midnight commander, requres glib),
      • gvim
      • expect (requres TCL).
      • Wireshark
      • nmap
      • Other packages that you might be interested in.

Modern Linux usually recognizes most devices and installs without major hardware problems on standard desktop and laptop.

Note: In Solaris, the installation program will use the current console as an I/O device, and if you are using a serial interface, it will use that same serial interface. In Linux for IA32 , the default console device used for installation is the graphics card. If you need to install the system using a serial interface, you need to specifically tell the installation program to do so by adding the nofb console=ttyS0 kernel boot options.

Installing using remote display

In Solaris, you can install using the local graphic display or the serial interface. In Linux, you can install using the local graphic display, serial interface, and a few other remote modes.

In RHEL, you have the following options:

In SLES, you have the following options:

Partitioning

The two minimum partitions are the root partition, signified with a forward slash (/), and a swap partition, which Linux uses as virtual memory. You also would benefit form a separate /home partitions for the reasons explained above. Traditionally, the size of the swap partition is equivalent to (or a multiple of) the amount of hardware RAM you have installed. 

If you have only these two partitions, YaST will put all standard directories on the same partition as /. These are

• /bin—Binary files.
• /boot—Files to boot Linux.
• /dev—Device drivers.
• /etc—Configuration files.
• /home—Each user gets a data directory under this one (should generally be a separate partition)
• /lib—Libraries, the underlying code for many applications.
• /media—Removable disks and drives (floppy discs, CD-ROMs, Zips).
• /mnt—The mount directory, where all partitions (mount points) are listed.
• /opt—Some applications install to this directory instead of /usr.
• /proc—A dynamic directory where all running processes are logged. Constantly changing.
• /root—Not to be confused with plain / (also called "root"), this is the home directory for the root user.
• /sbin—System binaries, usually files that need to run during startup.
• /srv—Services.
• /tmp—Temporary files.
• /usr—Most applications, libraries and man files.
• /var—Variable data, including syslog, print spoolers, mail files, cron files, are stored here.

Fun with Configuration Files

Instead of a single file linux configuration files are often spitted into chunks which are stored then into directories. The main file just assembles the chunks. As in other areas of life this quest to more flexibility actually sometimes backfire and turns into its opposite.

Some of those "hacked" files include /etc/profile (with chunks stored at /etc/profile.d), crontab (crontab.d), syslog (actually Suse uses syslog-ng instead of regular syslog), etc.  RC files are structures similarly to Solaris with links to each run level.

Some of the more important system-related configuration files in this directory are

• fstab — corresponds to Solaris vfstab. Lists mountable storage devices.  Structures are similar to Solaris and lists each device's mount point, its location on the partition table, instructions on whether it should mount automatically on boot, and a few other mounting options.
• inittab —The system initialization table defines the different runlevels that the system uses at boot to start your computer. Changes to this file (which can be made manually, but also in YaST) can determine whether you boot to a graphical GUI or to text mode. 
• modprobe.conf and modprobe.conf.local —The main modprobe.conf file contains directions and options used when loading kernel modules to operate various pieces of hardware, such as sound cards, USB devices, network cards, and the like. The contents of this file are used during boot. SUSE updates this file from time to time so manual chances can be overwritten. You can use modprobe.conf.local to keep those changes from being overwritten.
• passwd and shadow. Like Solaris Suse uses /etc/shadow to store hashes and password expiration information.

Choosing and Using a Shell

Suse uses different default shell. But it is a better interactive shell then Korn shell.

Also the default ksh in linux is broken and ksh93 should be installed. It is also more reliable for complex scripts.

NEWS CONTENTS

• 20110429 : Disabling avahi-daemon Len ( Disabling avahi-daemon Len, Apr 29, 2011 )
• 20110427 : Linux installation kickstart for Oracle database - Oracle Wiki ( Linux installation kickstart for Oracle database - Oracle Wiki, Apr 27, 2011 )

Old News ;-)

[Apr 29, 2011] Disabling avahi-daemon Len

It's a useless daemon for servers. It's enabled by default in RHEL 6 and OEL 6. Should be disabled.

One of the things I quickly found to be bothering me is the fact that there was an apparently long and unexplicable delay for all new network connections which resembled to a dns resolving. No reason for lengthy dns resolving though. So I did a strace:

socket(PF_FILE, SOCK_STREAM, 0)         = 4
fcntl64(4, F_GETFD)                     = 0
fcntl64(4, F_SETFD, FD_CLOEXEC)         = 0
connect(4, {sa_family=AF_FILE, path="/var/run/avahi-daemon/socket"}, 110) = 0
fcntl64(4, F_GETFL)                     = 0x2 (flags O_RDWR)
fstat64(4, {st_mode=S_IFSOCK|0777, st_size=0, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f35000_
llseek(4, 0, 0xbfa7d918, SEEK_CUR)     = -1 ESPIPE (Illegal seek)
write(4, "RESOLVE-ADDRESS 10.0.0.6\n", 25) = 25read(4,  <unfinished ...>

the results shows a connection to a avahi-daemon which I have no ideea what is good for so I should not need it. I disabled it in /etc/default/avahi-daemon

cat /etc/default/avahi-daemon # 0 = don't start, 1 = start
AVAHI_DAEMON_START=0

Hope it helps.

[Apr 27, 2011] Linux installation kickstart for Oracle database - Oracle Wiki

#############################################################################
#
# This Kickstart file is used to install the minimum RHEL server for use with
# Oracle database. It also removes unneeded services.
#
# The default root password is: password
#
# Values to change before using are:
# - Set the network - The hostname & Network configuration (DHCP/Static)
# - Partitioning & Swap size
# - Set a different timezone if not located in Israel
# - If going to use rawdevices comment the "chkconfig rawdevices off" line
# - Set the shmmax kernel parameter to the correct value
# - If you are not going to install Oracle RAC, you can comment the hangcheck line in this file
#
# This is OEL/RHEL 3/4. For OEL/RHEL 5, comment indicated lines!!!
#
# Things to do after the server is installed:
# Change the root and oracle accounts password
# Create the needed filesystem and set the owner to oracle
# Change the ORACLE_BASE, ORACLE_SID environment variables in 
# ~oracle/.bash_profile to the correct values
# Set the server default run level to 3 or 4 (if using NFS/CIFS)
#############################################################################

# Uncomment the next line to enable interactive installation
#interactive

# Comment the next line to use GUI installation
text

install
# reboot after installation
reboot

# From where to get the installation disks
cdrom
#url --url http://myserver/redhat/

lang en_US.UTF-8
#OEL5: langsupport deprecated 
langsupport --default=en_US.UTF-8 en_US.UTF-8
keyboard us
xconfig --resolution 800x600 --depth 16 --defaultdesktop gnome

# Change the hostname and adjust the eth configuration for your environment
# hostname_$$ adds random number to hostname to try and avoid name conflicts (tested on OEL 5.2+)
network --device eth0 --noipv6 --bootproto dhcp --hostname srvoracle01_$$
#network --device eth0 --bootproto static --ip x.y.z.v --netmask 255.255.255.0 --gateway x.y.z.v --nameserver x.y.z.t,x.y.z.v --hostname srvoracle01_$$

rootpw --iscrypted $1$uKWECPhN$Im66UG8MpWd2/kpcHoyuy/
firewall --disabled
selinux --disabled
authconfig --enableshadow --enablemd5

# Set a different timezone if not located in israel
timezone Asia/Jerusalem

bootloader --location=mbr --append="rhgb quiet"
# OEL5: for multipath systems 
#bootloader --location=mbr --driveorder=sda,mapper/mpathX --append="verbose"
# OEL5: for boot from SAN, match driveorder to BIOS boot disk order
#bootloader --location=mbr --driveorder=mapper/mpathY,sda --append="nousbstorage verbose"
# OEL5: any options to zerombr deprecated
#zerombr
zerombr yes
# initlabel clears the label present on most used disks; helps LVM/Oracle ASM to see disks as new
clearpart --all --initlabel --drives=sda
# for multipath systems
#clearpart --all --initlabel --drives=sda,mapper/mpath0,mapper/mpath1,mapper/mpath2,mapper/mpath3

# for boot from SAN, use your boot LUN. E.g. --ondisk=mapper/mpath4 or mapper/mpath2
part /boot --asprimary --fstype ext3 --size=100 --ondisk=sda
# Setup LVM
part pv.3 --size=1 --grow --ondisk=sda
volgroup vgsystem --pesize=32768 pv.3

# Create the file systems and logical volumes; assigning useful names to LVs
logvol / --fstype ext3 --name=lvroot --vgname=vgsystem --size=2048
logvol /tmp --fstype ext3 --name=lvtmp --vgname=vgsystem --size=2048
logvol /usr --fstype ext3 --name=lvusr --vgname=vgsystem --size=3072
logvol /usr/local --fstype ext3 --name=lvusrlocal --vgname=vgsystem --size=1024
logvol /var --fstype ext3 --name=lvvar --vgname=vgsystem --size=1024
logvol /var/log --fstype ext3 --name=lvvarlog --vgname=vgsystem --size=1024
logvol /opt --fstype ext3 --name=lvopt --vgname=vgsystem --size=1024
logvol /home --fstype ext3 --name=lvhome --vgname=vgsystem --size=512 --grow

# Edit the swap space to be twice the size of the machine RAM, 
# if you have 8GB of RAM or more set the swap to the same size as the RAM
logvol swap --fstype swap --name=lvswap --vgname=vgsystem --size=2048

# All packages needed by Oracle installation include Gnome desktop
# NOTE: despite explicitly removing a pkg, dependencies may add them later on. 
# Always clean up RPMs/packages after reboot and before an up2date/yum update
#OEL5: up2date deprecated! use yum
%packages
@ system-tools
@ gnome-desktop
@ dialup
# OEL5: comment out next line
@ compat-arch-support
-openldap-clients
-ckermit
-wireshark
-bluez-pin
-OpenIPMI-tools
-samba-client
e2fsprogs
-screen
-xdelta
-zsh
-nmap
lvm2
-open
# OEL5: comment out next line
kernel-smp
grub
binutils
compat-db
control-center
gcc
gcc-c++
glibc
glibc-common
# OEL5: comment out next line
gnome-libs
libstdc++
libstdc++-devel
make
# OEL5: comment out next line
pdksh
sysstat
# OEL5: comment out next line
xscreensaver
# SNMP makes monitoring server much easier, if configured right
#net-snmp-utils
# add Oracle package at install time to solve many dependencies
# NOTE: package is not on CD! Only yum/up2date. DVD untested
#oracle-validated

# This post script modify all the Oracle database prerequisites, includes oracle user and dba,oinstall groups
%post

# create user and groups
/usr/sbin/groupadd oinstall
/usr/sbin/groupadd dba
/usr/sbin/useradd -m -g oinstall -G dba oracle

# Change kernel parameters
cat >> /etc/sysctl.conf << EOF

##############################
# Oracle Database requirements
##############################

# The max value for shmmax on a 32bit machine is 4294967295
#kernel.shmmax = 536870912

kernel.shmall = 2097152
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 658576
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default = 262144
net.core.wmem_default = 262144
net.core.rmem_max = 1048536
net.core.wmem_max = 1048536
EOF

# Change ulimits for user oracle
cat >> /etc/security/limits.conf << EOF
oracle soft nproc 16384
oracle hard nproc 16384
oracle soft nofile 65536
oracle hard nofile 65536
EOF

# Add hangcheck-timer module for RAC; comment if you don't need RAC
echo "options hangcheck-timer hangcheck_tick=30 hangcheck_margin=180" >> /etc/modprobe.conf
echo "modprobe hangcheck-timer" >> /etc/rc.modules

# Set oracle account environment variables
cat >> /home/oracle/.bash_profile << EOF 

# Oracle environment variables
export ORACLE_BASE=/software/oracle
#export ORACLE_HOME=\$ORACLE_BASE/10g
#export TNS_ADMIN=\$ORACLE_HOME/network/admin
export ORACLE_SID=orcl1
export LD_LIBRARY_PATH=\$ORACLE_HOME/lib
export PATH=\$PATH:\$ORACLE_HOME/bin

if [ "`ls -ld \$ORACLE_HOME | awk '{print \$3" "\$4}'`" != "oracle oinstall" ]
then
echo "The ORACLE_HOME (\$ORACLE_HOME) is not owned by oracle:oinstall"
fi
EOF

#############################
# Remove unneeded services!!!
#############################

# Comment this line if you are using Oracle Clusterware
chkconfig rawdevices off

# Enable Network filesystems support for run level 4 only
chkconfig --level 12356 portmap off
chkconfig --level 12356 nfslock off
chkconfig --level 12356 rpcgssd off
chkconfig --level 12356 rpcidmapd off
chkconfig --level 12356 autofs off
chkconfig --level 4 portmap on
chkconfig --level 4 nfslock on
chkconfig --level 4 rpcgssd on
chkconfig --level 4 rpcidmapd on
chkconfig --level 4 autofs on

# Enable xfs support for run level 5 only
chkconfig --level 12346 xfs off
chkconfig --level 5 xfs on

chkconfig sendmail off
chkconfig cups off
chkconfig hplip off
chkconfig anacron off
chkconfig apmd off
chkconfig arptables_jf off
chkconfig atd off
chkconfig bluetooth off
chkconfig canna off
chkconfig cups-config-daemon off
chkconfig gpm off
chkconfig hidd off
chkconfig hpoj off
chkconfig iiim off
chkconfig isdn off
#OEL5: comment line below; pkg not longer exists
chkconfig pcmcia off
#if using SNMP, uncomment line below
#chkconfig snmpd on
chkconfig rhnsd off
chkconfig auth off

# OEL/RHEL 5 only services
#chkconfig yum-updatesd off
#chkconfig setroubleshoot off
#chkconfig restorecond off
#chkconfig pcscd off
#chkconfig mcstrans off
#chkconfig avahi-dnsconfd off
#chkconfig avahi-daemon off

Recommended Links

Softpanorama Recommended

Top articles

Sites