Home Switchboard Unix Administration Red Hat TCP/IP Networks Neoliberalism Toxic Managers
May the source be with you, but remember the KISS principle ;-)
Skepticism and critical thinking is not panacea, but can help to understand the world better

Installation of Grid Engine Master Host

News Installation Planning Recommended Links Son of Grid Engine Installation of Son of Grid Engine 8.1.8 RPMs for Master Host Installation of the Execution Hosts
SGE implementations Installation of SGE on CentOS 7 Usage of NFS Network File System (NFS) Mounting NFS directory owned by root Adding NFS partitions to /etc/fstab
Private and Public key managemen Passwordless SSH login SSH Configuration RHEL NTP configuration Troubleshooting NTP on Red Hat Linux SGE Parallel Environment
SGE Queues Grid Engine Config Tips Configuring Hosts From the Command Line Perl Admin Tools and Scripts Humor Etc


We will assume that installation is performed on RHEL 6.5 or 6.6.  We also assume that NFS is used for sharing files with master host.

Degree of sharing is not that important but generally $SGE_ROOT/$SGE_CELL should be shared.  Efficiency consideration that are sited by many are overblown and without careful measurements and determining real bottleneck you might fall into classical trap called "premature optimization". And as Donald Knuth used to say "Premature optimization is the source of all evil". and long before him Talleyrand gave the following advice to young diplomats: "First and foremost, not too too much zeal".  Just substitute "young diplomats" for novice SGE administrators.

The same issue applies to a choice between classic spooling vs. Berkeley DB. Without measurements the selection of Berkeley DB is fools gold.

We will use Son of Grid Engine as an example. Installation for other flavors of Grid Engine is very similar, although some like Oracle Grid engine and Univa are distributed as tar files, not RPMs (see Oracle Grid Engine for some information about tar files based installation).  RPMs produce huge pain with dependencies. The problem is that for Son of Grid Engine 8.1.8 they are not documented and that makes this task painful "trial and error" process. 

Installation consists of two main parts:

Some people also install shadow master on a different host which serves as backup headnode. This might help some maintenance tasks, but from the point of view of reliability does not get you much. It is simpler to have a cold reserve and move drives and network cable if head node hardware went south (chances for this event are pretty slim with modern hardware -- headnode is not loaded server that operates on the limit of its temperature regime, even if it simultaneously serves as NFS server).  Head node can benefit from using SSDs. For simplicity we will avoid installing it.  Remember the here we are discussing small clusters.

The master host installation procedure creates the appropriate directory hierarchy that the master daemon requires and starts the Grid Engine master daemon sge_qmaster on the master host (aka head node).  The master host is also registered as a host with administrative and submit privileges.

If you do not plan to run jobs on master hosts (and for sizable cluster you should not do it) you do not need to install execution host on the head node. and if you install it you can later remove it from list of running daemons and shut it down.

If, at any time during the installation, you think something went wrong, you can quit the installation procedure and restart it. All installation is of master daemon is less then 30 min operation.

RHEL checklist

Installation Planning Checklist


Content of RPM and installation from RPM

RPMs shipped with SGE are not real RPMs. Additional installation using SGE installation scripts is still required. They are more like proxy for tar files.   In addition to storing files RPM serve as a tool to check presence of necessary libraries and Perl modules . Attempt to install binaries compiled for RHEL will fail on SLES because of libraries problems. Welcome to RPM hell... 

Content  of RPM can be viewed by using rpm -qlp file command

You can relocate installation directory from the default location /opt/SGE during installation using RPM option --prefix, for example

rpm -iv --prefix=/opt/sge/sge gridengine-8.1.8-1.el6.x86_64.rpm

But /opt/sge is good enough for most cases.

After that "real" installation should be performed as described below

For example see instructions for installation of Son of Grid engine from RPMs

Exporting SGE_ROOT and /opt/sge directory

RPMs installation creates the installation directory. You can also create it beforehand, probably as a separate partition. For example:

mkdir /opt/sge

After that you need to create and export SGE_ROOT environment variable.

  1. Export SGE-ROOT env. variable: 
    export SGE_ROOT=/opt/sge
  2. Before exporting the directory to all computational nodes, ensure that the file systems and directories that are contain the Grid Engine software as well as the spool and configuration files are set up properly by setting the access permissions as defined in File Access Permissions.  This is done by running
    ./util/ $SGE_ROOT
    This step is important as it creates several SUID-root executables

    Note: this utility requres that gridengine-execd-8.1.8-1.el6.x86_64  RPM be installed on the master host too.

    We will set the the file ownership and permission to
       UserID:         0
       GroupID:        0
       In directory:   /opt/sge
    We will also install the following binaries as SUID-root:
    Do you want to set the file permissions (yes/no) [NO] >> yes
    Verifying and setting file permissions and owner in >bin<
    Verifying and setting file permissions and owner in >examples<
    Verifying and setting file permissions and owner in >inst_sge<
    Verifying and setting file permissions and owner in >install_execd<
    Verifying and setting file permissions and owner in >install_qmaster<
    Verifying and setting file permissions and owner in >lib<
    Verifying and setting file permissions and owner in >mpi<
    Verifying and setting file permissions and owner in >pvm<
    Verifying and setting file permissions and owner in >util<
    Verifying and setting file permissions and owner in >utilbin<
    Verifying and setting file permissions and owner in >doc<
    Verifying and setting file permissions and owner in >man<
    Verifying and setting file permissions and owner in >qmon<
    Verifying and setting file permissions and owner in >start_gui_installer<
    Your file permissions were set

Installing the Grid Engine Master Host Interactively

Full installation includes the following tasks:

 GUI installer does not work in Son of Grid Engine 8.1.8. So you need to use command line installer.

Preliminary Steps

  1. Create sgeadmin account. If it was already creates by RPMs change UID and GID to those that you consider right for this purpose.
  2. Set $SGE_ROOT environment variable and confirm that have set the $SGE_ROOT it properly:
    # echo $SGE_ROOT
  3. Change directory to the installation directory, $SGE_ROOT
    cd $SGE_ROOT
  4. Add host IP to the /etc/hosts (RHEL puts long name as host name which is not very convenient for SGE purposes)
  5. Add SGE services to /etc/services (for RPM based installation this is typically done by RPM)

Check the content of /etc/services. You need to add two ports that are used by SGE. But default those are 6444 and 6445. You can change them to higher ports if necessary. (typically people use the default ports 6444 and 6445, but your mileage may vary; for example if you need two SGE instances they should use different ports)

If lines are missing (RPMs add those lines) then add lines manually

sge_qmaster     6444/tcp                # Grid Engine Qmaster Service
sge_qmaster     6444/udp                # Grid Engine Qmaster Service
sge_execd       6445/tcp                # Grid Engine Execution Service
sge_execd       6445/udp                # Grid Engine Execution Service

Run the command line installer

Set an environment variable and then install the qmaster as such:

export SGE_ROOT=/opt/sge

Now go through the interactive install process:

Now that we are back to a shell (finally) we need to add a few things to our root .bashrc so that we can access the SGE binaries. Add the following lines to /root/.bashrc

   # SGE settings
   export SGE_ROOT=/usr/sge
   export SGE_CELL=default
   if [ -e $SGE_ROOT/$SGE_CELL ]
      . $SGE_ROOT/$SGE_CELL/common/

And then be sure to re-source your .bashrc

. /root/.bashrc

Now we can add our own username as an admin so that we can manage the system without becoming root.

qconf -am <myusername>

Post install steps

Export a common directory from the qmaster host

Most SGE installation share the whole /opt/sge directory from the master host with Apps/sge as a subdirectory.  It should be mounted under the same name on the execution host. See Usage of NFS in Grid Engine.


Check if you can mount it on onr of execution hosts

If master host is simultaneously NFS server restart the NFS daemon on qmaster host to reread export file:

# service nfs restart
Shutting down NFS mountd:                                  [  OK  ]
Shutting down NFS daemon:                                  [  OK  ]
Shutting down NFS quotas:                                  [  OK  ]
Shutting down NFS services:                                [  OK  ]
Starting NFS services:                                     [  OK  ]
Starting NFS quotas:                                       [  OK  ]
Starting NFS daemon:                                       [  OK  ]
Starting NFS mountd:                                       [  OK  ]

Create ssh passwordless keys and distribute public keys to execution hosts

Create passwordless login environment.

Tip: If you already have configured it just copy file authorized_hosts from already configured execution host.

cd /root/.ssh
scp sge01:/root/.ssh/authorized_hosts  . 
Check ssh access from the master host  to the node on which you install the execution host (b5 in the example below):
[0]root@m17: # ssh b5
The authenticity of host 'b5 (' can't be established.
RSA key fingerprint is 18:35:6e:96:11:77:27:fc:ac:1c:8e:46:36:2b:ae:2b.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'b5,' (RSA) to the list of known hosts.
Last login: Thu Jul 26 08:29:41 2012 from


Recommended Links

Google matched content

Softpanorama Recommended

Top articles



The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

Copyright 1996-2018 by Dr. Nikolai Bezroukov. was initially created as a service to the (now defunct) UN Sustainable Development Networking Programme (SDNP) in the author free time and without any remuneration. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License. Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.

This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...

You can use PayPal to make a contribution, supporting development of this site and speed up access. In case is down you can use the at


The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the author present and former employers, SDNP or any other organization the author may be associated with. We do not warrant the correctness of the information provided or its fitness for any purpose.

The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: March 12, 2019