Softpanorama

May the source be with you, but remember the KISS principle ;-)
Home Switchboard Unix Administration Red Hat TCP/IP Networks Neoliberalism Toxic Managers
(slightly skeptical) Educational society promoting "Back to basics" movement against IT overcomplexity and  bastardization of classic Unix

Prolog and epilog scripts

News Grid Engine Recommended Links Listing all existing queues Adding a new queue Modifying queue parameters Removing a queue
Some interesting queue params queue_conf - Grid Engine queue configuration file format slots queue attribute Excluding SGE execution host from scheduling Restricting number of slots per server Slot hacking SGE_types
qconf qdel qhold qalter -- Change Job Priority qmod qsub qstat
SGE hostgroups Execution hosts Resource Quotas  SGE Consumable Resources Prolog and epilog scripts SGE Submit Scripts Submitting parallel OpenMPI jobs
Monitoring Queues and Jobs Controlling Queues and Jobs Most important optiona of qconf Parallel environment ulimit problem with infiniband in SGE  Humor Etc

Prolog and epilog scripts can be used on three levels:

prolog

The executable path of a shell script that is started before execution of Sun Grid Engine jobs with the same environment setting as that for the Sun Grid Engine jobs to be started afterwards.

An optional prefix "user@" specifies the user under which this procedure is to be started.

The procedures standard output and the error output stream are written to the same file used also for the standard output and error output of each job. This procedure is intended as a means for the Sun Grid Engine administrator to automate the execution of general site specific tasks like the preparation of temporary file systems with the need for the same context information as the job.

Each sge_execd(8) may use a private prolog script. Correspondingly, the execution host local configurations is can be overwritten by the queue configuration (see queue_conf(5) ). Changing prolog will take immediate effect.

The default for prolog is the special value NONE, which prevents from execution of a prolog script.

The following special variables expanded at runtime can be used (besides any other strings which have to be interpreted by the procedure) to constitute a command line:

These three flags correspond to the DRMAA job template attribute 'drmaa_transfer_files' (see drmaa_attributes(3) ).

The global configuration entry for this value may be overwritten by the execution host local configuration.

Exit codes

Exit codes for the prolog attribute can be interpreted based on the following exit values:

epilog

The executable path of a shell script that is started after execution of Sun Grid Engine jobs with the same environment setting as that for the Sun Grid Engine jobs that has just completed. An optional prefix "user@" specifies the user under which this procedure is to be started. The procedures standard output and the error output stream are written to the same file used also for the standard output and error output of each job. This procedure is intended as a means for the Sun Grid Engine administrator to automate the execu- tion of general site specific tasks like the cleaning up of temporary file systems with the need for the same context information as the job. Each sge_execd(8) may use a private epilog script. Correspondingly, the execution host local configurations is can be overwritten by the queue configura- tion (see queue_conf(5) ). Changing epilog will take immediate effect.

The default for epilog is the special value NONE, which prevents from execution of a epilog script. The same special variables as for prolog can be used to constitute a command line.

The global configuration entry for this value may be overwritten by the execution host local configuration.

Exit codes for the epilog attribute can be interpreted based on the following exit values:

shell_start_mode Note: Deprecated, may be removed in future release. This parameter defines the mechanisms which are used to actually invoke the job scripts on the execution hosts. The following values are recognized:

unix_behavior  If a user starts a job shell script under UNIX interac- tively by invoking it just with the script name the operating system's executable loader uses the informa- tion provided in a comment such as `#!/bin/csh' in the first line of the script to detect which command interpreter to start to interpret the script. This mechanism is used by Sun Grid Engine when starting jobs if unix_behavior  is defined as shell_start_mode.

posix_compliant  POSIX does not consider first script line comments such a `#!/bin/csh' as significant. The POSIX standard for batch queuing systems (P1003.2d) therefore requires a compliant queuing system to ignore such lines but to use user specified or configured default command inter- preters instead. Thus, if shell_start_mode is set to posix_compliant  Sun Grid Engine will either use the command interpreter indicated by the -S option of the qsub(1) command or the shell parameter of the queue to be used (see queue_conf(5) for details).

script_from_stdin  Setting the shell_start_mode parameter either to posix_compliant  or unix_behavior  requires you to set the umask in use for sge_execd(8) such that every user has read access to the active_jobs directory in the spool directory of the corresponding execution daemon. In case you have prolog and epilog scripts configured, they also need to be readable by any user who may exe- cute jobs. If this violates your site's security policies you may want to set shell_start_mode to script_from_stdin. This will force Sun Grid Engine to open the job script as well as the epilog and prolog scripts for reading into STDIN as root (if sge_execd(8) was started as root) before changing to the job owner's user account. The script is then fed into the STDIN stream of the command interpreter indicated by the -S option of the qsub(1) command or the shell parameter of the queue to be used (see queue_conf(5) for details). Thus setting shell_start_mode to script_from_stdin  also implies posix_compliant  behavior. Note, however, that feeding scripts into the STDIN stream of a command interpreter may cause trouble if commands like rsh(1) are invoked inside a job script as they also process the STDIN stream of the command interpreter. These problems can usually be resolved by redirecting the STDIN channel of those commands to come from /dev/null (e.g. rsh host date < /dev/null). Note also, that any command-line options associated with the job are passed to the executing shell. The shell will only forward them to the job if they are not recognized as valid shell options.

Changes to shell_start_mode will take immediate effect. The default for shell_start_mode is posix_compliant.

This value is a global configuration parameter only. It can- not be overwritten by the execution host local configuration.


Top Visited
Switchboard
Latest
Past week
Past month

NEWS CONTENTS

Old News ;-)

RE [GE users] SGE Rescheduling - Gruhn Daniel J Contractor AF-A9IT - net.sunsource.gridengine.users - MarkMail

Subject: RE: [GE users] SGE Rescheduling permalink
From: Gruhn Daniel J Contractor AF/A9IT ([email protected])
Date: Aug 3, 2006 5:04:53 am
List: net.sunsource.gridengine.users

One additional thing, I don't think the bug with rescheduling is fixed yet.
That bug is that rescheduling seems to be an asyncronous process. That is,
the rescheduled job may be able to get started before the original job is
killed. In my case this makes a difference and I have to compensate for it.

Dan

//SIGNED//
Daniel J.Gruhn, CTR (Group W Inc.)
HQ USAF/A9IT
Studies & Analyses, Assesments and Lessons Learned

-----Original Message-----
From: Reuti [mailto:[email protected]]
Sent: Thursday, August 03, 2006 7:33 AM
To: [email protected]
Subject: Re: [GE users] SGE Rescheduling

Hi,

Am 02.08.2006 um 23:14 schrieb Sreenath Nampally:

Hello,

Could someone explain the sequence of events that happen in SGE (both
on qmaster and exec host) when a job is rescheduled and suspended?
What signals are sent to the job ?

if the job gets supended, it will get a SIGSTOP which you can't catch. But
you could submit the job with -notify, to get a warning before, which you
can catch. Have a look at `man qsub`, and you could even redefine the
signal: `man sge_conf`section execd_params. But be aware, that the signal
will be send to the whole process group, and this might need proper handling
in the jobscript and the compiled program.

If you reschedule a job, it will be killed, and also before this you could
get a warning by -notify. But I think, you will only get the information
about the kill, but not the reason that it will be rescheduled. Only during
the next run, you can test the variable RESTARTED, whether it's 1. If you
need a more sophisticated handling, you can also try to use the
checkpointing interface.

HTH - Reuti

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Recommended Links

Google matched content

Softpanorama Recommended

Top articles

Sites

Top articles

Sites



Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotesSomerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose BierceBernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOSProgramming Languages History : PL/1 : Simula 67 : C : History of GCC developmentScripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-MonthHow to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D


Copyright © 1996-2021 by Softpanorama Society. www.softpanorama.org was initially created as a service to the (now defunct) UN Sustainable Development Networking Programme (SDNP) without any remuneration. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License. Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.

This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...

You can use PayPal to to buy a cup of coffee for authors of this site

Disclaimer:

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: March, 12, 2019