Softpanorama

May the source be with you, but remember the KISS principle ;-)
Home Switchboard Unix Administration Red Hat TCP/IP Networks Neoliberalism Toxic Managers
(slightly skeptical) Educational society promoting "Back to basics" movement against IT overcomplexity and  bastardization of classic Unix

sge_types - Grid Engine type descriptions

News Grid Engine Recommended Links Listing all existing queues Adding a new queue Modifying queue parameters Removing a queue
Some interesting queue params queue_conf - Grid Engine queue configuration file format slots queue attribute Excluding SGE execution host from scheduling Restricting number of slots per server Slot hacking sge_types
qconf qdel qhold qalter -- Change Job Priority qmod qsub qstat
SGE hostgroups Execution hosts Resource Quotas Submitting binaries in SGE SGE Submit Scripts Submitting parallel OpenMPI jobs  
Monitoring Queues and Jobs Controlling Queues and Jobs Most important optiona of qconf Parallel environment ulimit problem with infiniband in SGE  Humor Etc

The Grid Engine user interface consists of several programs and files. Some command-line switches and several file attributes are types. The syntax for these types is explained in this page.

OBJECT TYPES

These types are used for defining Grid Engine configuration:

object_name An object name is a sequence of up to 512 ASCII printing characters except SPACE, "/", ":", "'", "\", "[", "]", "{", "}", "|", "(", ")", "@", "%", "," or the '"' character itself.

calendar_name A calendar name is the name of a Grid Engine calendar described in calendar_conf(5).

calendar_name := object_name 

ckpt_name A "ckpt_name" is the name of a Grid Engine checkpointing interface described in checkpoint(5).

ckpt_name := object_name 

complex_name A complex name is the name of a Grid Engine resource attribute described in complex(5).

complex_name := object_name 

host_identifier A host identifier can be either a host name or a host group name.

host_identifier := host_name | hostgroup_name 

hostgroup_name A host group name is the name of a Grid Engine host group described in hostgroup(5). Note, to allow host group names to be distinguished easily from host names, a "@" prefix is used.

hostgroup_name := @object_name  

host_name A host name is the official name of a host node. Host names with a domain specification such as "gridmaster.sun.com" are called fully- qualified host names, whereas host names like "gridmaster" are called short host names. Note that the install time parameters default_domain and ignore_fqdn (see bootstrap(5)) affect how Grid Engine deals with host names in general.

The following host names are generally invalid or reserved: global, template, all, default, unknown, none. However, it may sometimes be useful to define a dummy host name of global for convenient use of qhost(1).

jsv_url The jsv_url has following format:

 
jsv_url := jsv_client_url | jsv_server_url 
 
jsv_server_url := [ type ':' ] [ user '@' ] path 
 
jsv_client_url := [ type ':' ] path 
 
type := 'script' 
At the moment only the type script is allowed. This means that path is either the path to a script or to a binary application which will be used to instantiate a JSV process. The type is optional until other types are supported by Grid Engine.

Specifying a user is only allowed for server JSVs. Client JSVs will automatically be started as the submit user, and server JSVs as the admin user if not otherwise specified.

The path has always to be the absolute path to a binary or application.

memory_specifier Memory specifiers are positive decimal, hexadecimal or octal integer constants which may be followed by a multiplier letter. Valid multiplier letters are k, K, m, M, g, G, t, and T, where k means multiply the value by 1000, K multiply by 1024, m multiply by 1000x1000, M multiply by 1024x1024, g multiply by 1000x1000x1000, G multiply by 1024x1024x1024, t multiply by 1000x1000x1000x1000, and T multiply by 1024x1024x1024x1024. If no multiplier is present, the value is just counted in bytes. Whether memory values above the 32-bit limit are representable on 32-bit systems, even for disk space, is system-dependent.

pe_name A PE name is the name of a Grid Engine parallel environment described in sge_pe(5).

pe_name := object_name 
project_name A project name is the name of a Grid Engine project described in project(5).
project_name := object_name 

queue_name A queue name is the name of a Grid Engine queue described in queue_conf(5).

queue_name := object_name 

time_specifier A time specifier either consists of a positive decimal, hexadecimal or octal integer constant, in which case the value is interpreted to be in seconds, or is built from 3 decimal integer numbers separated by colon signs, where the first number counts the hours, the second the minutes and the third the seconds. If a number would be zero it can be left out but the separating colon must remain (e.g. 1:0:1 = 1::1 means 1 hour and 1 second).

user_name A user name can be the name of a login(1) user or of the Grid Engine user object described in user(5).

user_name := object_name 
userset_name A user set name is the name of a Grid Engine access list or department described in access_list(5).
userset_name := object_name  
date_time A date_time value must conform to [[CC]YY]MMDDhhmm[.SS], where:
 
CC denotes the century in 2 digits. 
YY denotes the year in 2 digits. 
MM denotes the month in 2 digits. 
DD denotes the day in 2 digits. 
hh denotes the hour in 2 digits. 
mm denotes the minute in 2 digits. 
ss denotes the seconds in 2 digits (default 00). 
 
If any of the optional date fields are omitted, the corresponding value of the current date is assumed. If CC is not specified, a YY of <70 means 20YY. Use of this option may cause unexpected results if the clocks of the hosts in the Grid Engine pool are out of sync. Also, the proper behavior of this option very much depends on the correct setting of the appropriate timezone, e.g. in the TZ environment variable (see date(1) for details), when the Grid Engine daemons sge_qmaster(8) and sge_execd(8) are invoked.

time A time value must conform to hh:mm:ss, or seconds where:

hh denotes the hour in 2 digits. 
mm denotes the minute in 2 digits. 
ss denotes the seconds in 2 digits (default 00). 
seconds is a number of seconds (used for duration values) 
 
name A name is an arbitrary string of ASCII printing characters, but may not contain "/", ":", "@", "\", "*", or "?".

account_name Identifies the account to which the resource consumption of a job should be charged.

account_name := name 
job_name A job name is a name as above, with the restriction that it cannot start with a digit (to avoid ambiguity with a job number in some contexts).

ar_name An advance reservation name is a name as above, with the restriction that it cannot start with a digit (to avoid ambiguity with an AR number in some contexts).

MATCHING TYPES

These types are used for matching Grid Engine configuration:

expression A wildcard expression is a regular boolean expression that consists of one or more patterns joined by boolean operators. When a wildcard expression is used, the following definition applies:

expression= ["!"] ["("] valExp [")"] [ AND_OR expression ]* 
valExp = pattern | expression 
AND_OR = "&" | "|" 
where:
"!" not operator: negate the following pattern or expression 
"&" and operator: logically and with the following expression 
"|" or operator: logically or with the following expression 
"(" open bracket: begin an inner expression. 
")" close bracket: end an inner expression. 
"pattern" see the pattern definition that follows 
 
If typed at a shell, the expression itself should be quoted to ensure that it is not expanded by the shell. e.g.
"(lx*|sol*)&*64*" any string beginning with either "lx" or 
"sol" and containing "64" 
"rh_3*&!rh_3.1" any string beginning with "rh_3", except 
"rh_3.1" 
 
pattern When patterns are used the following definitions apply:
"*" matches any character and any number of characters 
(between 0 and infinity). 
"?" matches any single character. 
"." is the character ".". It has no other meaning. 
"\" escape character, making the following character match literally; 
"\\" matches "\", "\*" matches "*", "\?" matches "?". 
"[...]" specifies an array or a range of allowed 
characters for one character at a specific position.
Character ranges may be specified using the a-z notation. The caret symbol (^) is not interpreted as a logical not; it is interpreted literally.

For more details please see fnmatch(5), glob(7).

A pattern on a shell command line should normally be quoted to avoid it being interpreted by the shell as a file match.

range The task range specifier has the form

n [-m [:s]][,n [-m [:s]],...] 
or
n [-m [:s]][ n [-m [:s]] ...]  
and thus consists of a comma- or blank-separated list of range specifiers n[-m[:s]]. The ranges are concatenated to the complete task id range. Each range may be a single number, a simple range of the form
n -m, or a range with a step size. 
wc_ar The wildcard advance reservation (AR) specification is a placeholder for AR ids and AR names including AR name patterns. An AR id always references one AR, while the name and pattern might reference multiple ARs.
wc_ar := ar_id | ar_name | pattern 
wc_ar_list The wildcard advance reservation (AR) list specification allows referencing multiple ARs with one command.
wc_ar_list := wc_ar[, wc_ar
, ...] 
wc_host A wildcard host specification (wc_host) is a wildcard expression which might match one or more hosts used in the cluster. The first character of that string never begins with an at-character ('@'), even if the expression begins with a wildcard character.

e.g.

 
* all hosts 
a* all host beginning with an 'a' 
wc_hostgroup A wildcard hostgroup specification (wc_hostgroup) is a wildcard expression which might match one or more hostgroups. The first character of that string is always an at-character ('@').

More information concerning hostgroups can be found in hostgroup(5)

e.g.

@* all hostgroups in the cluster 
@solaris the @solaris hostgroup 
wc_job The wildcard job specification is a placeholder for job ids, and job names including job name patterns. A job id always references one job, while the name and pattern might reference multiple jobs.
wc_job := job-id | job_name | pattern 
 
wc_job_range The wildcard job range specification allows referencing specific array tasks for one or multiple jobs. The job is referenced via wc_job and in addition gets a range specifier for the array tasks.
wc_job_range := wc_job [-t range] 
 
If present, the task_range restricts the effect of the qalter etc. operation to the array job task range specified as a suffix to the job id. (See the -t option to qsub(1) for further details on array jobs.)

wc_job_list The wildcard job list specification allows referencing multiple jobs with one command.

wc_job_list := wc_job[, wc_job, 
...] 
wc_job_range_list The wildcard job range list (wc_job_range_list) allows referencing multiple job ranges with one command. one of the following forms:
wc_job_range_list := wc_job_range[,wc_job_range...] 
wc_qdomain
wc_qdomain := wc_cqueue "@" wc_hostgroup 
A wildcard expression queue domain specification (wc_qdomain) starts with a wildcard expression cluster queue name (wc_cqueue) followed by an at-character '@' and a wildcard expression hostgroup specification (wc_hostgroup).

A wc_qdomain is used to address a group of queue instances. All queue instances residing on a host which is part of matching hostgroups will be addressed. Please note, that wc_hostgroup always begins with an at- character.

e.g.

*@@* all queue instances whose underlying 
host is part of at least one hostgroup 
a*@@e* all queue instances beginning with a whose underlying 
host is part of at least one hostgroup beginning with e 
*@@solaris all queue instances on hosts in 
the @solaris hostgroup 
wc_cqueue A wildcard expression cluster queue specification (wc_cqueue) is a wildcard expression which might match one or more cluster queues used in the cluster. That string never contains an at-character ('@'), even if the expression begins with a wildcard character. e.g.
* all cluster queues 
a* all cluster queues beginning with an 'a' 
a*&!adam all cluster queues beginning with an 'a', but not adam 
wc_qinstance
wc_qinstance := wc_cqueue "@" wc_host 
A wildcard expression queue instance specification (wc_qinstance) starts with a wildcard expression cluster queue name (wc_cqueue) followed by an at-character '@' and a wildcard expression hostname (wc_host).

wc_qinstance expressions are used to address a group of queue instances whose underlying hostname matches the given expression. Please note that the first character of wc_host never matches the at-character '@'.

e.g.

*@* all queue instances in the cluster 
*@b* all queue instances whose 
hostname begins with a 'b' 
*@b*|c* all queue instances whose 
hostname begins with a 'b' or 'c' 
wc_queue
 
wc_queue := wc_cqueue | wc_qdomain |
wc_qinstance 
A wildcard queue expression (wc_queue) might either be a wildcard expression, cluster queue specification (wc_cqueue), a wildcard expression queue domain specification (wc_qdomain), or a wildcard expression queue instance specification (wc_qinstance).

e.g.

big_*1 cluster queues which begin with 
"big_" and end with "1" 
big_*&!*1 cluster queues which begin with 
"big_", but do not end with "1" 
*@fangorn all qinstances residing on host 
fangorn 
wc_queue_list wc_queue_list := wc_queue ["," wc_queue "," ...]

Comma-separated list of wc_queue elements.

e.g.

big, medium_*@@sol*, *@fangorn.sun.com 
wc_user A wildcard user name pattern is either a wildcard user name specification or a full user name.
wc_user := user_name | pattern 
wc_user_list A list of user names.
 
wc_user_list := wc_user[,wc_user,...] 
wc_project A wildcard project name pattern is either a wildcard project name specification or a full project name.
 
wc_project := project | pattern 
wc_pe_name A wildcard parallel environment name pattern is either a wildcard PE name specification or a full PE name.
 
wc_pe_name := pe_name | pattern 
parallel_env The parallel_env specification has the format
 
wc_pe_name n[-[m]]|[-]m,... 
specifying a parallel programming environment (PE) to select for a submitted job or an AR. The range descriptor following the wildcard PE name specifies the number of slots to allocate, which is usually equivalent to the total number of parallel processes to be run (for simple distributed memory jobs) or the number of threads (for shared memory or mixed distributed/threaded jobs), as implied by the PE definition. Grid Engine will allocate the appropriate resources, as available. sge_pe(5) contains information about the definition of PEs.

You can specify a PE name which uses wildcards. Thus the request "mpi*" will match any parallel environment with a name starting with the string "mpi". In the case of multiple parallel environments whose names match a name string, when it is required to select one the one with the most available slots is chosen.

The range specification is a list of range expressions of the form "n-m", where n and m are positive, non-zero integers. The form "n" is equivalent to "n-n". The form "-m" is equivalent to "1-m". The form "n-" is equivalent to "n-infinity". The range specification is processed as follows: The largest number of queues requested is checked first. If enough queues meeting the specified attribute list are available, all are reserved. If not, the next smaller number of queues is checked, and so forth.

Recommended Links

Google matched content

Softpanorama Recommended

Top articles

Sites

Top articles

Sites



Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotesSomerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose BierceBernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOSProgramming Languages History : PL/1 : Simula 67 : C : History of GCC developmentScripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-MonthHow to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D


Copyright © 1996-2021 by Softpanorama Society. www.softpanorama.org was initially created as a service to the (now defunct) UN Sustainable Development Networking Programme (SDNP) without any remuneration. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License. Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.

This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...

You can use PayPal to to buy a cup of coffee for authors of this site

Disclaimer:

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: March, 12, 2019