May the source be with you, but remember the KISS principle ;-)
Home Switchboard Unix Administration Red Hat TCP/IP Networks Neoliberalism Toxic Managers
(slightly skeptical) Educational society promoting "Back to basics" movement against IT overcomplexity and  bastardization of classic Unix

qhold / qrls - hold/release batch jobs

News SGE Commands Recommended Links SGE cheat sheet Reference SGE Submit Scripts Submitting binaries in SGE
qstat qhold qping qacct qmod qalter qrsh
Creating and modifying SGE Queues Getting information about hosts Monitoring and Controlling Jobs Monitoring Queues Starting and Killing Daemons qhost qdel
SGE hostgroups Creating and modifying SGE Queues SGE Execution Host Installation SGE Parallel Environment Tips Humor Etc

The qhold command is essentially a special case of the qalter command.

Command qhold puts a job, which hasn't been started yet on hold so that it will not be  scheduled for execution by the SGE till the hold is removed. Requires the job-id as argument.  qrls command releases the job

A hold is placed on a batch job by a request to the batch server that manages the batch job. A batch job that has one or more holds is not eligible for execution. The qhold  utility is a user-accessible client of batch services that requests one or more types of hold to be placed on one or more batch jobs.

The qhold  utility allows users to place a hold on one or more jobs. A hold makes a batch job ineligible for execution.

The qhold  utility has options that allow the user to specify the type of hold. Should the user wish to place a hold on a set of jobs that meet a selection criteria, such a list of jobs can be acquired using the qselect  utility.

The option -h  allows the user to specify the type of hold that is to be placed on the job. This option allows for USER, SYSTEM, OPERATOR, and implementation-defined hold types. The USER and OPERATOR holds are distinct. The batch server that manages the batch job will verify that the user is authorized to set the specified hold for the batch job.

Mail is not required on hold because the administrator has the tools and libraries to build this option if he or she wishes. Historically, the qhold  utility has been a part of some existing batch systems

The qhold  utility shall place holds on batch jobs in the order in which their batch job_identifiers are presented to the utility. If the qhold  utility fails to process any batch job_identifier  successfully, the utility shall proceed to process the remaining batch job_identifiers, if any.

The qhold  utility shall place holds on each batch job by sending a Hold Job Request  to the batch server that manages the batch job.

The qhold  utility shall not exit until holds have been placed on the batch job corresponding to each successfully processed batch job_identifier.

qhold [-h hold_list] job_identifier  ...


The qhold  utility shall conform to the Base Definitions volume of IEEE Std 1003.1-2001, Section 12.2, Utility Syntax Guidelines.

The following option shall be supported by the implementation:

For each unique character in the hold_list  option-argument, the qhold  utility shall add a value to the Hold_Types  attribute of the batch job as follows, each representing a different hold type:

If any of these characters are duplicated in the hold_list  option-argument, the duplicates shall be ignored.

An existing Hold_Types  attribute can be cleared by the following hold type:


The qhold  utility shall consider it an error if any hold type other than 'n' is combined with hold type 'n' .

Strictly conforming applications shall not repeat any of the characters 'u', 's', 'o', or 'n' within the hold_list  option-argument. The qhold  utility shall permit the repetition of characters, but shall not assign additional meaning to the repeated characters.

An implementation may define other hold types. The conformance document for an implementation shall describe any additional hold types, how they are specified, their internal behavior, and how they affect the behavior of the utility.

If the -h option is not presented to the qhold  utility, the implementation shall set the Hold_Types  attribute to USER.


The qhold  utility shall accept one or more operands that conform to the syntax for a batch job_identifier  (see Batch Job Identifier  ).

Stdin is not used.

There are no input files

Environment Variables

The following environment variables shall affect the execution of qhold:

The standard error shall be used only for diagnostic messages.

Exit Status

The following exit values shall be returned:

Consequences of Errors

In addition to the default behavior, the qhold  utility shall not be required to write a diagnostic message to standard error when the error reply received from a batch server indicates that the batch job_identifier  does not exist on the server. Whether or not the qhold  utility waits to output the diagnostic message while attempting to locate the job on other servers is implementation-defined.


Portions of this text are reprinted and reproduced in electronic form from IEEE Std 1003.1, 2003 Edition, Standard for Information Technology -- Portable Operating System Interface (POSIX), The Open Group Base Specifications Issue 6, Copyright © 2001-2003 by the Institute of Electrical and Electronics Engineers, Inc and The Open Group. In the event of any discrepancy between this version and the original IEEE and The Open Group Standard, the original IEEE and The Open Group Standard is the referee document. The original Standard can be obtained online at

Top updates

Bulletin Latest Past week Past month
Google Search


Old News ;-)

Cornell Virtual Workshop


Once a job has been submitted, there are times when you may want to suspend or stop a job outright. Perhaps the job depends on data that hasn't finished uploading yet, whatever the reason, SGE provides you with the qhold tool for that.

qhold 28780 

Specifically, when you issue a hold on a job you are setting a user hold on the job. There are also system and operator holds, but you'll only be setting user holds, which you implicitely set. To explicitely set a user hold, specify:

login3% qhold -h u 28780
job-ID  prior   name       user         state submit/start at     queue                          slots ja-task-ID
  28780 0.00000 testJob    username     hqw  02/25/2008 07:56:57

We can see that the state has changed from the queued state (qw) to a held state (hqw). At this point, we have two options, either we are ready for the job to continue, or we want delete the job. For these two sitations, we use the qrls (release) and qdel commands.

login3% qrls -h u 28780
modified hold of job 28780
login3% qdel 28785
userName has deleted job 28785

There are situations where qdel won't work, and won't delete the job. For instance, if a node lit on fire while running your job, you won't be able to delete that job. qdel will send a signal to the daemon (ge_execd) to delete the job, but won't get a response. In this case, the only way to delete the job will be to add the -f (force) option to qdel. This tells the scheduler to not wait for confirmation from the node, you should only use this option when you have to!

Show only job that are not on hold in qstat

I'm running some jobs on an SGE cluster. Is there a way to make qstat show me only jobs that are not on hold?

qstat -s p shows pending jobs, which is all those with state "qw" and "hqw".

qstat -s h shows hold jobs, which is all those with state "hqw".

I want to be able to see all jobs with state "qw" only and NOT state "hqw". The man pages seem to suggest it isn't possible, but I want to be sure I didn't miss something. It would be REALLY useful and it's really frustrating me that I can't make it work.

Other cluster users have a few thousand jobs on hold ("hqw") and only a handful actually in the queue waiting to run ("qw"). I want to see quickly and easily the stuff that is not on hold so I can see where my jobs are in the queue. It's a pain to have to show everything and then scroll back up to find the relevant part of the output.


share|improve this question

asked Feb 14 at 23:08



So I figured out a way to show what I want by piping the output of qstat into grep:
qstat -u "*" | grep " qw"

(Note that I need to search for " qw" not just "qw" or it will return the "hqw" states as well.)

But I'd still love to know if it's possible using qstat options only.

Recommended Links

Top Visited

Bulletin Latest Past week Past month
Google Search



Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy


War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotesSomerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose BierceBernard Shaw : Mark Twain Quotes


Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law


Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOSProgramming Languages History : PL/1 : Simula 67 : C : History of GCC developmentScripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-MonthHow to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Haterís Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

Copyright © 1996-2021 by Softpanorama Society. was initially created as a service to the (now defunct) UN Sustainable Development Networking Programme (SDNP) without any remuneration. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License. Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.

This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...

You can use PayPal to to buy a cup of coffee for authors of this site


The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: March 12, 2019