Maui Scheduler
Maui Cluster Scheduler, the precursor to
Moab HPC
Suite, is an open source job scheduler for clusters and supercomputers. It is an optimized,
configurable tool capable of supporting an array of scheduling policies, dynamic priorities,
extensive reservations, and fairshare capabilities. It is currently in use at hundreds of
government, academic, and commercial sites throughout the world. All of the capabilities found in
Maui are also found in Moab, while Moab has added features including basic trigger support,
graphical administration tools, and a Web-based user portal among many others. To learn more about
the features available on Maui and, see the
Maui and Moab comparison brief for all the additional capabilities and value Moab offers.
Maui Scheduler is an batch system for cluster systems. It allows site administrators extensive control over which jobs are considered
eligible for for scheduling, how the jobs are prioritized, and where these jobs are run. Maui supports advance reservations, QoS levels,
backfill, and allocation management.
Its scheduling scheme is based on advanced wall-time reservations with backfill. The main difference from other common batch queue
schedulers (e.g. NQS, DQS) is that Maui allows jobs to overtake a job with higher priority only if it does not delay the start of the
prioritized job (i.e. backfill).
Opportunistic Scheduling versus Advance Reservation
In theory, unless there is a way of reserving resources in advance, any job
that need more than a single piece of allocatable resource, runs the risk of starvation[1,2]. In practice, due to limited workload
this is a danger only for jobs that require extensive resources. It is often intervened by periodically putting small, less demanding
jobs on hold until the large, starving job have started. This is a rather intrusive
operation since it also affects jobs that is not involved in the starvation.
An advance reservation scheme, as in the Maui scheduler, makes it possible to allocate resources in the future. (Compare scheduling
a meeting with several participants. It is almost impossible unless there is a calendar available.)
The starvation can also be resolved by using a preemptive job scheduler. Unfortunately not all computer systems can handle this.
Moreover, if jobs can be preempted it is more difficult to predict when the job
will finish.
Queues versus Quality of Service
When an idle job becomes eligible to run, it is assigned a priority. This priority is used to sort the jobs before the scheduler
selects a job to start.
Many batch systems use queues to divide and classify the workload. Each queue is then assigned a priority and sometimes each job
is assigned a second priority to sort themselves within the queue. This classifying scheme is often too coarse. To take into account
all parameters that set a batch job policy, you may end up with more queues than jobs.
In Maui, "queues" have lost their importance in classification and priority calculations. Instead a Quality-of-Service (QoS) attribute
can be used to classify the jobs. However, QoS is not a hierarchical scheme. It is merely a method of setting the parameters of a job
when it enters the scheduler. All jobs eligible to run remain in one common idle-queue and their priorities are compared with all others.
Job State
Jobs in Maui can be in one of three major states:
- Running
- A jobs that have been alloted its required resources and have started its computation is considered running until it finish.
- Queued (idle)
- Jobs that are eligible to run. The priority is calculated here and the jobs are sorted according to calculated priority. Advance
reservations are made starting with the job up front.
- Non-queued
- Jobs that, for some reason, are not allowed to start. Jobs in this state does not gain any queue-time priority.
There is a limit on the number of jobs a group/user can have in the Queued state. This prohibit users from acquiring longer queue-time
than deserved by submitting large number of jobs.
Job Priority
Maui present numerous factors in the expression used to calculate the job priority to achieve a site's goals of fairness and utilization.
Each factor is weighted according to its importance and the sum is used as the total priority of the job. The most important factors
are described below together with the importance they have in the current configuration of Maui on Ingvar
Resource
The resource factor consist of several terms that describes the required resource to run the
job; number of processors, amount of memory, size of empty disk space, and swap
size. Depending on what type of jobs is favored, jobs can be pushed the front of the queue. Experience shows that favoring large jobs
often improves system utilization.
Ingvar: Fairly low rating. A high utilization is desired but the fairness between users should not be
affected.
Queue Time
This factor is based on the time the job has been eligible to run. This factor
often has a very low weight in the priority calculation. Instead, more important is the expansion factor
Ingvar: Low rating. A fall-back.
Expansion
The expansion factor or XFactor is calculated using the equation:
XFactor = (Queue_Time + Job_Time_Limit) / Job_Time_Limit
This relates the job time limit the user request to the total queueing and expected
run time. A job with low time limit will increase its priority more quickly than
a long job, pushing it to the front of the queue.
Ingvar: The most important factor after QoS. It verbalizes the general
job scheduling policy.
Target
If the expansion factor is not enough to meet the scheduling goals, there is a Target factor that is increased exponentially as the
actual queue time approach the target queue time.
Ingvar: Not used ...yet.
Fair Share
The fair share value is based on historical usage. It is divided into the user, group, and account associated with the
job. Fair Share is a provocative factor. Although the intention is good, the
effect of this factor is not easy to understand and rate to achieve fairness[3].
Ingvar: Excluded from any priority calculation.
Quality of Service
The QoS factor is a fixed number used to offset jobs with high quality-of-service.
Ingvar: Three different QoS exist; Normal, Bonus, and Disabled. Normal has a ten times higher QoS-factor than Bonus, always pushing
bonus jobs to the back of the queue. Another feature in Maui prohibit Disabled jobs to make any reservations.
Reservation
There are two types of reservations in Maui:
- Job Reservation
Every scheduling cycle, after the job priority have been calculated, Maui examines the jobs in the queued state and schedules advance
reservations.
- User Reservation
- If a number of nodes is needed for a certain purpose on a certain time, it is possible, for the Maui administrator, to make
an user reservation. This reservation is permanent and kept between scheduling cycles.
Also, there is standing reservations. This is user reservations which are scheduled automatically and repeatedly.
Links
| User Commands |
The following commands are available for users:
- showq
- Shows information about 1) running, 2) idle, and 3) non-queued jobs. Non-queued jobs are jobs that have been submitted
to the batch queue system but are not considered eligible to run. These jobs does not gain any queued time priority.
- showbf
- Shows what resources are available for immediate use.
- showstart <jobid>
- Shows the earliest time the queued job <jobid> is eligible to
start.
- checkjob <jobid>
- Shows various details about a submitted job. Users are only permitted to see details of jobs they own.
|
Maui is an advanced job scheduler for use on clusters and supercomputers. It is a highly optimized and configurable tool capable
of supporting a large array of scheduling policies, dynamic priorities, extensive reservations, and fairshare. It is currently in
use at hundreds of leading government, academic, and commercial sites throughout the world. It improves the manageability and efficiency
of machines ranging from clusters of a few processors to multi-teraflop supercomputers.
Maui is a community project* and may be downloaded, modified, and distributed.
It has been made possible by the support of Cluster Resources, Inc
and the contributions of many individuals and sites including the U.S. Department of Energy,
PNNL, the Center for High Performance Computing at the University of Utah (CHPC),
Ohio Supercomputing Center (OSC), University of Southern California (USC), SDSC, MHPCC,
BYU, NCSA,
and many others.
Features:
Maui extends the capabilities of base resource management systems by adding the following features:
Maui interfaces with numerous resource management systems supporting any of the following scheduling API's
PBS Scheduling API - TORQUE,
OpenPBS and
PBSPro
Loadleveler Scheduling API - Loadleveler
(IBM)
SGE Scheduling API - Sun Grid Engine (Sun)*
BProc Scheduling API - BProc (Scyld)**
SSS XML Scheduling API*
LSF Scheduling API - LSF (Platform)
Wiki FlatText Scheduling API (Wiki)
*partial support or under development
**supported under Clubmask
Maui is currently supported on all known variants of Linux, AIX, OSF/Tru-64, Solaris, HP-UX, IRIX, FreeBSD, and other UNIX platforms.
The Maui scheduler is mature, fully documented, and supported. It continues to be agressively developed and possesses a very
active and growing user community. Its
legacy of pushing the scheduling envelope
continues as we promise to deliver the best possible scheduler supporting systems software will allow.
Softpanorama Recommended
The Maui Job Scheduler
Maui
Cluster Scheduler - Sourceforge page
Maui
Society
Groupthink :
Two Party System
as Polyarchy :
Corruption of Regulators :
Bureaucracies :
Understanding Micromanagers
and Control Freaks : Toxic Managers :
Harvard Mafia :
Diplomatic Communication
: Surviving a Bad Performance
Review : Insufficient Retirement Funds as
Immanent Problem of Neoliberal Regime : PseudoScience :
Who Rules America :
Neoliberalism
: The Iron
Law of Oligarchy :
Libertarian Philosophy
Quotes
War and Peace
: Skeptical
Finance : John
Kenneth Galbraith :Talleyrand :
Oscar Wilde :
Otto Von Bismarck :
Keynes :
George Carlin :
Skeptics :
Propaganda : SE
quotes : Language Design and Programming Quotes :
Random IT-related quotes :
Somerset Maugham :
Marcus Aurelius :
Kurt Vonnegut :
Eric Hoffer :
Winston Churchill :
Napoleon Bonaparte :
Ambrose Bierce :
Bernard Shaw :
Mark Twain Quotes
Bulletin:
Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient
markets hypothesis :
Political Skeptic Bulletin, 2013 :
Unemployment Bulletin, 2010 :
Vol 23, No.10
(October, 2011) An observation about corporate security departments :
Slightly Skeptical Euromaydan Chronicles, June 2014 :
Greenspan legacy bulletin, 2008 :
Vol 25, No.10 (October, 2013) Cryptolocker Trojan
(Win32/Crilock.A) :
Vol 25, No.08 (August, 2013) Cloud providers
as intelligence collection hubs :
Financial Humor Bulletin, 2010 :
Inequality Bulletin, 2009 :
Financial Humor Bulletin, 2008 :
Copyleft Problems
Bulletin, 2004 :
Financial Humor Bulletin, 2011 :
Energy Bulletin, 2010 :
Malware Protection Bulletin, 2010 : Vol 26,
No.1 (January, 2013) Object-Oriented Cult :
Political Skeptic Bulletin, 2011 :
Vol 23, No.11 (November, 2011) Softpanorama classification
of sysadmin horror stories : Vol 25, No.05
(May, 2013) Corporate bullshit as a communication method :
Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law
History:
Fifty glorious years (1950-2000):
the triumph of the US computer engineering :
Donald Knuth : TAoCP
and its Influence of Computer Science : Richard Stallman
: Linus Torvalds :
Larry Wall :
John K. Ousterhout :
CTSS : Multix OS Unix
History : Unix shell history :
VI editor :
History of pipes concept :
Solaris : MS DOS
: Programming Languages History :
PL/1 : Simula 67 :
C :
History of GCC development :
Scripting Languages :
Perl history :
OS History : Mail :
DNS : SSH
: CPU Instruction Sets :
SPARC systems 1987-2006 :
Norton Commander :
Norton Utilities :
Norton Ghost :
Frontpage history :
Malware Defense History :
GNU Screen :
OSS early history
Classic books:
The Peter
Principle : Parkinson
Law : 1984 :
The Mythical Man-Month :
How to Solve It by George Polya :
The Art of Computer Programming :
The Elements of Programming Style :
The Unix Hater’s Handbook :
The Jargon file :
The True Believer :
Programming Pearls :
The Good Soldier Svejk :
The Power Elite
Most popular humor pages:
Manifest of the Softpanorama IT Slacker Society :
Ten Commandments
of the IT Slackers Society : Computer Humor Collection
: BSD Logo Story :
The Cuckoo's Egg :
IT Slang : C++ Humor
: ARE YOU A BBS ADDICT? :
The Perl Purity Test :
Object oriented programmers of all nations
: Financial Humor :
Financial Humor Bulletin,
2008 : Financial
Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related
Humor : Programming Language Humor :
Goldman Sachs related humor :
Greenspan humor : C Humor :
Scripting Humor :
Real Programmers Humor :
Web Humor : GPL-related Humor
: OFM Humor :
Politically Incorrect Humor :
IDS Humor :
"Linux Sucks" Humor : Russian
Musical Humor : Best Russian Programmer
Humor : Microsoft plans to buy Catholic Church
: Richard Stallman Related Humor :
Admin Humor : Perl-related
Humor : Linus Torvalds Related
humor : PseudoScience Related Humor :
Networking Humor :
Shell Humor :
Financial Humor Bulletin,
2011 : Financial
Humor Bulletin, 2012 :
Financial Humor Bulletin,
2013 : Java Humor : Software
Engineering Humor : Sun Solaris Related Humor :
Education Humor : IBM
Humor : Assembler-related Humor :
VIM Humor : Computer
Viruses Humor : Bright tomorrow is rescheduled
to a day after tomorrow : Classic Computer
Humor
The Last but not Least Technology is dominated by
two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt.
Ph.D
Copyright © 1996-2021 by Softpanorama Society. www.softpanorama.org
was initially created as a service to the (now defunct) UN Sustainable Development Networking Programme (SDNP)
without any remuneration. This document is an industrial compilation designed and created exclusively
for educational use and is distributed under the Softpanorama Content License.
Original materials copyright belong
to respective owners. Quotes are made for educational purposes only
in compliance with the fair use doctrine.
FAIR USE NOTICE This site contains
copyrighted material the use of which has not always been specifically
authorized by the copyright owner. We are making such material available
to advance understanding of computer science, IT technology, economic, scientific, and social
issues. We believe this constitutes a 'fair use' of any such
copyrighted material as provided by section 107 of the US Copyright Law according to which
such material can be distributed without profit exclusively for research and educational purposes.
This is a Spartan WHYFF (We Help You For Free)
site written by people for whom English is not a native language. Grammar and spelling errors should
be expected. The site contain some broken links as it develops like a living tree...
Disclaimer:
The statements, views and opinions presented on this web page are those of the author (or
referenced source) and are
not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness
of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be
tracked by Google please disable Javascript for this site. This site is perfectly usable without
Javascript.
Last modified:
March, 12, 2019
