|
|
Home | Switchboard | Unix Administration | Red Hat | TCP/IP Networks | Neoliberalism | Toxic Managers |
| (slightly skeptical) Educational society promoting "Back to basics" movement against IT overcomplexity and bastardization of classic Unix | |||||||
|
|
|
|
According to Wikipedia (Oracle Grid Engine) in 2000, Sun acquired Gridware, Inc. a private software development company with offices in San Jose, Calif., and Regensburg, Germany. Later that year, Sun offered a free version of Gridware for Solaris and Linux, and renamed the product Sun Grid Engine.
In 2001, Sun made the source code available, and adopted the open source development model. Ports for Mac OS X and *BSD were contributed by the non-Sun open source developers.
In 2010, after the purchase of Sun by Oracle, the Grid Engine 6.2 update 6 source code was not included with the binaries, and changes were not put back to the project's source repository. In response to this, the Grid Engine community started the Open Grid Scheduler project to continue to develop and maintain a free implementation of Grid Engine.
On January 18, 2011, it was announced that Univa had recruited several principal engineers from the former Sun Grid Engine team. Initially Univa produces both commercial and open source version (so called Core version) but very soon open source version disappeared and only commercial was offered. This is the only version of SGE which includes commercial support. Quality of version is not impressive.
On Oct 22, 2013 Univa has announced that it had acquired the intellectual property and trademarks pertaining to the Grid Engine technology and that Univa will take over support for Oracle Grid Engine customers. Those who have permanent license (the majority of Oracle customers) can use Oracle Grid Engine without support indefinitely. it is actually pretty well debugged although GUI tool called mon does not work on newer version of RHEL such as 5.x and 6.x.
Right now SGE exists in several versions.
The main difference with the versions of SGE floating around is not functionality, which is adequate even in version 6.2u5, but reliability. Lack of manpower hurt attempts to develop viable open source derivatives of version 6.2u5. Son of grid engine (which is the only actively maintained open source project for SGE) is essentially a one man project which is not enough man power for such a complex software. Unfortunately despite the fact that many labs use open source version no strong team of researchers supporting open source version development ever emerged. And even existing efforts splintered the codebase (Open Grid Scheduler vs ‘Son of Grid Engine’)
Open Grid Scheduler/Grid Engine is a free and open-source batch-queuing system for distributed resource management. OGS/GE is based on Sun Grid Engine, and maintained by the same group of external (i.e. non-Sun) developers who started contributing code since 2001.
In Dec 2010, Oracle officially passed on the torch for maintaining the Grid Engine open source code base to the Open Grid Scheduler project.
They planned big, but somehow their expectations did not materialize as probably they were unable to secure a sizable support revenue stream (The True Story of the Grid Engine Dream The memories of a Product Manager)
Sweden's Gridcore Computing has joined Open Grid Scheduler, They pre-announced that an update version of Grid Engine 2011.11 will be released at the Gompute User Meeting on May 12 2012 in Gothenburg, Sweden. They added Dev Subramanian, a top notch Grid Engine architect to the open source Grid Engine resources.
This version is used in major universities and companies and new installations still happen in 2014.
[gridengine users] Multiple slots per task in array job
Christopher Heiny cheiny at synaptics.com
Tue Nov 11 05:37:10 UTC 2014
- Previous message: [gridengine users] More on testing of Son of Grid Engine 8.1.8l
- Next message: [gridengine users] Multiple slots per task in array job
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Hi all,
We're running OGS 2011.11p1, with a 1:1 slot:core mapping. We mostly
run submit single threaded array jobs, one slot per task. We now want
to multi-thread some of the programs in order to speed up processing,
but need a way to allocate <n> slots to an <n> threaded task.
The first idea was to use a parallel environment to manage this, but
since the jobs are all with script (rather than a binary), we got bitten
by this bug:http://gridengine.org/pipermail/dev/2011-December/000081.html
Unfortunately, patching/updating the GE software is not an option at the
moment. We've got to wait a few months for that, and in the meantime we
need a workaround for slot allocation.Currently, I'm thinking of using a consumable equal to the number of
slots/cores on a machine to control allocation. I *think* this will
work OK as an interim solution. What I was wondering was: has anyone
encountered a similar situation? Did you use this trick to work around
it? Or is there a better workaround?
Thanks very much!
Chris
Son of Grid Engine Well maintained open source distribution. This is so far the most debugged open source distribution that might be especially attractive for those who what have experience with building software, but can be used by everybody on RHEL for which precompiled binaries exist.
Initially Univa SGE team planed to follow Sun path and produced both "core" open source version and "enhanced" professional paid version. And they started along this path produced open source version of Univa grid engine posting Univa open source version 8.0.0. But soon. with the change of CEO who came from IBM, they changed their mind and switched to fully commercial software development path. Open source version soon disappeared from the Web
Dave Love decided to save it and started development of his own SGE distribution which he aply named Son of Grid Engine. On July 27, 2011 he posted on the Grid Engine mailing list (on 27th of ) that he compiled a set of the Univa 8.0.0-tagged github code, but without ARCo. It still can be found at: http://arc.liv.ac.uk/downloads/SGE/releases/8.0.0a/. It doesn't contain the GUI installer and has no support for Hadoop and the Java interface (libjgdi) in all versions. Architectures are Linux as well as Intel based Solaris.
Installation is pretty raw, but I tried to compensate for that by creating several pages which together document installation process of RHEL 6.5 or 6.6 pretty well:
Hopefully they might serve as useful add-on to the old Sun SGE6.2u5 installation documentation.
Oracle grid engine.
Abandonware. Oracle-rebranded Sun version
6.2 Update 7 -- the last known
version produced by Sun. This is probably the most reliable abandonware version that this floating
around the WEb. For enterprises it requires license from Oracle to use so it is available only to
former Oracle customers (who typically has lifetime license). Fully corresponds to Sun SGE
6.2u7 with logos changed. Binary are dated 2010. Open source version of
Sun Grid Engine 6.2u5 with patches included in
some Linux distributions, such as Debian are the only alterative as for stability. GUI installer
works, but qmon does not work on RHEL 5/6 in this version. Other then that this version
is remarkably stable and well debugged.
Univa Grid Engine 8.x is API and configuration partially compatible with existing Sun Grid Engine
version 6.2u5. Format of queries is different so importing queries requires editing.
Support is the most valuable part of their offering. They also provide training (intermediate and
advanced administration courses). The level support is
typical for enterprise software offerings (nothing to boast about) and comparable with, say, HP
OpenView support level. In other words not very impressive.
The key problem with Univa engine is that they charge per core for the software, which I consider
to be unfair pricing model. This is the same problem that I have with IBM software.
As of November 2014 Son of Grid Engine is the only "datacenter class" open source implementation that is still under active development. Dave Love, the maintainer of this version also created and an very useful web (https://arc.liv.ac.uk/trac/SGE) site with "remnants" of mercilessly discarded by Oracle Sun documents, open source software and other materials. It is such a pity that Oracle people proved to be such a clueless vandals.
Technical difficulties and the lack of funding were not the only problems facing Dave Love. Univa considered him to be a threat to their revenue stream and tried to suppress Dave Love effort by sending a pretty threatening latter. He withstood this attack.
Note that, contrary to multiple claims, there is no evidence that any Sun (or other) material here is being distributed without an appropriate licence allowing it. See the rebuttal. Reports of licensing bugs are very welcome, of course.
As of version 8.1.8 it is the most well debugged open source distribution. It is also very attractive for those who what have experience with building software. Installation is pretty raw, but I tried to compensate for that creating several pages which together document installtion provess of RHEL 6.5 or 6.6 pretty well:
With the current size of codebase he is engaged in a very difficult, exhausting battle. We all need to support his efforts. Son of Grid engine is essentially a one man project which is not enough man power for such a complex software. Unfortunately despite the fact that many labs use open source version no strong team of researchers supporting open source version development ever emerged. And even existing efforts splintered the codebase (see the story of Open Grid Scheduler vs ‘Son of Grid Engine’).
In any case we can state that lack of manpower and lack of support revenue from the open source version of SGE hurts attempts to develop viable derivatives of classic version 6.2u5 and Dave Love was and still is the only guy who successfully swims against the stream.
Like any software development, the development of Grid engine is essentially a battle of ideas. And this is one of those cases when open source software managed to withstand and survive attempts of commercial developers to monopolize the field for themselves.
We own much gratitude to the Sun brass which opened the codebase and release pretty advanced version (version 6.2u5) as an open source. It looks like if opened source version released is sufficiently advanced and has certain (critical) number of customers it has better chances to survive. But which project survives and which dies dependents of particular circumstances.
We also owve much gratitude to Dave Love who managed to launch and sustain his open source project -- Son of Grid engine, which now is of version 8.1.8. I wonder if Debian maintainers can serve as a quality assurance team for this product. This would be a valuable change.
See SGE implementations for more details about SGE development and the issues of selection of the right version of SGE. There is an active SGE users list at gridengine.org site.
Among similar open source products the most prominent are Maui Scheduler and OpenPBS (commercial version is called PBS Pro).
See history for more information. There is an active SGE users list at gridengine.org site.
Society
Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers : Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism : The Iron Law of Oligarchy : Libertarian Philosophy
Quotes
War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda : SE quotes : Language Design and Programming Quotes : Random IT-related quotes : Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce : Bernard Shaw : Mark Twain Quotes
Bulletin:
Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 : Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law
History:
Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds : Larry Wall : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS : Programming Languages History : PL/1 : Simula 67 : C : History of GCC development : Scripting Languages : Perl history : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history
Classic books:
The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month : How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite
Most popular humor pages:
Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor
The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D
Copyright © 1996-2021 by Softpanorama Society. www.softpanorama.org was initially created as a service to the (now defunct) UN Sustainable Development Networking Programme (SDNP) without any remuneration. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License. Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.
FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.
This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...
|
|
You can use PayPal to to buy a cup of coffee for authors of this site |
Disclaimer:
The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.
Last modified: March, 12, 2019
