MCA Parameters

MCA parameters are a way to tweak Open MPI's behavior at run-time. For example, MCA parameters can specify:

• Which interconnect networks to use
• Which interconnect networks not to use
• The size different between eager sends and rendezvous protocol sends
• How many registered buffers to pre-pin (e.g., for GM or mVAPI)
• The size of the pre-pinned registered buffers
• ...etc.

It can be quite valuable for a system administrator to play with such values a bit and find an "optimal" setting for a particular operating environment. These values can then be set in a global text file that all users will, by default, inherit when then run Open MPI jobs.

For example, say that you have a cluster with 2 ethernet networks -- one for NFS and other system-level operations, and one for MPI jobs. The system administrator can tell Open MPI to not use the NFS TCP network at a system level, such that when users invoke mpirun or mpiexec to launch their jobs, they will automatically only be using the network meant for MPI jobs.

See  the run-time tuning FAQ category for information how to set global MCA parameters.

How do I know what MCA parameters are available?

The ompi_info command can list the parameters for a given component, all the parameters for a specific framework, or all parameters. Most parameters contain a description of the parameter; all will show the parameter's current value.

For example:

shell$ ompi_info --param all all

Shows all the MCA parameters for all components that ompi_info finds, whereas:

shell$ ompi_info --param btl all

Shows all the MCA parameters for all BTL components that ompi_info finds. Finally:

shell$ ompi_info --param btl tcp

Shows all the MCA parameters for the TCP BTL component.

9. How do I set the value of MCA parameters?

There are three main ways to set MCA parameters, each of which are searched in order.

1. Command line: The highest-precedence method is setting MCA parameters on the command line. For example:

   shell$ mpirun --mca mpi_show_handle_leaks 1 -np 4 a.out
   

   This sets the MCA parameter mpi_show_handle_leaks to the value of 1 before running a.out with four processes. In general, the format used on the command line is "--mca <param_name>< value>".

   Note that when senting multi-word values, you need to use quotes to ensure that the shell and Open MPI understand that they are a single value. For example:

   shell$ mpirun --mca param "value with multiple words" ...
   

2. Environment variable: Next, environment variables are searched. Any environment variable named OMPI_MCA_<param_name> will be used. For example, the following has the same effect as the previous example (for sh-flavored shells):

   shell$ OMPI_MCA_mpi_show_handle_leaks=1
   shell$ export OMPI_MCA_mpi_show_handle_leaks
   shell$ mpirun -np 4 a.out
   

   Or, for csh-flavored shells:

   shell% setenv OMPI_MCA_mpi_show_handle_leaks 1
   shell% mpirun -np 4 a.out
   

   Note that setting environment variables to values with multiple words requires quoting, such as:

   # sh-flavored shells
   shell$ OMPI_MCA_param="value with multiple words"
   
   # csh-flavored shells
   shell% setenv OMPI_MCA_param "value with multiple words"
   

    

3. Aggregate MCA parameter files: Simple text files can be used to set MCA parameter values for a specific application. See this FAQ entry (Open MPI version 1.3 and higher).

    

4. Files: Finally, simple text files can be used to set MCA parameter values. Parameters are set one per line (comments are permitted). For example: >

   # This is a comment
   # Set the same MCA parameter as in previous examples
   mpi_show_handle_leaks = 1
   

   Note that quotes are not necessary for setting multi-word values in MCA parameter files. Indeed, if you use quotes in the MCA parameter file, they will be used as part of the value itself. For example:

   # The following two values are different:
   param1 = value with multiple words
   param2 = "value with multiple words"
   

   By default, two files are searched (in order):

    

   1. $HOME/.openmpi/mca-params.conf: The user-supplied set of values takes the highest precedence.
   2. $prefix/etc/openmpi-mca-params.conf: The system-supplied set of values has a lower precedence.

   More specifically, the MCA parameter mca_param_files specifies a colon-delimited path of files to search for MCA parameters. Files to the left have lower precedence; files to the right are higher precedence.

   Keep in mind that, just like components, these parameter files are only relevant where they are "visible" (see this FAQ entry). Specifically, Open MPI does not read all the values from these files during startup and then send them to all nodes in the job -- the files are read on each node during each process' startup. This is intended behavior: it allows for per-node customization, which is especially relevant in heterogeneous environments.

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: November 14, 2014