HPOM Conditions

Note: HP renamed the product called now HP operations manager way too many times. Also it is very inconsistent with using abbreviations. Here we will assume that the term "HP Operations manager" and abbreviations HPOM, OMU, and OVO  mean the same thing :-)

Essentially this type of policy is a complex multilevel pattern matching rule with some subject area specific twists.

 Policy include one or more conditions that select appropriate messages from the message stream for processing. In other words condition is the term for rules against which each event is checked. Each condition consists of three major parts

• Message matching template. You use so called HPOM patterns in this part of the policy. Each incoming message will be tried to match against this template. If matching is successful other parts of the policy are processes.  Within pattern matching template you can extract some parts and populate with them so called user variables that can be used in other parts of the policy, for example in the message rewriting template and or actions.
• Message rewriting template. In this part of the condition you can change fields of the matched message to produce a news message, the one that will be displayed on operator console. New message is injected into message stream and can be processed by other policies. 
• Actions. In this part of condition you can execute arbitrary script or binary passing both system and user defined variables as parameters. You can also suppress displayed message on operator console (acknowledge the message) if the script of binary returned success as the result of their execution.

Edition conditions

After you create a condition you can edit it. The top bar for each conditions contains seven submenus:

• Condition
• Message
• Actions
• Custom Attributes
• Correlation
• Instructions
• Advanced

The firs and the most important submenu is "Condition". Here you specified predicate by which message will be matched.

Typically you do not need to be fancy and can match message based on application field of the message (option -a in opcmsg).

Note: After typing an appropriate field you need  to press Enter to move the input field down. Only when the text you entered "migrates down" and input field became clean you can be save it. Otherwise your input will be lost.

If match is found then some processing will occurs. If nor event is discarded or processed by default mechanism that can send it intact to the message browser.  Conditions can either "suppress" the event or "transform" the event into message. Actually there are three types of conditions:  

• Suppress Matched Condition. A suppress on match condition means that you drop the event and do not send any message to operator console.  If a message matches all attributes for a suppress condition, it is excluded from HPOM. The message is never processed by any other condition or policy. It is dropped.  Suppress conditions enable you to reduce the number of messages that HPOM must process and that are displayed in the Java GUI Message Browser. This way you can discard messages from installed HPOM smart plug-ins that you consider to be spam (and which often are spam ;-)
• Message on Matched Condition. A message on match condition transforms the event into a new message and optionally can invoke some scripts (actions). The transformed message is injected into message stream of the HPOM server. Such condition contain a pattern matching template against which each message is checked. If a message matches all attributes set in this template, this condition triggers and other two parts of the condition (message transformation part and action part) are executed. Message transformation template allow you to set message attributes, co change the message text based on system and user defined variables. If this message does not trigger additional policies it is sent to Operations management server, where they can be assigned to specific operators.
• Suppress Unmatched Condition. If a message does not match the attributes for a suppress unmatched condition, it is excluded from HPOM. The message is not processed further.  If a message matches all attributes for a suppress unmatched condition, it is processed further by the succeeding conditions in the conditions list. Suppressing unmatched conditions enables you to improve HPOM performance by filtering out (at the condition level) all messages that are irrelevant to the policy, thus reducing the number of messages processed by the policy condition list.
  • Note: When the FORWARDUNMATCHED keyword is used in a policy body, the set of unmatched messages that are forwarded to the server are restricted to only those that directly relate to the policy.

Conditions within a policy are  numbered and are evaluated sequentially. The first condition to match the event ends processing and none of  subsequent conditions is checked against this event. That means that when you build a set of conditions, more precise conditions should be in the top and more generic in the bottom.  Unless you set of conditions consumes all messages of the particular type the last condition should be  Suppress Unmatched Condition.

Conditions match the message against several  attributes. Among them are

• node name,
• application name,
• message key,
• a text or object pattern.

 HPOM compares each incoming message with each condition in the order they are listed in the policy body.

You can set up as many message, suppress, and suppress unmatched conditions as you need. I saw policies that analyze log files with hundreds of conditions. Such policies are difficult to maintain and are generally counterproductive, but it looks like to have tremendous amount of conditions is quite possible.  Not that it is recommended.

To set up conditions, follow these steps:

1. Define Match Conditions Define a matching pattern that will be used to match messages that will be processed by this condition (incoming events)
2. Test the pattern. Test to make sure that the pattern matching works as expected and picks up only the extected messages -- no missing  and no "foreign" (accidentally captured) messages.
3. Set Up Message Correlation  Set up message correlation options to automatically acknowledge messages with a specific message key. That can help to eliminate frequently repeated messages and prevent cluttering up the Java GUI Message Browser.
4. Configure Operator-initiated Actions Configure operator-initiated actions so that, every time a selected message is matched, HPOM allows a selected operator to run a script or program that you have configured.
5. Configure Automatic Actions Configure automatic actions so that, every time a message is matched, HPOM runs a script or program automatically.
6. Select the format and wording of output message. Configure output message.  This is the test that will be displayed to operators in Java GUI Message Browser.
7. Define Message Attributes Define the attributes of the message to be displayed in the Java GUI Message Browser. These attributes are not necessarily the same as those for the original text matched from the message source.
8. Define Custom Message Attributes Define your own message attributes of the message to be displayed in the Java GUI Message Browser to provide operators with more relevant information about the message. Implementing Message Policies
9.  Write Instructions Write instructions to accompany the message displayed in the Java GUI Message Browser.

If you do not define any filters for a message source, all messages from that source are brought into HPOM for processing, provided you have chosen to forward unmatched messages to the management server.

HPOM provides a  pattern-matching language that permits  parts of messages to be extracted, assigned to variables, and used as parameters to build new message text or to set other attributes. These parameters can also be used for automatic and operator-initiated action commands. For a full list of HPOM and SNMP variables, see HP Operations Manager/Policy variables

Pattern Matching

See HPOM Patterns

Transformation rules

After a message matches a message condition, you can assign certain settings to the message before it is displayed in a browser. Assigning Message Settings You can assign new values for the following settings:

• Severity level
• Node
• Application
• Message group
• Object
• Message text
• Message type
• Message key
• Service name

Custom message attributes allow you to add your own attributes to a message. This means that in addition to the default message attributes, you can extend HPOM messages with attributes of your choice, for example, the attribute “Customer” or the attribute “SLA” for service level agreements. Custom message attributes can only be set for message conditions and are only available for log file, HPOM interface, and threshold monitor policies.

The simplest way to specify the transformation is using Admin GUI.

You can also use that command opccmachg to assign attributes of your choice to a message. For more information, see the opccmachg(1m) manpage. When creating and assigning custom message attributes, you can specify attribute name and value, for example:

# opccmachg -user opc_op -id 
55d3604a-536f-71db-08c0-0a1108c90000 CUSTOMER=VIP SLA=none Device=Device1 Source=Node1 

A message matching the following condition would display with four additional columns in the Java GUI browser:

• Customer
• Device
• SLA
• Source

• Hard-coded text
• Variables returned by the HPOM pattern-matching mechanism F
• Predefined HPOM variables

NOTE: Custom message attributes are only displayed in the browser and message properties windows of the Java GUI.

If so configured, custom message attributes are passed to the Message Stream Interface (MSI) on the agent, the management server, or both. Custom message attributes are also passed to the trouble ticket system, the notification service, or both.

Adding Instructions to Your Message

You can add instructions to your message. Typically, these instructions describe an automatic action, provide details of how an operator should perform an operator-initiated action, or describe other manual steps for resolving a problem.

To add instructions to your message, use one of the following methods:

• Write Instructions Write instructions for the message condition. All messages matching the condition then have the instructions associated with them. The text can be viewed in the Message Properties window of the Java GUI Message Browser. The advantage of using simple, text-based instructions is that they are stored in the database. This method has the following advantages:
• Instructions are highly reliable.
• Instructions are resolved at a very high speed.
• Call an External Application to Send Instructions Use the instruction text interface to call an external application to present instructions to an operator. The advantage of this method is that parameters of many kinds can be passed to the interface. For an example of how to set up text-based interfaces, see the following file: /opt/OV/OpC/examples/progs/oii_readme This method is potentially very flexible:
• Variables Variables may be included in the text, which among other things enables the complete localization of the instructions.
• Message-specific Instructions can be message-specific, rather than just message-condition-specific. For the Java GUI, special HPOM variables are available to call an external application or to open a web browser on the client where the Java GUI is currently running. For details, see the HPOM Administrator’s Reference.

Responding to a Message

HPOM provides several options for responding to messages that match conditions. Operators use some of these options in Message Browser to respond to messages. Some of these responses are transparent to operators.

Responses You can choose from the following response types:

• Logging Messages on the Management Server Only If you choose this option, messages that match a message condition are logged on the management server and stored in the history database. They are not processed further, but operators can view them in the Java GUI History Message browser. If you log the messages on the management server only, the other actions are ignored.
• Defining an Automatic Action An automatic action starts immediately when the message is received. For each action, you must define the node on which it is performed, and the command to be executed (shell script, program, application start, or other response). Operators can stop currently running actions, and restart automatic actions, if necessary. You can also specify that an automatic action provides an annotation, and that the annotation be acknowledged automatically if successful. If you set up an automatic action with automatic acknowledgment, the operator might not see the message in the Java GUI Message Browser.
• Defining an Operator-Initiated Action Operators can start an operator-initiated action after reviewing the message in a Java GUI Message Browser. As is the case with automatic actions, operators can stop currently running actions, and restart them if necessary. You can define the node and command. You can also specify that an annotation and acknowledgment be provided automatically.

• Stopping the database before starting a backup.
• Informing users that, before print spooling, the subsystem will go into maintenance mode.
• Forwarding Messages

• Start and stop time of action
• Exit value of action
• Action information written to stdout and stderr

NEWS CONTENTS

• 200102 : by ( by , )

Old News

Subject: Unable to Duplicate Alerts(Logfile) in HPOM by Suresh Reddy

May 27, 2010 Dear Experts,

We have a requirement from server team to monitor a log pattern called "DEAD PATHS DETECTED BY POWER PATH" on all HP-UX nodes. I have created a log template with message matching condition as follows

<*>DEAD PATHS DETECTED BY POWER PATH<*>

Two systems are generating these logs frequently. We are receiving alerts on java console but instead of getting Duplicate a new alert is generating when the same pattern find in the server logs.
Please suggest how can I modify the log file template to make sure the template matching the above condition instead of generating a new alert, existing alert should be duplicated in Java console.

Note : I have tried by giving the suppress time interval in advanced options in template.

Thanks & Regards,
Suresh Reddy.

Sumit Kumar

May 27, 2010

Hi Suresh,

Open your Motif GUI and go to Actions-->Server-->Configure and check mark the Suppress and Count Duplicate Messages.

Suresh Reddy

May 28, 2010

Dear Sumit,

Thanks for your replay. Suppress and Count Duplicate Messages is already enabled. Please find the attachment for the same.

Thanks & Regards,
Suresh Redddy.

lauri

May 28, 2010 11:16:50 GMT

There is probably differences in full error line every time (like date/time etc). You HAVE TO create message key for those messages. In that case duplication is done based on message keys and not on full patterns.

Goran Koruga

May 28, 2010

Hello.

Indeed - your pattern is very generic, and comparision is not done using patterns but string comparision for standard message fields (severity, text, application ...) or messages keys created automatically from those (but note that in case field contents change, so will message keys).

Use message keys like already suggested.

Regards,
Goran

Suresh Reddy

Thanks a lot for your replies.

Please find the attached the file contains the alerts and the server logs. Please suggest how can I create a message key for a log file template.

My message group is OS and Application is HP OSSPI.

Shane Mann

May 31, 2010

You could use a message key like this:

<$MSG_NODE_NAME>:<$LOGFILE>:<DEAD PATHS DETECTED BY POWER PATH>

So any occurrence of that string from the same node, and same logfile will duplicate match.

Cheers,
Shane

Shane Mann

May 31, 2010

Sorry - to apply the message key - Edit the template, edit the condition which you've already added and put those values in the message key field.

Cheers,
Shane

Suresh Reddy

May 31, 2010 11:42:22 GMT

Perfect It is worked...

But I have one logfile template with 21 conditions in that this DEAD PATH is the one of the conditions.

Please let me know If I will give this as a messagecorelation :
<$MSG_NODE_NAME>:<$LOGFILE>
will it work without log file pattern
OR wtih the below condition
<$MSG_NODE_NAME> lauri

May 31, 2010

Could you please explain a bit closer, what you mean? Seems like your duplication is working already, what are you trying to correlate there?

Shane Mann

May 31, 2010

You need to make the message key 'unique' enough not to duplicate messages that are not related. If you just use: <$MSG_NODE_NAME> - then *any* message that is generated with just the node name as the message key will be treated as a duplicate.

By putting the node name and the logfile, you narrow it down to messages from the same source eg: same server, same logfile. Then a string to match an equivalent line in that logfile.

You can make it much broader, if you leave the node name out - eg: <$LOGFILE>:<DEAD PATHS DETECTED BY POWER PATH> It will duplicate all messages from that logfile, with that text, regardless of which server they came from. Does that make sense?

Probably the better way to do it is to setup your message output text so that it has no unique date/time in it. Then suppress duplicate output messages (if you don't care about the number of duplicates). You are matching using:
<*>DEAD PATHS DETECTED BY POWER PATH<*>

Put this in your output text field:
syslog: DEAD PATHS DETECTED BY POWERPATH

And you'll find the server duplicate matching will work, even without a message key.

Either way - for your 21 conditions, you need to set a 'unique' message key, or set the output text to be consistent for each condition (no date time stamps).

Cheers,
Shane

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: March 12, 2019