Softpanorama May the source be with you, but remember the KISS principle ;-)	Home	Switchboard	Unix Administration	Red Hat	TCP/IP Networks	Neoliberalism	Toxic Managers
	(slightly skeptical) Educational society promoting "Back to basics" movement against IT overcomplexity and bastardization of classic Unix

Deduplication

News	Event Correlation	Recommended Links	Message Keys	Automatic acknowledgement
Default Policy Groups	Policies	Correlation Composer	ECS Designer	Humor	Etc

Note: HP renamed the product called now HP operations manager way too many times. Also it is very inconsistent with using abbreviations. Here we will assume that the term "HP Operations manager" and abbreviations HPOM, OMU, and OVO mean the same thing :-)

There are many configuration tasks for the management server. After the initial installation, customize the server environment. A few of the most popular initial configuration tasks are described in this section.

The management server produces messages immediately after installing the agent software, policies, actions, monitors, and commands. The agent software running on the server generates the messages. Many of the messages are duplicates; this happens if the HPOM agent detects the same error condition multiple times. If you disable the policies, you will not see the messages from that node in the message browser. Initially, you just want to suppress the duplicate messages.

After a new installation, you may find it necessary to reduce the number of duplicate messages that appear in the message browser. You can reduce the amount of redundant information appearing in the message browser by changing the server configuration with the duplicate message suppression feature.

Messages for duplicate suppression are grouped using so called message key which can be concatenation of several message attributes and variables. Message key represent the signature (primary filter) based on event attributes. Further processing takes place when an event matches all attributes set in the message key .

Deduplication is performed set of messages with identical message keys only.

Example Server Duplicate Message Suppression

Select Actions→Server→Configure. Select duplicate message suppression

This configures the server to display and increment a counter for duplicate messages in the message browser. If there is information that should accompany a message in the form of a note, it is referred to as an annotation. You can either discard the duplicate messages or add them as annotations to the original message.

The runtime engine for HPOM event-correlation is available for the HP Operations management server and the HP Operations agent. See the

HPOM Installation Guide for the Management Server for a list of platforms on which the runtime engine currently runs.

For more information about the concepts behind event correlation, as well as the way event correlation works in HPOM, see the HPOM Concepts Guide.

Configuring Event Correlation for HPOM

The HPOM message-source policy allows you to specify which conditions generate a message and whether or not the generated message is copied or diverted to the message stream interface (MSI) from where it may be passed to and processed by the event correlation template. Follow the procedure below to configure the event correlation:

1. Enable output from the HPOM internal message stream to the MSI as required at either (or both) management server or managed node level as follows:

On the HP Operations management server, run the following command:
# opcsrvconfig -msi -enable
On the HP Operations agent, enable output to the MSI using the Administrator’s GUI (CVPL). See the CVPL user documentation available for download in the HP Operations Manager for UNIX directory at: http://support.openview.hp.com/selfsolve/manuals
Alternatively, you can use the opcnode_modify() API, though HPOM does not provide any command line tool to use with this API in the current version. For API related information, refer to the HPOM Developer’s Reference.

2. Set up the HPOM message-source policy so that the message conditions that you are interested in output messages as intended. For each condition statement (policy block starting with CONDITION), make sure that: You specify a message-type attribute in the CONDITION or SET block of the policy body; attribute can be any of the allowed attributes specified in the policy grammar. For information on policy grammar, see the HPOM Concepts Guide.

The specified message-type attribute matches the corresponding attribute referenced in the Input node that starts the flow in the event-correlation circuit which you want to process the message in question.

3. Enable the Copy/Divert to MSI option for each condition that you wish to output a message, using one of the following keywords in the SET section(s): MPI_SV_COPY_MSG Copy message to MSI and pass it on to HPOM server processes. MPI_SV_DIVERT_MSG Send message to MSI and remove it from HPOM processing chain on the management server. MPI_AGT_COPY_MSG On a managed node, copy

message to MSI and pass it on to HPOM server processes. MPI_AGT_DIVERT_MSG On a managed node, send message to MSI and remove it from HPOM processing chain on the management server.

4. Enable, if required, the logging options for each template, by using one or more of the following keywords in the policy body, before specifying conditions: LOGMATCHEDMSGCOND Logs messages matching message conditions (section starting with MSGCONDITIONS). LOGMATCHEDSUPPRESS Logs messages matching suppress conditions (section starting with SUPPRESSCONDITIONS). LOGUNMATCHED Logs unmatched messages.

Suppressing Duplicate Messages

In general terms, duplicate messages are messages that report the same event or a similar event. For example, each time a user switches to another user, HPOM generates a message. If the user always changes to the same user, you could consider this information superfluous, declare it an identical event, and have HPOM suppress further messages relating to it. In addition, HPOM lets you decide how often or for how long further messages are suppressed before a “new” duplicate message is sent. NOTE If an incoming, duplicate message has a different severity or message text than the existing duplicate messages, you can configure HPOM to display the new values instead of the previous data. For more information, see “Updating Severity and Message Text of Duplicate Messages” on page 375.

You can configure HPOM to suppress duplicate messages on the managed node or on the management server. Filtering out (or suppressing) messages on the managed node reduces network traffic and keeps the management server free for other tasks. Because suppressing messages on the management node is more beneficial to performance, HPOM offers a variety of suppression types and settings on the managed node, but only a global setting on the management server. Use the configuration variable OPC_MAX_DUPL_ANNO to limit the number of duplicate messages for which annotations are added.

Verifying Suppression Types

If the condition event matches the condition, HPOM verifies that one of the following suppression types has been selected:

Suppress Messages Matching Condition HPOM suppresses all duplicate messages matching the chosen condition. That is, HPOM checks whether a message generated by the same condition already exists. If a message already exists, and the second event occurs within a specified time interval or below a configured counter, the second message is suppressed.
Suppress Identical Input Events HPOM suppresses only those duplicate messages whose event attributes are identical to each other. That is, HPOM checks whether a message already exists that was generated by the same input events. A message’s input events are defined in the Condition section of the policy condition. Two or more messages are considered to be identical if the following event attributes are the same:
Severity
Node
Application
Message group
Object
Message text (= original message text) If an identical message already exists, and the second event occurs within a specified time interval or below a configured counter, the second message is suppressed.
Suppress Identical Output Messages HPOM suppresses only those duplicate messages whose message attributes are identical to each other. That is, HPOM checks whether a message already exists that has the same message attributes. The attributes of a message are defined in the Set Attributes section of the policy condition. Two or more messages are considered to be identical if their message keys are the same.

If either message does not have a message key, the following message attributes must be identical:

Severity
Node
Application
Message group
Object
Message text
Service name

If an identical message already exists, and the second event occurs within a specified time interval or below a configured counter, the second message is suppressed.

The log file of the syslog daemon, /var/adm/syslog/syslog.log on HP-UX, illustrates suppressing duplicate messages with the Suppress Identical Output Messages option. For example, consider the following lines from the syslog log file:

>Mar 14 14:39:01 server inetd[9900]: telnet/tcp:
Connection from node1 at Tue Mar 14 14:39:01 2009
Mar 14 12:46:02 server inetd[9005]: login/tcp: Connection
from node2 at Tue Mar 14 12:46:02 2009
The pattern-matching text might look like this:
"inetd\[<#>\] <@.service>: Connection from <@.from_node>"

The important characteristics of the inetd connection messages are the local node, the node from which the connection is made, and the inetd service. The syslog time stamp, the PID, and the connection time are not important.

A message key might look like this: inetd_connect_from:<$MSG_NODE_NAME>:<from_node>: <service>

This message key would suppress all messages from the same nodes that connect to the same node and connect for the same service. Duplicate message suppression processes only those messages that are generated by the same condition. If you want to suppress duplicate messages that are generated by different conditions, enable this functionality on the management server. For details, see “Suppressing Duplicate Messages on the Management Server” on page 374.

NOTE Duplicate message suppression on managed nodes is not available for threshold monitor policies. To find out how to suppress duplicate messages from threshold monitor policies, see “Suppressing Duplicate Messages on the Management Server” on page 374.

You can specify the type of duplicate message suppression, as well as the time interval and counter or threshold settings. There are three types of duplicate message suppression. Use the following keywords to achieve the desired results:

SUPP_DUPL_COND Suppresses all messages that match the same condition.
SUPP_DUPL_IDENT Suppresses all messages that have same original message text (input).
SUPP_DUPL_IDENT_OUTPUT_MSG Suppresses all messages that have the same output message text.

The value following these keywords represents the time interval within which messages are suppressed. If instead of a time value, a keyword COUNTER_THRESHOLD (followed by a number) is used, the messages are suppressed until the defined number of messages are received, after which one message is sent and the counter is reset. Using keyword RESET_COUNTER_INTERVAL will set a time interval after which the counter is reset regardless of the number of messages received. Keyword RESEND sets the time limit after which messages are being sent again. For more information, see the policy body grammar in Appendix A, “Policy Body Grammar,” on page 475.

Type of Suppression Settings

You can choose between the following suppression settings:

Time Interval Specify a time interval during which duplicate events are ignored, and a time period after which you want to start sending messages again. In the example shown in Figure 4-10 on page 372, the suppression time interval is set to 30 seconds, but the suppression time is limited to 60 seconds.
Counter Specify a threshold for a duplicate message counter. HPOM increments the counter until it equals or crosses the threshold. At that point, HPOM allows the transmission of the duplicate message. In the example shown in Figure 4-11 on page 373, the counter threshold is set to two. The counter is reset after 30 seconds.
Combination of Time Interval and Counter If you use the time interval and counter together, events are evaluated first by the timer. If an event passes the timer, it is then evaluated by the counter, which either suppresses it, or sends a message to the management server.

Suppression Based on Time

Figure 4-10 illustrates suppression based on time. For suppression of identical input events, use SUPP_DUPL_IDENT keyword instead. For suppression of identical output messages, use SUPP_DUPL_IDENT_OUTPUT_MSG keyword.

The first event (E1) matches a condition. A message is sent. The timer is started.
A second event (E2) occurs 25 seconds later. This event occurs less than 30 seconds after the first event. So it is suppressed.
A third matching event (E3) occurs less than 30 seconds after the second event. So it is also suppressed.
The next matching event (E4) occurs less than 30 seconds after the third event. But it also occurs more than 60 seconds after the first event. So a new message is sent.

E1 E2 E3 E4
1m = 60s
25s 25s 25s
No message is sent.
Message is sent.
Ex Events that are identical.
SUPP_DUPL_COND 30s
RESEND 1m
Implementing Message Policies
Strategies for Optimal Message Filtering
Chapter 4 373
Suppression Based on Counter
Figure 4-11 illustrates suppression based on counter.
Figure 4-11 Suppression Based on Counter
1. The first event (E1) matches a condition. The counter increments to
one. No message is sent because the threshold of two has not yet
been reached.
2. A second matching event (E2) occurs 25 seconds later. The counter
increments to two. A message is sent. The counter resets to zero.
3. A third matching event (E3) occurs. The counter increments to one.
No message is sent.
4. The next matching event (E4) occurs more than 30 seconds after the
third event. At 30 seconds, the counter was reset to zero. So the
counter now increments to one. No message is sent.
E1 E2 E3 E4
25s 25s 35s
No message is sent.
Message is sent.
Ex Events that are identical.
No message is sent.
COUNTER_THRESHOLD 2
RESET_COUNTER_INTERVAL 30s

Suppressing Duplicate Messages on the Management Server Suppression of duplicate messages can also be configured for the management server. By suppressing duplicate messages on the management server, you can significantly reduce high system loads caused by large numbers of messages. In addition, you can correlate messages from more than one managed node. The suppression method used by the management server is identical to the Suppress Identical Output Messages method used on the managed node. HPOM compares the message attributes of incoming messages with the message attributes of existing messages. If either message does not have a message key, the following message attributes must be identical:

Severity

Node

Application

Message group

Object

Message text

Service name

If an identical message already exists, HPOM suppresses the duplicate message and all subsequent duplicate messages. HPOM also starts a counter of duplicates on the first message. This counter is displayed in the browser windows of the responsible operator, as well as in the Message Properties window. The counter gives an indication of how often the problem occurred. The Message Properties window also shows when the last duplicate message was received (and suppressed). If suppression is enabled, HPOM stores information about suppressed duplicate messages in annotations to the first message.

NOTE Automatic actions started by duplicate messages on the managed node are still executed. Any action responses, however, are lost.

Enabling Duplicate Message Suppression on the Management Server

Duplicate message suppression on the management server can be enabled with the opcsrvconfig -dms command. You can also specify that the duplicated messages are added as annotations. For example:

# opcsrvconfig -dms -enable anno

NOTE Duplicate message suppression is a global setting that affects all messages and operators.

For more information, see the opcsrvconfig(1m) manpage. Suppressing Duplicate Messages in Flexible Management Environments

In flexible MoM environments, each management server is responsible for counting the messages it receives from its direct managed nodes. This means that messages received from other management servers are not aggregated into one message but are displayed as multiple messages, each with the count as received from the originating management server.

If you do not want a management server to send or receive message count events, set the following variables on the HP Operations management server by using the ovconfchg command-line tool:

Send message count events ovconfchg -ovrg <OV_resource_group> -ns opc -set \ OPC_SEND_MSG_COUNT FALSE
Receive message count events ovconfchg -ovrg <OV_resource_group> -ns opc -set \ OPC_ACCEPT_MSG_COUNT FALSE <OV_resource_group> is the name of the management server resource group. By default, both variables are set to TRUE.

Updating Severity and Message Text of Duplicate Messages

If an incoming, duplicate message has a different severity or message text than the already existing messages, you can configure HPOM to display the new values instead of the previous data:

Update severity of duplicate messages The following command configures HPOM to display the severity of the last duplicate message received: ovconfchg -ovrg server -ns opc -set OPC_UPDATE_DUPLICATED_SEVERITY LAST_MESSAGE
Update message text of duplicate messages The following command configures HPOM to display the message text of the last duplicate message received: ovconfchg -ovrg server -ns opc -set OPC_UPDATE_DUPLICATED_MSGTEXT LAST_MESSAGE When set to LAST_MESSAGE, the appropriate value will be changed in the message browser.

Logging Messages

The message handling facility of HPOM produces the following types of message results:

Messages Matching Message Conditions These messages are filtered into HPOM. They are forwarded from the local node to the management server for further processing. You can also specify logging on the source node of these messages. Matched messages are displayed in the browser windows of the responsible operators.
Messages Matching Suppress Conditions These messages are filtered out of HPOM without further processing. You can choose to log these messages on their source nodes.
Messages Not Matching Any Conditions

If no filtering occurs, unmatched messages can still be routed into HPOM. Typically, unmatched messages are messages you have not seen before, so you have not yet set up filtering conditions for them. You can choose to log these unmatched messages locally on the managed node, or forward them to the management server for processing. If you forward them to the management server, you can then choose to display them in the Java GUI Message Browser, or to put them directly into the history database. If they are displayed in the browser, they are identified by an X in the U column of the Java GUI Message Browser.

You can set up logging options after you have defined message source policies and their message and suppress conditions. You specify how you want message types logged by using the following keywords in the policy body:

LOGMATCHEDMSGCOND
LOGMATCHEDSUPPRESS
LOGUNMATCHED

NOTE If you switch on logging for any of the event correlation policies, logging is automatically enabled for all other event correlation policies.

After a message has been integrated and filtered by HPOM, you can change its current default message group on the management server. You can customize message groups specifically suited to an operator’s tasks and responsibilities. This means that you are not required to change the conditions for a message source. And you are not required to redistribute the policies to move messages from one message group to another.

NOTE For information about the use of the service name attribute in regroup conditions, see the Service Navigator Concepts and Configuration Guide. Message conditions and attributes are processed as follows:

Message and Suppress Conditions Message conditions and suppress conditions are processed on managed nodes, where they are compared with all incoming messages.
Regroup Conditions Regroup conditions are processed on the management server only. Regroup conditions are compared to messages that have already passed through the filters on managed nodes. Messages that match a regroup condition are forwarded to a different message group, according to your policies. You can group all messages from the spooling applications can be grouped into the message group Output.
Messages Attributes As with message and suppress conditions, message attributes you define for the regroup condition are used in any check against the actual values of the messages.

Defining a Regroup Condition

If you want to define a regroup conditions, you need to use the Administrator’s GUI (CVPL). See the CVPL user documentation available for download in the HP Operations Manager for UNIX directory at: http://support.openview.hp.com/selfsolve/manuals. NOTE If you regroup messages into a message group that does not exist, all messages relating to this message group belong, by default, to the message group Misc until that message group is created. To check the original message group of a message currently assigned to Misc, use the Java GUI Message Properties window of the Java GUI Message Browser.

Alternatively, you can use the regroup conditions APIs. For more information, see the HPOM Developer’s Reference.

Examples of Regroup Conditions

The message conditions shown in earlier examples form the basis for the regroup condition example shown in this section. All messages filtered into HPOM have been forwarded to the message group FINANCE.

Now you want to split the messages into two groups:

Payroll
Accounting

The regroup conditions for these two groups are listed below: Regroup Condition No. 1

Application: idris4|idris5
Application: FINANCE PAYROLL
Text Pattern: ^***PAYROLL:[ERROR|WARNING]
New Message Group: payroll

Regroup Condition No. 2
Application: idris4|idris5
Application: FINANCE ACCOUNTING
Text Pattern: ^***ACCOUNTING:[ERROR|WARNING]
New Message Group: accounting
Implementing Message Policies

Top Visited <p>Your browser does not support iframes.</p>					Switchboard
					Latest
					Past week
					Past month

NEWS CONTENTS

Old News ;-)

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers : Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda : SE quotes : Language Design and Programming Quotes : Random IT-related quotes : Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce : Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 : Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds : Larry Wall : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS : Programming Languages History : PL/1 : Simula 67 : C : History of GCC development : Scripting Languages : Perl history : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month : How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

Copyright © 1996-2021 by Softpanorama Society. www.softpanorama.org was initially created as a service to the (now defunct) UN Sustainable Development Networking Programme (SDNP) without any remuneration. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License. Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.

This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...

You can use PayPal to to buy a cup of coffee for authors of this site

Disclaimer:

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: March 12, 2019