Tivoli TEC State Correlation Engine

IBM Tivoli Enterprise Console state correlation engine was introduced in TEC 3.8.  The idea was both to resolve the problem of low speed of the prolog-based correlation in TEC as well as provide more lightweight simple filtering mechanism (TEC rules are an overkill for simple tasks). Both goals were successfully met. The only problem is that this is a completely different engine, written in Java which uses XML-style configuration files.

One on the best descriptions is at Tivoli Field Guide - TEC 3.9 State Correlation Engine: How to Prevent TEC from Becoming Flooded  

The purpose of this field guide is to describe the functionality of the IBM Tivoli Enterprise Console state correlation engine introduced in ITEC version 3.8. It also outlines the configuration and design aspects as well as gives hints for installing and troubleshooting. Some case studies from different customers are discussed at the end.

Tivoli Event Integration Facility User's Guide - Rules

Correlation is achieved with state-based and stateless rules. You specify these rules by using XML syntax, defined by the supplied DTD file, tecsce.dtd. The rules also have non-XML elements that define the associated rule predicates. The location of the default XML file is $BINDIR/TME/TEC/default_sm/ tecroot.xml. This same directory also contains other samples of state correlation XML files. These files are only found on the system where you installed Tivoli Event Integration Facility or the Adapter Configuration Facility. They are not distributed with the default profile to other systems. For more information about the additional sample files, see the readme.txt file in the same location.

Note:

Rules in state correlation are not the same as IBM Tivoli Enterprise Console rules.

You define each rule in a state machine. The state machine gathers and summarizes information about a particular set of related knowledge. It is composed of states, transitions, summaries, and other characteristics, such as expiration timers and control flags.

State-based rules are the following: duplicates, threshold, and collector, all based on state machines. Each state machine looks for a trigger event to start it. Additionally, there is the matching rule, which is a stateless rule.

State-based rules rely on a history of events, whereas the stateless rules operate on a single, current event. Rules are specified by the following:

• Predicates for matching events relevant to that rule
• Actions that run after the rule triggers
• Attributes, such as a threshold limit

Predicates

A predicate in the predicate library consists of a boolean operator and zero or more arguments. Each argument can be a predicate returning the following:

Table 12. Predicate types and examples
 

Predicate Type	Example
Boolean value	Equality
Function returning a value	Addition
Event attribute	&hostname
Constant	The string foobar

See "Predicate Library" for more information.

Actions

The two actions for state correlation are the Discard action and the TECSummary action. These actions support a common, optional boolean attribute, named singleInstance. If this attribute is false, the action is not shared among different rules. Thus, one instance of the action is created for every rule that triggers it. This is the default behavior. If the attribute is true, a single instance of the action is created and shared among all rules that trigger it.

The Discard action explicitly discards an event when a state machine is triggered. Thus, the event is not forwarded. This action has no arguments. The following XML fragment shows an example with the Discard action:

<rule id="root.match_discard_tec_notice">
  	 <eventType>TEC_Notice</eventType>
  	 <match>
      		<predicate>
         			<![CDATA[
             				# always succeeds
             				true
         			]]>
      		</predicate>
   	</match>
   	<triggerActions>
      		<action function="Discard" singleInstance="true"/>
   	</triggerActions>
</rule>

The summary action, TECSummary, compacts a list of correlated events into the summary event, which is then sent to the event server. The action packs all the events that match a specific rule in a single event. Additionally, this action has an optional msg parameter. The msg parameter specifies the value of the msg attribute to be added to the TECSummary event. The msg attribute acts as an identifier for different types of the TECSummary events. Thus, you can use the msg attribute as a means to identify events easily in the event console.

If the event is generated using more than one source event, the repeat_count attribute is added to it. It then contains the number of events that were originally processed. Also, if the original events already had a repeat_count attribute, their values are preserved by adding them to the final repeat_count value of the summary. For example, the following events are received:

EVENT;repeat_count=3;msg=event1;
EVENT;repeat_count=5;msg=event2;
EVENT;msg=event3;

The generated summary has a repeat_count of the following:

repeat_count = 3 + 5 + 1 = 9

The following XML fragment shows how to configure the TECSummary action:

<rule id="root.duplicate_tec_db">
   	<eventType>TEC_DB</eventType>
   	<duplicate timeInterval="10000">
      		<cloneable attributeSet="sql_code"/>
      		<predicate>
         			<![CDATA[
            				# If we reach this point then
            				# the sql_code is already duplicated
            				# because it is used as a cloneable
            				# parameter.
            				true
         			]]>
      		</predicate>
   	</duplicate>
   	<triggerActions>
      		<action function="TECSummary" singleInstance="false">
         			<parameters>
            				SET:msg=root.duplicate_tec_db.summary
         			</parameters>
      		</action>
   	</triggerActions>
</rule>

Attributes Common to All Rules

The following are attributes common to all rules:

id

Specifies the identifier for each rule. It must be unique within the correlation engine where it is registered. Periods are treated as directories. For example, if you have the id test.threshold, you cannot have another rule with test.threshold.1 as the identifier.

 

eventType

Specifies the set of event classes this rule applies to and optimizes performance. When you omit this parameter, state correlation applies the rule to all event classes.

Matching Rules

Matching rules are stateless. They perform passive filtering on the attribute values of an incoming event. A matching rule consists of a single predicate; if the predicate evaluates to true, the trigger actions, which are specified in the rule, run. The following is an example of the rule:

<!--
Match all heartbeat events for my hostname
that have msg="please match me".
-->
<rule id="test.match" >
    <eventType>TEC_Heartbeat</eventType>
    <match>
      <predicate>
        <![CDATA[ 
           &msg == "please match me" &&
           &hostname == "hostname1"
           ]]>
      </predicate>
    </match>
</rule>

Duplicates Rules

The duplicates rule blocks the forwarding of duplicate events within a time interval . It requires these arguments:

• A time interval during which state correlation blocks duplicates of the trigger event. You control the interval with the timeInterval attribute, specified in milliseconds. The trigger event is the first event detected by the duplicates rule and is the only one that is not actually discarded.
• A predicate that is used in detecting the trigger event.

Figure 4 shows the state transitions for the duplicate rule:

Figure 4. State transitions for the duplicate rule

 

In Figure 4, state one is the initial state. Transition 1 occurs when there is a match on an incoming event. At that time, state correlation forwards the matching event, and the timer starts. Transition 2 occurs when the time interval expires, and the state machine resets. The following is an example of the rule:

<!--
Show me only the first error number 10
for my hostname that happens each 10
seconds.
-->
<rule id="test.duplicate" >
    <eventType>TEC_Error</eventType>
    <duplicate timeInterval="10000">
      <predicate>

        <![CDATA[ 
           &msg == "internal error on my adapter" &&
           &hostname == "hostname1" &&
           &errno = 10
          ]]>
      </predicate>
    </duplicate>
</rule>

Threshold Rules

The threshold rule looks for n occurrences of an event within a time interval . When the threshold is reached, it sends events to the defined actions. The threshold rule requires the following parameters:

• One of the sending modes specified by the triggerMode attribute:

  
  FIRST_EVENT

  Sends the first event.

  LAST_EVENT

  Sends the last (nth) event.

  ALL_EVENTS

  Sends all events 1 through n, the default mode.

  FORWARD_EVENTS

  Sends all events after the nth until it resets.

• A time interval during which the threshold has to be reached. You control the interval with the timeInterval attribute, specified in milliseconds.
• The time interval mode that indicates if the time interval is fixed. The attribute timeIntervalMode=fixedWindow | slidingWindowinterval controls the mode. The default value is fixedWindow.
• The number of events to match, specified by the thresholdCount attribute.
• A trigger predicate that is used in detecting 1 through n events.

Figure 5 and Figure 6 show the operation of the threshold rule with timeIntervalMode=fixedWindow specified.

Figure 5. State transitions for the basic threshold rule


 

Figure 5 shows the state machine for the modes FIRST_EVENT, LAST_EVENT, and ALL_EVENTS. Transition 1 occurs when state correlation detects the trigger event (trigger predicate matches). Transition 2 takes place when an incoming event matches the second predicate. When the time interval expires, transition 3 occurs and the state machine resets. Transition 4 resets the state machine after the threshold is reached. When the state SN is reached, either the first event, the last event, or all n events are sent before resetting.

Figure 6. State transitions for the threshold rule using FORWARD_EVENTS


 

In FORWARD_EVENTS mode (Figure 6), the threshold rule operates as in the previous case. Except, it sends all events matching the second predicate after the threshold is reached and until the time interval expires.

When the state machine has timeIntervalMode=slidingWindow specified, the operation of the threshold rule is the same as the fixedWindow time interval. Except that from each node K, there is a transition of 1, 2, .., K-1. This transition accounts for events that are not in the sliding time window. The following is an example of the rule:

<!--
I'm only interested when at least 5 Node_Down
events for hostnames in my local subnet happen
within 1 minute.
-->
<rule id="test.threshold">
    <eventType>Node_Down</eventType>
    <threshold thresholdCount="5" timeInterval="60000"
    timeIntervalMode="slidingWindow" triggerMode="allEvents">
      <predicate>
         <![CDATA[
            (&msg == "node down") &&
            (isMemberOf(&hostname, [ 192.168./16 ]))
            ]]>
      </predicate>
    </threshold>
</rule> 

Threshold rules can also define complex aggregate values, instead of a simple count of events. Use the aggregate configuration tag to define this rule. You can construct an aggregate value similar to the definition of a predicate. But instead of a simple true or false result, define a progressive value using the functions listed in Appendix D, Predicates and Functions for State Correlation. Threshold rules with aggregate values trigger only when the aggregate value is equal or greater than the thresholdCount value. The following is an example of the rule:

<!--
If I receive a slot value with a relative percentage between
0 and 1, but I want to check my threshold using the normal
percentage value of 100%, I can define an aggregate of the
slot relative_percentage, by multiplying it by 100 and counting
all percentages until it reaches 100%.
-->
<rule id="test.aggregate_threshold">
    <eventType>Temperature_Variation</eventType>
    <threshold 
        thresholdCount="100" 
        timeInterval="2000"
        triggerMode="allEvents"
        timeIntervalMode="fixedWindow" >
      <aggregate>
         <![CDATA[
           &relative_percentage * 100
           ]]>
      </aggregate>
      <predicate>true</predicate>
    </threshold>
  </rule> 

Collector Rules

The collector rule gathers events that match the given predicate for a specified period of time . The rule triggers when the timer expires and sends all collected events to the defined actions. The collector rule requires these arguments:

• A time interval during which matching events are collected. You control the interval with the timeInterval attribute, specified in milliseconds.
• A predicate, which is part of filtering the relevant events to add to the collection.

Figure 7 shows the state transitions for the collector rule:

Figure 7. State transitions for the collector rule


 

In Figure 7, S1 is the initial state. Transition 1 occurs when there is a match on an incoming event; the initial event is not sent but collected. A timer is set to the specified interval. Before the timer expires, all incoming and matching events are collected (transition 2). Transition 3 occurs when the time interval expires, and the state machine resets. At this time, all collected events are sent. The following is an example of the rule:

<!--
Collects 10 seconds of Server_Down
events for my database.
-->
<rule id="test.collector">
    <eventType>Server_Down</eventType>
    <collector timeInterval="10000" >
      <predicate>
        <![CDATA[ 
            &servername == "my_database"
            ]]>
      </predicate>
    </collector>
  </rule>

NEWS CONTENTS

Old News

Description: Creating Custom Actions for the TEC 3.9 State Correlation Engine (SCE) and Configuring the Windows Logfile Adapter to use the SCE
Version: | File size: 0 bytes
Added on: 04-May-2004 | Downloads: 1006

Recommended Links

Softpanorama Recommended

Top articles

Sites

State Based Correlation Engine (SBCE) Performance

Determining the length of time a single event enters and leaves the State Based Correlation Engine

Determining tec_gateway and state based correlation performance

In large and small environments, it may be necessary to look at how much time it takes for the tec_gateway process to receive an event, pass it to state based correlation, get it back from state based correlation and send it on to the TEC Server

Tivoli Field Guide - TEC 3.9 State Correlation Engine: How to Prevent TEC from Becoming Flooded

The purpose of this field guide is to describe the functionality of the IBM Tivoli Enterprise Console state correlation engine introduced in ITEC version 3.8. It also outlines the configuration and design aspects as well as gives hints for installing and troubleshooting. Some case studies from different customers are discussed at the end.

Tivoli Field Guide - Event Processing Tools Available in IBM Tivoli Enterprise Console 3.8

The purpose of this paper is to provide an overview of the event processing tools available in IBM Tivoli Enterprise Console version 3.8. It also ties these tools together so the customer can make informed decisions when planning event management strategies and implementations. Pre-filtering, filtering, state-based correlation, rules, gateways, and endpoints are all discussed.

Tivoli Field Guide - The Tivoli Management Enterprise Endpoint Gateway: A Technical Look at the Internals

An unstable endpoint manager or endpoint gateway can have a catastrophic ripple effect on the reliability and effectiveness of the Tivoli® Management Environment. This is primarily because of the number of management units that are affected by their instability. An unreliable gateway with 1,000 endpoints logged into it affects the ability of the administrator to effectively manage those 1,000 machines. In an environment of 10,000 endpoints, that represents a 10% failure rate...

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: March 12, 2019