Softpanorama

May the source be with you, but remember the KISS principle ;-)
Home Switchboard Unix Administration Red Hat TCP/IP Networks Neoliberalism Toxic Managers
(slightly skeptical) Educational society promoting "Back to basics" movement against IT overcomplexity and  bastardization of classic Unix

Profiling and Tuning TEC rules

News See also Recommended Links Creation Compiling Loading Testing

TEC Documentation

WRB command options TEC Rules Programming Consistency checking Tips Humor Etc

The following list describes suggested techniques for tuning rules.

SG246614

Rule profiling

Rule profiling generates a report that holds rule execution information. Such a report contains the following information for each rule being profiled:

Profiling can only be activated via CLI commands.

chatham> wrb -lscurrb
The currently used rule base was loaded from the rule base named 'spooler_rb'.
chatham> wrb -comprules -profile spooler_rb
...
chatham> wrb -loadrb -use spooler_rb
...
chatham> wstopesvr
chatham> wstartesvr
The Tivoli Enterprise Console Server is initializing...
The Tivoli Enterprise Console Server is running.

After restarting the Event Server, the profiling starts. A report is written to the $DBDIR/tec/profile file. For each new event processed, a record is appended to this file.

Tip: On restart of the Event Server, the old file contents are still kept. This can cause a large file if the profiling runs for some time, especially if many events are processed. For this reason, we recommend checking periodically for the file size if profiling is used to avoid filling the $DBDIR file system.

Example 14-34 shows an example of a profile report.

Example 14-34 Profile report

===========================================
Timing Summary                
--------------------------------------------
prefix:
Time for last Event: 1.000000000000001e-02
Event Count:         13
Total Time:          1.200000000000001e-01
Events per second:   1.083333333333332e+02
--------------------------------------------
prefix_msgg_rl_of_set_prefix:
Time for last Event: 0.000000000000000e+00
Event Count:         13
Total Time:          8.000000000000007e-02
Events per second:   1.624999999999999e+02
--------------------------------------------
--------------------------------------------
============================================

SG246614

14.5.2 Event Activity Reports

Event Activity Reports represent a textual listing of event activities in the rule engine. The following items can be specified to report and can be dynamically changed during reporting:

How often to write the report
The file and path name for the report
Events to exclude from the report
Attribute criteria to include in the report
Counter threshold (for example, "do not include counts of less than 5 in the report")

The relevant parts of the im.rls file are shown Example 14-35. The im_configure rule includes the initial configuration settings to create an Event Activity Report (the editable parts are shown in bold):

Example 14-35 Event Activity Report

----------------------------------------------------------------
% NAME - IM Configuration.
% This rule is used to configure the IM rule behavior.
% Event activity reporting, event filtering, maintenance mode,
% heartbeat, thresholding and event correlation can be activated.
%---------------------------------------------------------------- 
rule:  im_configure : (
event: _event of_class  'TEC_Start'
where [],
reception_action: im_parameters: (
...

 

      %----------------------------------------------
% Activate rule functionality (active/inactive)
%----------------------------------------------
record(event_activity,inactive),
...

 

      %------------------------------------------------------------------
% Event activity
% This section allows the event activity report to be customized.
% The name of the report file, the reporting frequency, 
      % event reporting threshold, events to exclude from the report and 
      % attribute criteria to include in the report can be specified.
%------------------------------------------------------------------
( recorded(event_activity,active) ->
_reporting_frequency is 20,                     % Write statistics this often (in seconds)
init_event_activity('/tmp/event_activity',      % Store report in this file  
                              ['TEC_Heartbeat',          % Do not report these events
'TEC_Maintenance'],
[ source,                  % Attribute reporting criteria
hostname,                % Can be count of single attribute
severity,
status,
[hostname,severity],     % Or multiple attributes
[class,hostname]         % use class key word to report by class name
],
5                          % Do not report counts less than this 
                              ),
first_instance(event:_ev of_class 'TEC_Tick' where [ ]),
set_timer(_ev, _reporting_frequency, 'Event Activity Report')
;
true
),
...
).

 

To enable Event Activity Reporting, change the line:

record(event_activity,inactive),

to:

record(event_activity,active),

In the Event activity section, all settings for what should be reported can be applied. The _reporting_frequency can be specified as well as the initial event activity settings, such as:

init_event_activity(	output_file_name,
event_exclusion_list,
attribute_reporting_criteria,
reporting_threshold)

The Event Activity rules provided with the im.rls file in the default rule base are shown Example 14-36.

Example 14-36 Event Activity rules

%-----------------------------------------------------------------------------------------
% Collect event activity statistics:
% This rule is used to collect event activity statistics. It is activated when statistics are turned on in the initialization rule.
%-----------------------------------------------------------------------------------------
rule: update_event_activity: (
event: _event of_class _class
where [],
reception_action: update_activity: ( 
     recorded(event_activity,active),   
     update_event_activity(_event)
)
). 

 

%-----------------------------------------------------------------------------------------
% Print Event Activity Report:
% This periodic call prints the event activity report. Report properties are set in the initialization rule. After the report is printed, all statistics are reset.
%-----------------------------------------------------------------------------------------
timer_rule: reset_event_activity: (
event: _event of_class _class
where [],
timer_info: equals 'Event Activity Report',
timer_duration: _rep_freq,
action: reset_activity: ( 
     recorded(event_activity,active),
print_event_activity,
reset_event_activity,     
     set_timer(_event,_rep_freq,'Event Activity Report')
)
).

Speed TEC vs. Netcool - Who processes events faster Gulf Breeze Software Partners Tivoli Consulting Training We are Tivoli experts

Posted by: justin_eagleson on May 05, 2006 - 05:43 PM

BlogArticle  OK, now I'm treading on sacred ground...
Obviously hardware matters since both are CPU driven applications, but assuming we're dealing with like hardware, the answer is it depends...

TEC

IBM has published benchmarking speeds of the TEC rule engine (a while ago with TEC 3.8 FP whatever running on AIX) at ~65 events per second.

How did they come to derive this number? They start a timer when events are flowing at TEC and stop the timer when the CPU on TEC goes below 5% used.

Is that a good measurement? Maybe, but IMHO not a good way to tell whether the rule engine is really done processing an event, that's for sure. It would be better to actually measure whether the event is PROCESSED in the reception log to really know when the rule engine is finished processing an event. I've also seen events per second increase with a TEC database installed locally rather than remotely.

So documented by IBM, TEC is around ~65 events per second. Of course that is without running any real rules against the incoming events. Another factor to consider is I.V., who as we all know has achieved 300+ events per second using his non-tme rule engine.

Netcool Omnibus

Based on what I've heard in the past from MicroMuse when asked how fast Omnibus processes events, I've heard "thousands of events per second". When we drilled deeper and asked more questions, we were told those were "raw" events and it's more like "hundreds of events per second". What is unclear is whether these events were acted upon by any rules (i.e. is this merely event reception speed). Unfortunately, I don't have any hard benchmarking numbers to share at this point until I get my hands on Netcool code.

So based on this, it really depends on a number of factors such as what kind of rules are you running against your events, how you tell when an event is "processed" by the rule engine, and whether or not you are I.V. ;-)

Re: Speed: TEC vs. Netcool - Who processes events faster?

(Score: 1)
by ivb on May 06, 2006 - 02:43 AM
(User information http://) Firstly, it not my non-tme rule engine, it is IBM's shipped totaly unsupported with TEC as a tar file in $BINDIR/TME/TEC directory. Just wanted to get that straight ;-). Beides myself, Loren Cain is the only other guy I know of brave enought to use it in a production environment.

I found the IBM benchmark flawed as well.. it's trivial to write Prolog that will track processing time, but it is hard to get more than one second granularity. What I have done in the past is use strace/truss with milisecond timestamps to get a higher resolution of processing time. You simply have to get the tec_rule process to make trackable system call such as opening or closing a file. They also go into no detail about how complex the rulebase is. How large is the event cache. In my experience TEC running good rules that perform only one cache search to do duplicate/cause/effect/clearing processing on an 10K cache should handle 40 to 50 events per second. Also, just because you have room for 10K events does not mean you have to have 10K events in your cache, keep your cache lean.

What I found to be a limiting factor with the non-tme rule engine was the network. Think about it, hundreds of connections negotiated, event data transmitted, and the connections torn down a second. It would be very easy to saturate a 100mb network or system TCP/IP stack. I think the same will hold true for Netcool.

I think what we TEC guys think of as "deep" correlation and "enrichment" will be handled by the Netcool "Impact" product. At the conference this year there was mention of "entitlements" when IBM converges the correlation environment. Customers should contact thier sales reps to find out what they will be entitled to down the road. Impact may cost more... or may not... we will have to wait and see.

So what makes TEC so slow? IIHTG, it's the dependency on a database (accessed by two separate processes) through and absraction pipe of the RIM coupled with the overhead of IPC calls to get event data from tec_reception to tec_rule to tec_dispatch.

I would like to see some performance numbers on the Netcool products with Impact running

 



Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotesSomerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose BierceBernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOSProgramming Languages History : PL/1 : Simula 67 : C : History of GCC developmentScripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-MonthHow to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D


Copyright © 1996-2021 by Softpanorama Society. www.softpanorama.org was initially created as a service to the (now defunct) UN Sustainable Development Networking Programme (SDNP) without any remuneration. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License. Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.

This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...

You can use PayPal to to buy a cup of coffee for authors of this site

Disclaimer:

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: March 12, 2019