Snort-related Perl Scripts

Several programs are available to post-processing Snort log files:

• snort2html -- generator of  HTML pages from snort entries in a log file is  (http://escert.upc.es/tools/snort/contrib/snort2html). This is a simple Perl script that will parse the log file and generate a report with all these entries.
   
• Snortalog is a Perl script that summarizes Snort logs, making it easy to view any network attacks detected by Snort. It can generate charts in HTML, PDF, and text output. It works with all versions of Snort, and can analyze logs in three formats: syslog, fast, and full snort alerts. Moreover, it is able to summarize other logs like CheckPoint Fw-1 (NG and 4.1), Netfilter, IPFilter, Packet Filter, CISCO PIX, and Lucent BRICK in a similar way. Output options include ASCII text, HTML, and graphs (formatted in JPEG, GIF, or PNG). Snortalog is configured and managed from a GUI interface, and it runs on either Linux or Windows. It reads output from Snort in any format (no other tool that we've seen has this feature), including syslog, provides fast and full alerts, and then builds flat text or HTML summary reports. Snortalog's summary reports are similar to ACID's reports, but more compact.
   
• SnortSnarf: SnortSnarf is a Perl program that will read your log files and generate HTML output. This lets you look into your Snort log to find possible problems. The output contains links to WHOIS lookup and DNS lookup. You can also choose to find out what attacks are associated with certain IP addresses.

SnortSnarf comes with other functions besides log analysis. It contains nmap2html that will turn the result of a nmap scan into a HTML page. The latest addition is SISR (SnortSnarf Incident Storage and Reporting mechanism), which will help you to create a report for an incident and store that report or send the report by email efficiently.

To install SnortSnarf, first download it from the Web site (the latest version is 041501.1):

http://www.silicondefense.com/software/snortsnarf/index.htm

Make sure you have the Perl module you need.

• The Time::JulianDay module is included in the tarball. Prepare a directory for the HTML output, and then issue the following command:

Usage:

snortsnarf.pl -d /where/is/html/output /var/log/messages /var/log/snort/alert

This command will parse /var/log/messages and /var/log/snort/alert files, and then generate the HTML output in /where/is/html/output directory. There are other possible options listed in the snortsnarf.pl file.

The example output of SnortSnarf can be found at:

http://www.silicondefense.com/software/snortsnarf/example/index.html

NEWS CONTENTS

• 20061022 : DefCon 13 ( DefCon 13, Oct 22, 2006 )
• 20061022 : Raffy's Computer Security Blog " Visualization by Raffael Marty ( Raffy's Computer Security Blog " Visualization, Oct 22, 2006 )
• 20061022 : Unix Review : Snort 2.6 and Afterglow by Kristy Westphal ( Unix Review : Snort 2.6 and Afterglow, Oct 22, 2006 )
• 20061022 : SourceForge.net AfterGlow ( SourceForge.net AfterGlow, Oct 22, 2006 )
• 20061021 : SnortRule - Perl extension for dynamically building snort rules - search.cpan.org ( SnortRule - Perl extension for dynamically building snort rules - search.cpan.org, Oct 21, 2006 )
• 20061001 : freshmeat.net Project details for fwsnort by Michael Rash ( freshmeat.net Project details for fwsnort, Oct 1, 2006 )
• 20061001 : [Snort-users] Script to cleanup ACID-Snort Alerts in MySQL DB... - MARC ( [Snort-users] Script to cleanup ACID-Snort Alerts in MySQL DB...' - MARC, )
• 20061001 : README ( README, )
• 20061001 : filtering snort alerts - Google Search ( filtering snort alerts - Google Search, )
• 20061001 : There is a very fine line between hobby and mental illness. by Brian Caswell ( There is a very fine line between 'hobby' and 'mental illness.', )
• 20031222 : snortconfig 1.13 is out ( snortconfig 1.13 is out, Dec 22, 2003 )

Old News ;-)

[Oct 22, 2006] DefCon 13

General security tools discussion. Some info on afterglow

[Oct 22, 2006] Raffy's Computer Security Blog " Visualization by Raffael Marty

Some examples of afterglow output from Afterglow author

Yes, I am still playing around with visualization. In fact, I am writing a Chapter for a book. It's going to be about using line graphs and treemaps to visualize firewall and IDS logs.

I just realized something that you have to be careful about when using AfterGlow's fan-out filtering!

If you have a graph where you are going to filter based on the source-node's fan-out, you will still see all the eventNode->targetNode pairs! It won't restrict them to only that ones that are from the remaining source nodes! Here is an example:

My log contains thousands of line, but only these three entries should remain after doing a (-f 2) filtering:

12.108.194.82,nounce,111.222.69.44|113
12.108.194.82,block,111.222.69.44|113
12.108.194.82,pass,111.222.69.44|25

However, the graph will look as this:

[Oct 22, 2006] Unix Review : Snort 2.6 and Afterglow by Kristy Westphal

Very incoherent article. It looks like the author does not understand the topic he is writing about.

... What other tools are available with Snort? How about an alert trending tool called pigsentry? How about another log analyzer called snortalog (Perl based) that also does some reporting? How about a tool that helps do automatic IP blocking (snortsam)? Snortsam does this by interacting with many major firewalls (Cisco, Checkpoint, IP Filter).

... ... ....

Afterglow, by Raffael Marty, specified that I needed the AT&T Graphviz library installed. They had the pre-complied rpm for Fedora, so I decided that was the best way to go. The install went flawlessly. By this time I had generated a lot of traffic, so I was ready to use Afterglow to see what it looked like. I ran the command per the readme file:

tcpdump -vttttnnelr snort.log.1157826308 | tcpdump2csv.pl "sip dip ttl"  \
   afterglow.pl -c color.properties -p 2  | neato -Tgif -o test.gif

(Note: make sure you have all the appropriate paths set up in your environment file so that it can find all the executables). As always, there are some gotchas. I got an error when I ran the command (nothing that couldn't be cleared up with a little troubleshooting, though). I broke it down section by section in the command.

It looked as though I successfully got this far with my command line:

tcpdump -vttttnnelr snort.log.1157826308 | tcpdump2csv.pl "sip dip ttl"

because I was able to generate a csv file that indeed had the sip, dip, and ttl listed by line. Things started to get murky when I tried to cat the csv file to the afterglow command listed:

afterglow.pl -c color.properties -p 2  

'Cannot open color.properties: No such file or directory at afterglow.pl line 950.'

Ah, so somewhere I was missing a definition file for color properties. Where do I magically create one of these? Even when I pointed to the color.properties that I found buried in a directory, it still didn't like it. I noticed in the Perl code for Afterglow that it would just use a default, if not specified, so I simply left out the statement and ran it. Finally! A dot file had been created. The output of the dot file was definitely different from just the csv file:

192.168.1.101,192.168.1.255,128
192.168.1.102,66.187.224.4,64
66.187.224.4,192.168.1.102,40
192.168.1.102,66.187.224.4,64
66.187.224.4,192.168.1.102,40
192.168.1.102,66.187.224.4,64
66.187.224.4,192.168.1.102,40

Tmp.dot

"192.168.1.102" -> "68.2.16.25 64" ;
"192.168.1.100" -> "192.168.1.255 128" ;
"192.168.1.101" -> "192.168.1.255 128" ;
"192.168.1.102" -> "209.132.177.50 64" ;
"192.168.1.100" -> "192.168. " ;
"192.168.1.102" -> "66.187.224.4 64" ;
"66.187.224.4" -> "192.168.1.102 40" ;

Once I had the dot file, I was able to produce the gif file, which looks like Figure 1. The entire gif was too big to fit on my page, but you get the idea.

Figure 1: End resulting graph

Afterglow, when used in conjunction with a tool like Snort, can truly put your log files into a different visual dimension. Relationships that can be hard to visualize with pure text can easily be rearranged into something that can be very telling. The best part is how easy it is to create the graph! You can slice and dice your information however you wish. Just keep in mind that the more data you have in there, the more complicated the graph, and the more eye strain you will suffer.

[Oct 22, 2006] SourceForge.net AfterGlow

AfterGlow is a collection of scripts which facilitate the process of generating event graphs and treemaps. AfterGlow 1.x is written in Perl and generates output that can be read by GraphViz or LGL, whereas AfterGlow 2.0 is written in Java...

[Oct 21, 2006] SnortRule - Perl extension for dynamically building snort rules - search.cpan.org

Usability is questionable, but the idea is not bad and usage of Perl rules generation or syntax checking script might help to avoid some stupid errors in writing own rules.

This is a very simple snort rule object. It was developed to allow for scripted dynamic rule creation. Ideally you could dynamically take a list of bad hosts and build an array of snort rule objects from that list. Then write that list using the string() method to a snort rules file.

use Snort::Rule;
$rule = Snort::Rule->new(
-action => 'alert',
-proto => 'tcp',
-src => 'any',
-sport => 'any',
-dir => '->',
-dst => '192.188.1.1',
-dport => '44444',
);

$rule->opts('msg','Test Rule"');
$rule->opts('threshold','type limit,track by_src,count 1,seconds 3600');
$rule->opts('sid','500000');

print $rule->string()."\n";

OR

$rule = 'alert tcp $SMTP_SERVERS any -> $EXTERNAL_NET 25 (msg:"BLEEDING-EDGE POLICY SMTP US Top Secret PROPIN"; flow:to_server,established; content:"Subject|3A|"; pcre:"/(TOP\sSECRET|TS)//[\s\w,/-]*PROPIN[\s\w,/-]*(?=//(25)?X[1-9])/ism"; classtype:policy-violation; sid:2002448; rev:1;)';

$rule = Snort::Rule->new(-parse => $rule);
print $rule->string()."\n";

[Oct 1, 2006] freshmeat.net Project details for fwsnort by Michael Rash

fwsnort translates snort rules into an equivalent iptables ruleset. By making use of the iptables string match module, fwsnort can detect application layer signatures which exist in many snort rules. fwsnort adds a --hex-string option to iptables, which allows snort rules that contain hex characters to be input directly into iptables rulesets without modification. In addition, fwsnort makes use of the IPTables::Parse Perl module in order to (optionally) restrict the snort rule translation to only those rules that specify traffic that could potentially be allowed through an existing iptables policy.

[Snort-users] Script to cleanup ACID-Snort Alerts in MySQL DB...' - MARC

[prev in list] [next in list] [prev in thread] [next in thread]

List: snort-users
Subject: [Snort-users] Script to cleanup ACID/Snort Alerts in MySQL DB...
From: "Dusty Hall" <halljer () auburn ! edu>
Date: 2003-04-03 23:34:23
[Download message RAW]

Gang,

I just thought I'd pass this script along.. hopefully it will save someone some time/grief. The main reason I wrote it is because we are still in the process of tweaking Snort and our number of Alerts get out of hand quickly. ACID's frontend to delete the Alerts timed out most of the time and I wanted a way to schedule the cleanup of Alerts..

Later,

-Dusty

--CODE--

#!/usr/bin/perl -w
#----------------------------------------
# name: alert_cleanup.pl
#
# description: script to cleanup snort/acid db (only tested w/mysql)
#
# goal: allows you to schedule db cleanup without using php frontend
#
# usage: snort_db_cleanup.pl "2003-04-01 8:00:00" "2003-04-01 9:00:00"
#
# comments: dusty hall, halljer@<NOSPAM>auburn.edu
#----------------------------------------

use strict;
use DBI;

my $ds = "dbi:mysql:snort";
my $db_user = "acid_user";
my $db_pass = "secret";
my $db = DBI->connect($ds, $db_user, $db_pass) or die $DBI::errstr;

my ($cid,$sid,$sql,$time_select,$exec_time_select);
my
($event,$iphdr,$tcphdr,$udphdr,$icmphdr,$opt,$data,$acid_ag_alert,$acid_event);
my
($exec_event,$exec_iphdr,$exec_tcphdr,$exec_udphdr,$exec_icmphdr,$exec_opt,$exec_data, \
$exec_acid_ag_alert,$exec_acid_event); my %timeframe;

$timeframe{start} = $ARGV[0];
$timeframe{finish} = $ARGV[1];
chomp $timeframe{start};
chomp $timeframe{finish};

$time_select = "select acid_event.sid,acid_event.cid from acid_event
where timestamp >= '$timeframe{start}' and timestamp <=
'$timeframe{finish}'";
$exec_time_select = $db->prepare($time_select);

$exec_time_select->execute();
$exec_time_select->bind_columns(undef,\$sid,\$cid);

while ($exec_time_select->fetch) {

$event = "delete from event where sid='$sid' and cid='$cid'";
$iphdr = "delete from iphdr where sid='$sid' and cid='$cid'";
$tcphdr = "delete from tcphdr where sid='$sid' and cid='$cid'";
$udphdr = "delete from udphdr where sid='$sid' and cid='$cid'";
$icmphdr = "delete from icmphdr where sid='$sid' and cid='$cid'";
$opt = "delete from opt where sid='$sid' and cid='$cid'";
$data = "delete from data where sid='$sid' and cid='$cid'";
$acid_ag_alert = "delete from acid_ag_alert where ag_sid='$sid' and
ag_cid='$cid'";
$acid_event = "delete from acid_event where sid='$sid' and
cid='$cid'";

$exec_event = $db->prepare($event);
$exec_iphdr = $db->prepare($iphdr);
$exec_tcphdr = $db->prepare($tcphdr);
$exec_udphdr = $db->prepare($udphdr);
$exec_icmphdr = $db->prepare($icmphdr);
$exec_opt = $db->prepare($opt);
$exec_data = $db->prepare($data);
$exec_acid_ag_alert = $db->prepare($acid_ag_alert);
$exec_acid_event = $db->prepare($acid_event);

$exec_event->execute();
$exec_iphdr->execute();
$exec_tcphdr->execute();
$exec_udphdr->execute();
$exec_icmphdr->execute();
$exec_opt->execute();
$exec_data->execute();
$exec_acid_ag_alert->execute();
$exec_acid_event->execute();

$exec_event->finish();
$exec_iphdr->finish();
$exec_tcphdr->finish();
$exec_udphdr->finish();
$exec_icmphdr->finish();
$exec_opt->finish();
$exec_data->finish();
$exec_acid_ag_alert->finish();
}

$exec_time_select->finish;

--CODE--

README

helpful programs are in the contrib directory:

• classification.pl: Adds the classification.config to the database
  
• some cgi scripts to analyze alerts via a web server
  
• stats.pl: A script to gather statistics from snort
  
• rules.pl: A perl script to add the signatures and references of the rule files to the database

snort2html

Snort - the de facto standard for intrusion detection-prevention

#!/usr/bin/perl -- # # rework of snort log to plot graphicaly attack signatures # # History # Syslog analysis script orignially written by # Angelos Karageorgiou and # original tweaked by Martin Roesch if($ARGV[1] eq undef) { print "USAGE: snortplot \n"; print "EXAMPLE: snortlog /var/log/messages sentinel\n"; print "Note: The machine name is just the hostname, not the FQDN!\n"; exit; } $HTDIR="/home/unix/htdocs/packetplot"; $GNUPLOT="/usr/bin/gnuplot"; $HOST={}; # DNS table $timeoutalarm=3; # in 3 second the DNS resolver should timeout $machine = $ARGV[1]; use Socket; $l=0; open(LOG,"< $ARGV[0]") || die "No can do"; while() { chomp(); if ( ( ! /$machine snort/gi ) ) { next ; } $date=substr($_,0,15); $rest=substr($_,16,500); @fields=split(": ", $rest); $j=1; $text=$fields[$j++]; if ( $text =~ /spp_http_decode/ ){ $text=$fields[$j++]; } $fields[$j] =~ s/ \-\> /-/gi; ($source,$dest)=split('-', $fields[$j]); ($host,$port)=split(':',$source); $skipit=0; ($shost,$sport)=split(':',$dest); $sport =~ s/ //gi; # $name=resolv($host); $name = $name . ":" . $port; # $sname=resolv($shost); $sname = $sname . ":" . $sport; if ( $port >=65535 ) { next; } # just linux kernel weirdness ( I think ) $key= $shost . " " . $sport; $SCANNED{$key}=$host; $TEXT{$key}=$text; $SCAN{$key}++; print $l++; print " lines read\r"; if ( $text =~ /portscan/i ) { $rest =~ s/$machine snort.*\]\://gi; $rest =~ s/ spp_portscan\://gi; $mystring=sprintf("%15s %s\n", $date, $rest); push(@PSCAN,$mystring); } } close(LOG); $i=0; open(FILE,">/tmp/snortchart.spl") || die "Cannot write data"; foreach $key (keys(%SCAN)){ ($sname,$port)=split(' ', $key); $i++; printf(FILE "%d\t%-20s\t\t\t%s\t\t%d\t%d\n",$i,$sname,$SCANNED{$key},$port,$SCAN{$key}); } close(FILE); open(PRG,">/tmp/snortchart.prg") || die "Cannot write Program"; print PRG << "EOF" set title 'Attack Signatures' set xlabel 'Originator (See Below)' set ylabel 'Port (Log Sc.)' set logscale y set zlabel 'Count' set terminal png color set output '$HTDIR/snortplot.png' set grid set ticslevel 0 splot '/tmp/snortchart.spl' using 1:4:5 title '' with impulses , '/tmp/snortchart.spl' using 1:4:5 title '' with points EOF ; print "\n"; close(PRG); system("$GNUPLOT /tmp/snortchart.prg")|| die "cannot run \'$GNUPLOT\'"; open(IDX,">$HTDIR/snortplot.html") || die "cannot create html"; print IDX << "EOF2"

Syslog analysis script orignially written by Angelos Karageorgiou

#!/usr/bin/perl # Syslog analysis script orignially written by # Angelos Karageorgiou , and # tweaked by Martin Roesch # # August of 2001 changes to accomodate snort 1.8 logs # jeez guys you make life difficult # if($ARGV[1] eq undef) { print "USAGE: snortlog \n"; print "EXAMPLE: snortlog /var/log/messages sentinel\n"; print "Note: The machine name is just the hostname, not the FQDN!\n"; exit; } $HOST={}; # DNS table $timeoutalarm=1; # in 5 second the DNS resolver should timeout $machine = $ARGV[1]; $targetlen=25; $sourcelen=35; $protolen=12; use Socket; $SIG{ 'ALRM' } = "cannotresolve"; open(LOG,"< $ARGV[0]") || die "No can do"; printf("%-15s %-35s %-25s %-25s\n","DATE","WARNING", "FROM", "TO"); print "=" x 100; print "\n"; while() { chomp(); if ( ( ! /$machine snort/gi ) ) { next ; } # get rid of various warnings if ( ( /WARNING/ ) || ( /ERROR/) || ( /SIG/) || ( /Restarting/) || ( /received signal/) || ( /Initialization/i) ) { next; } $date=substr($_,0,15); $rest=substr($_,16,500); # quick hack for 1.8 logs I mean really quick $rest =~ s/\[.*?\]//giox; $rest =~ s/\(.*?\)//giox; if ($rest =~ /ICMP/ ) { doicmp(); next; } @fields=split(": ", $rest); $text=$fields[1]; if ( $text =~ /spp_stream/i ) { $text=$fields[2]; if ( ( $text =~ /NMAP/i) || ( $text =~ /STEALTH/i) ) { $text=~ s/\{.*$//; # get rid of extraneous data $text=" ". $text; } } $fields[2] =~ s/ \-\> /-/gi; ($source,$dest)=split('-', $fields[2]); ($host,$port)=split(':',$source); $host =~ m /(\{.*\})\s(.*)/; $proto=$1; $host=$2; $skipit=0; ($shost,$sport)=split(':',$dest); $sport =~ s/ //gi; $name=resolv($host); $name = $name . ":" . $port; $sname=resolv($shost); $sname = $sname . ":" . $sport; if ( $text =~ /portscan/i ) { $rest =~ s/$machine snort.*\]\://gi; $rest =~ s/ spp_portscan\://gi; $mystring=sprintf("%15s %s\n", $date, $rest); push(@PSCAN,$mystring); } else { printf("%15s %-35s %7s %-30s %s\n", $date, $text, $proto, $name,$sname); } } close(LOG); print "\n\n"; print "=" x 100; print "\n"; print " " x 40; print "PORTSCANS\n\n"; #printf("%-15s %-35s %-25s %-25s\n","DATE","WARNING", "FROM", "TO"); print "=" x 100; print "\n"; foreach $sc (@PSCAN) { print $sc; } sub cannotresolv { print "cannot resolve\n"; alarm($timeoutalarm); return 1; } sub resolv #resolv and cache a host name { local $mname,$miaddr,$mhost; $mhost=shift; $miaddr = inet_aton($mhost); # or whatever address if (! $HOSTS{$mhost} ) { alarm( $timeoutalarm ); $mname=''; $mname = gethostbyaddr($miaddr, AF_INET); alarm( 0 ); if ( $mname =~ /^$/ ) { $mname=$mhost; } $HOSTS{$mhost}=$mname; } return $HOSTS{$mhost} } sub doicmp { $rest=~ s/^.*\://; $rest =~ m/\s(.*?)\s\{ICMP}\s([\d\.]+\d)\s\-\>\s([\d\.]+\d).*/; $text=$1; $source=$2; $dest=$3; $name=resolv($source); $sname=resolv($dest); $proto='{ICMP}'; printf("%15s %-35s %7s %-30s %s\n", $date, $text, $proto, $name,$sname); }

This program gets each ip number from a snort_portscan.log file Very Old script (NT 4 with activestate 5.6.0 )You need to pass this via beautifier before trying to use or understand the code:

#This program gets each ip number from a snort_portscan.log file. #It looks up the email contact(s) for that ip number. # #It requires libwww which comes with activestate. # #output is suitable for cut and paste to email to contacts #to, subject, and pertinent log file section. #I only tested this on NT 4 with activestate 5.6.0 #robin stubbs uw madison [email protected] #usage: perl getcontact.pl snort_portscan_file #output is in getcontact.log require LWP::UserAgent; $message = "Hello. We recorded a scan of our subnet that could have come from your network(s)."; $ipnum = ""; $infile = $ARGV[0]; $myline = ""; $whichserver = ""; @contacts = (); @netblocks = (); $logfile = "getcontact.log"; open (INFILE,"<$infile") || die ("Cant open input file $infile\n"); open (LOGFILE,">$logfile") || die ("cant open log file $logfile\n"); $output = ""; while ($ipnum = &get_ipnumber($ipnum)) #get ip number from file {#print "checking $ipnum \n"; &getinfo($ipnum); #contact whois if ($whichserver eq "ARIN") {&findcontact_arin;} #parse response if ($whichserver eq "RIPE") {&findcontact_ripe;} #parse response if ($whichserver eq "APNIC") {&findcontact_apnic;} #parse response $numcontacts = @contacts; if ($numcontacts == 0) {print LOGFILE "$ipnum contact not found\n";} elsif ($numcontacts == 1) {print LOGFILE "to: ",$contacts[0],"\nsubject: $ipnum\n$message\n";} else {print "choose one contact for $ipnum\n"; for ($i = 0; $i < $numcontacts; $i++) {print $i+1," $netblocks[$i]: $contacts[$i]\n";} while (1) { $resp = ; last if ($resp >= 1 && $resp <= $numcontacts); } print LOGFILE "to: ",$contacts[$resp - 1],"\nsubject: $ipnum\n$message\n"; } }#end while print "output in $logfile\n"; #******************* subroutines *********************** sub get_ipnumber { my ($previp) = @_; if (length($previp) > 0) {print LOGFILE $myline; #this is from prev call to subroutine } while ($myline = ) {@portscan = split(' ',$myline); @ipn_port = split(':',$portscan[3]); $currip = $ipn_port[0]; if ($currip ne $previp) {print LOGFILE "\n"; return $currip; #this line is saved in global $myline } else {print LOGFILE $myline;} }# end while return ""; } #**************************** findcontact_apnic ************* sub findcontact_apnic { @contacts = (); @netblocks = (); @stuff = (); $len = length($output); @stuff = split("\n",$output); $num = @stuff; for ($i = 0; $i < $num; $i++) {$line = $stuff[$i]; if (index($line,"e-mail") >= 0) {$line =~ s/\</ \ s $line="~" html of out email get to #need g;>/ /g; @words = split (' ' ,$line); $numwords = @words; for ($j = 0; $j < $numwords; $j++) {if (index($words[$j],"\@") >= 0) {$contact = $words[$j]; last;} } if (length($contact) > 0) {$contacts[0] = $contact; last;} } }# end for } #**************************** findcontact_ripe ****************** sub findcontact_ripe { @contacts = (); @netblocks = (); @stuff = (); $len = length($output); @stuff = split("\n",$output); #print $output,"\n"; $num = @stuff; for ($i = 0; $i < $num; $i++) {$line = $stuff[$i]; if (((substr($line,0,8) eq "remarks:") || (substr($line,0,7) eq "notify:")) && (index($line,"\@") >= 0)) {@words = split(' ',$line); $numwords = @words; for ($j = 0; $j < $numwords; $j++) {if (index($words[$j],"\@") >= 0) {$contact = $words[$j]; last;} } if (length($contact) > 0) { $contacts[0] = $contact; last;}# } } } #************************* findcontact_arin ********************** sub findcontact_arin { $len = length($output); @contacts = (); @netblocks = (); $numcontacts = 0; #if (index($output,'abuse') >= 0) $prevcontact = ""; $prevblock = ""; $output =~ s/\n/ /g; $output =~ s/\"/ /g; #makes it easier to parse @stuff = split(' ',$output); $num = @stuff; for ($i = 0; $i < $num; $i++) {$word = $stuff[$i]; if ($word eq 'Netblock:') {$block_start = ""; $block_end = ""; for ($j = $i + 1; $j < $num; $j++) {$jword = $stuff[$j]; $loc = index($jword,"queryinput\="); if ($loc >= 0) {if (length($block_start) == 0 ) {$block_start = substr($jword,$loc + 11);} else {if (length($block_end) == 0) {$block_end = substr($jword,$loc + 11);} } } } }#netblock seen if ($word eq 'Coordinator:') {for ($j = $i + 1; $j < $num; $j++) {$jword = $stuff[$j]; $loc = index($jword,"\@"); if ($loc > 0) #dont want at home {if ($prevcontact ne $jword && $prevblock ne $block_start . '-' . $block_end) {$contacts[$numcontacts] = $jword; $netblocks[$numcontacts++] = $block_start . '-' . $block_end; $prevcontact = $jword; $prevblock = $block_start . '-' . $block_end; } } } }#end coordinator } #end for all words } #****************************** getinfo ********************* sub getinfo { my ($ip) = @_; $ua = LWP::UserAgent->new; $whichserver = ""; $query = "http://www.arin.net/cgi-bin/whois.pl?queryinput=FULL\+" . $ip; $output = ""; $request = HTTP::Request->new('GET',$query); #note output is a global variable $response = $ua->request($request); if ($response->is_success) { $output = $response->content; $whichserver = "ARIN"; if (index($output,"Maintainer: RIPE") >= 0) {$whichserver = ""; #keep looking for more precise info $query = "http://www.ripe.net/cgi-bin/whois?query=$ip"; $request = HTTP::Request->new('GET',$query); $response = $ua->request($request); if ($response->is_success) { $output = $response->content; $whichserver = "RIPE"; } } elsif (index($output,"Maintainer: AP") >=0) { $query = "http://www.apnic.net/apnic-bin/whois2.pl?results=all&search=\+$ip"; $request = HTTP::Request->new('GET',$query); $response = $ua->request($request); if ($response->is_success) { $output = $response->content; $whichserver = "APNIC"; } } }

FLoP - Fast Logging Project for Snort

FLoP - Fast Logging Project for snort, version 1.5.1
----------------------------------------------------
The aim of the Fast Logging Project is to decouple the output to a database from the real snort process.

snort has to process the network packets to find alert signatures. If an alert is found the result has to be stored somewhere to process or simply store it.

The problem with the normal database output plugin is that snort is blocked until all data is written to the database. If there are any problems with the database then snort is hanging and probably some network packets get lost.

The FLoP project decouples this output and processing from snort. Further all alerts are spooled to a central server where they are feed in a database via unix sockets which is much faster than TCP/IP via any physical network. (Starting with version 1.5.1 a TCP connection is supported but should only be used on a loopback device.)

Since snort will write to an unix domain socket there is no blocking possible. (Of course you can lose some alerts if no process reads these alerts. But this is another story...)

Addition: There is still blocking possible if the reading process blocks. But this will never happen with sockserv since one thread is still waiting for input.

The program sockserv generates such an unix socket and reads all incoming alerts. The program consists of two threads, one reads the alert and buffers them in memory. The second take the alerts and spools them to the central server.

Here the program servsock reads these alerts. This program forks off a child process for each sockserv connection. These child processes consists of two threads. One thread receives the alerts and the second one feeds them to the database. Again a buffering takes place to overcome some slow downs due to too
many inserts in the database.

Finally to avoid a buffer overrun in form of a SIGSEGV there is a 'drop' feature implemented. This works with a high and low water mark, if more then high water alerts are stored in memory they are dropped to an unix socket until the low water mark is reached.

The drop program is able to receive these dropped alerts and will send an email consisting of a short message which alerts were dropped. (The only real thing missing is the payload.)

Additionally there is an alert feature. This is intended for sending email if alerts with a high priority arrive. This should inform an admin earlier then the periodic check of the database.

And last but not least there is a false positive generator to build network packets which are able to raise alerts within snort. (Of course there exist some similar programs but I needed one which is able to generate alerts with a high rate to check if all programs are able to deal with it or will break.

With a slight extension of the database it is possible to store the full network packet in the database. A program called getpacket is able to rebuild a pcap file from this data. This pcap file can then be analyzed with tools like ethereal

filtering snort alerts - Google Search POE::Filter::Snort - a POE stream filter that parses Snort logs into hashes

Rocco Caputo > POE-Filter-Snort-0.02 > POE::Filter::Snort

Download: POE-Filter-Snort-0.02.tar.gz

Annotate this POD

Module Version: 0.02 Source

• NAME
• SYNOPSIS
• DESCRIPTION
• SEE ALSO
• AUTHORS
• COPYRIGHT

POE::Filter::Snort - a POE stream filter that parses Snort logs into hashes

        #!/usr/bin/env perl

        use warnings; use strict;
        use POE qw(Filter::Snort Wheel::FollowTail);

        POE::Session->create(
                inline_states => {
                        _start  => \&start_log,
                        got_rec => \&display_record,
                },
        );

        POE::Kernel->run();
        exit;

        sub start_log {
                $_[HEAP]->{watcher} = POE::Wheel::FollowTail->new(
                        Filename   => "/var/log/snort/alert",
                        Filter     => POE::Filter::Snort->new(),
                        InputEvent => "got_rec",
                );
        }

        sub display_record {
                my $rec = $_[ARG0];

                print "Got a snort record:\n";
                print "\tComment: $rec->{comment}\n"  if exists $rec->{comment};
                print "\tClass  : $rec->{class}\n"    if exists $rec->{class};
                print "\tPrio   : $rec->{priority}\n" if exists $rec->{priority};
                if (exists $rec->{src_ip}) {
                        if (exists $rec->{src_port}) {
                                print "\tSource : $rec->{src_ip} $rec->{src_port}\n";
                                print "\tDest   : $rec->{dst_ip} $rec->{dst_port}\n";
                        }
                        else {
                                print "\tSource : $rec->{src_ip}\n";
                                print "\tDest   : $rec->{dst_ip}\n";
                        }

                        foreach my $xref (@{$rec->{xref}}) {
                                print "\tXref   : $xref\n";
                        }
                } 
        }

POE::Filter::Snort parses streams containing Snort alerts. Each alert is returned as a hash containing the following fields: comment, class, priority, src_ip, dst_ip, src_port, dst_port, xref, raw.

Most fields are optional. For example, some snort alerts don't contain a source and destination IP address. Those that do aren't always accompanied by a source and destination port.

The xref field refers to an array of URLs describing the alert in more detail. It will always exist, but it may be empty if no URLs appear in snort's configuration file.

The raw field is an arrayref containing each original line of the snort alert.

POE::Filter::Snort is written by Rocco Caputo. His CPAN ID is RCAPUTO. You can reach him by e-mail by his CPAN ID @cpan.org.

There is a very fine line between 'hobby' and 'mental illness.' Snort-Perl by Brian Caswell

I'm calling the latest version of the Snort+Perl patch snort-perl 1.0. For now, I am maintaining snort+perl as a separate distribution of Snort. The changes for Snort to integrate with perl on a wide scale are rather large. Keeping up with that diff gets painful. As such, I plan on syncing features from Snort "regular" to Snort+Perl on a semi-regular basis.

The 1.0 distro of "snort+perl" is available here.

[Dec 22, 2003] snortconfig 1.13 is out

Last updated at 2003/12/22 20:15 in Security :: IDS :: Snort

I've added a number of new features to the underlying modules which make rule validation much more useful. I have a PAUSE id now, so the underlying module (Net::Snort::Parser) is going to be uploaded to CPAN soon enough. I've been writing perl for nearly 7 years and I'm just now releasing the first library to the public that I'd like to see other people use.

Recommended Links

Softpanorama Recommended

Top articles

Sites

Unix Review Survey of Log Analysis Tools for Snort

In this section, I will introduce existing tools for analyzing Snort logs. I will categorize them according to the type of log they process. A brief introduction to the functions and a simple how-to will be described in each paragraph.

For Text-Based Logs

• Snort_stat Perl script -- I wrote this Perl script in 1999 because I needed a program to generate some statistical result from the Snort log files periodically. This script will generate the following information:
  • The count of source and destination pairs with the same signature
  • The count of signatures with the same source and destination pair
  • The count of attempts with the same source and signature
  • The count of destinations with the same signature
  • Signature distributions
  • Portscan and SPADE logs

  The script will periodically send such information to the administrator as an email or store it in an HTML format file. Adding the following entry to your crontab file can perform an automated process of executing this script:

  59 23 * * * root cat /your/snort/logfile | /your/snort_stat.pl -r | sendmail someone@somewhere

  Then, the script will parse /your/snort/logfile at 11:59 p.m. every day. Do the reverse DNS lookup, and send the result in email to "someone@somewhere." Using -h switch can generate HTML output. You can get this script through the Web page:

  http://xanadu.incident.org/snort/

  And an example of HTML output is at:

  http://xanadu.incident.org/snort/example.html

  The latest version is 1.15. I am in the process of updating it for Snort 1.8.
  

• SnortSnarf: SnortSnarf is a Perl program that will read your log files and generate HTML output. This lets you look into your Snort log to find possible problems. The output contains links to WHOIS lookup and DNS lookup. You can also choose to find out what attacks are associated with certain IP addresses.

  SnortSnarf comes with other functions besides log analysis. It contains nmap2html that will turn the result of a nmap scan into a HTML page. The latest addition is SISR (SnortSnarf Incident Storage and Reporting mechanism), which will help you to create a report for an incident and store that report or send the report by email efficiently.

  To install SnortSnarf, first download it from the Web site (the latest version is 041501.1):

  http://www.silicondefense.com/software/snortsnarf/index.htm

  Make sure you have the Perl module you need. The Time::JulianDay module is included in the tar-ball. Prepare a directory for the HTML output, and then issue the following command:

  snortsnarf.pl -d /where/is/html/output /var/log/messages /var/log/snort/alert

  This command will parse /var/log/messages and /var/log/snort/alert files, and then generate the HTML output in /where/is/html/output directory. There are other possible options listed in the snortsnarf.pl file.

  The example output of SnortSnarf can be found at:

  http://www.silicondefense.com/software/snortsnarf/example/index.html

  SnortSnarf has a problem with printing correct destination and source IP addresses for Snort 1.8 at this point.
  

• Snort-sort: Andrew R. Baker contributed this script that produces a sorted list of Snort alerts from a Snort alert format log. You can find the script under the contrib directory within the Snort distribution. This script can do reverse DNS lookups for you and also put a link on each IP address to do the whois lookup. This script currently does not handle 1.8 alerts well.

  To run it, simply use:

  snort-sort.pl /your/alert/file > result.html

  It will generate the HTML output in the result.html file.
  

• Snort2html: Daniel Swan wrote this Perl script to turn Snort alerts in your syslog files into HTML format, with links to Max Vision's ArchNIDS database, from which you can get more information about the signature. You can also click on the source host and look up its information in the ARIN database. The script does not perform any further analysis on the log. The script does not handle the new format of Snort 1.8 well at this point.

  To install it, simply change $logfile and $outputfile in the Perl script, then run it in your crontab every day or periodically by executing snort2html.pl. No command switch is needed.
  

• Snortlog.pl: Snortlog.pl was written by Angelos Karageorgiou, and performs similar function as Snort-sort and Snort2html. It converts a Snort alert file into HTML pages. The usage is:

  Snortlog.pl /var/log/snort/alert machinename

  Then snortlog will process /var/log/snort/alert and generate HTML pages. It could not handle the alert format from Snort 1.8. It can be found at: http://packetstorm.securify.com/sniffers/snort/snortlog.pl.
  

• ARIS Extractor: ARIS stands for "Attack Registry and Intelligence Service" and is a free service from SecurityFocus that provides a large-scale correlation of IDS and firewall logs. ARIS Extractor is the program to extract the information from your Snort log and send it in XML format to their ARIS database for further analysis. You must register with SecurityFocus first to obtain a username and password in order to send your data to them. The data will be sanitized to keep your privacy, though certain information will still be exposed in order for the correlation to happen. It is not meant for personal use, but to create a worldwide database that can be used to trend attacks and perform analysis on a macro level. More details are available at: http://aris.securityfocus.com/.

For libpcap Format Logs There are tools to analyze a tcpdump binary file--tcpreplay and tcpshow, for example. These tools can read the file and decode the packet to reassemble the network traffic captured in that file. None of these tools is specifically written or designed for Snort.

For Database

• ACID: ACID stands for "Analysis Console for Intrusion Databases." It is an application written in PHP by Roman Danyliw as part of the AIRCERT project. The latest version is 0.9.6b9, and the homepage is at:

  http://www.cert.org/kb/acid/

  ACID is capable of performing searches, grouping, maintaining, and plotting the data in the database. The data can be either Snort log/alert or log from some other firewall product; ipchains is the one currently supported. It displays various statistical data including:

  • Traffic profile by protocol
  • Unique alerts
  • Most recent/frequent alerts
  • Most frequent addresses

  ACID also provides a link to the WHOIS lookup for IP addresses. You can group alerts and delete alerts in the database. It runs well with Snort 1.8 beta 5. To install it, download the tarball from their Web site first. You also need ADODB for supporting multiple databases. If you want to enable the graphic plotting functions, you will need GD 1.8 and PHPlot 4.4.6. The order of the procedures should be:

  • Make sure your Web server supports PHP
  • Install ADODB, GD, and PHPlot
  • Install ACID

  In the document for ACID, set up the following variables in acid_conf.php:

  • $DBlib_path: Full path to the ADODB install. For example: /usr/local/www/php/adodb.
  • $ChartLib_path: Full path to the PHPlot install, if you want to use it.
  • $DBtype: Type of database used ("mysql", "postgres").
  • $alert_dbname: MySQL database name where the alerts are stored. For example, snort.
  • $alert_host: Host where the database is stored.
  • $alert_port: Port where the database is stored.
  • $alert_user: Username into the database.
  • $alert_password: Password for the username.

After setting up these variables, you can run acid_main.php from your Web browser to create some tables in the database to help optimize ACID's performance.

Figure 1 shows the main page of ACID.

Snortalog

Snortalog is a powerful Perl script that summarizes Snort logs, making it easy to view any network attacks detected by Snort. It can generate charts in HTML, PDF, and text output. It works with all versions of Snort, and can analyze logs in three formats: syslog, fast, and full snort alerts. Moreover, it is able to summarize other logs like CheckPoint Fw-1 (NG and 4.1), Netfilter, IPFilter, Packet Filter, CISCO PIX, and Lucent BRICK in a similar way.

Output options include ASCII text, HTML, and graphs (formatted in JPEG, GIF, or PNG). Snortalog is configured and managed from a GUI interface, and it runs on either Linux or Windows. It reads output from Snort in any format (no other tool that we've seen has this feature), including syslog, provides fast and full alerts, and then builds flat text or HTML summary reports. Snortalog's summary reports are similar to ACID's reports, but more compact.

http://jeremy.chartier.free.fr/snortalog/

SnortALog is a powerful perl script that summarizes snort logs making it easy to view any attacks against your network. SnortALog works with all versions of SNORT and is the only script who can analyze snort's logs in all formats (Syslog, Fast and Full alerts). Also, it is able to summarize Fw-1 (NG and 4.1), Netfilter and IPFilter logs in a similar way.

Current Capabilities:
* Create HTML, PDF and text reports
* Generate GIF, PNG or JPG graph in HTML output
* CLI (Command Line Interface) and GUI (Graphic User Interface
* Works with Syslog, Fast and Full SNORT alerts
* Works with all SNORT preprocessor (spp_stream4, spp_portscan, spp_decoder, flow, flow-portscan ...)
* Has the possibility to link the SNORT signature to the web reference attack description
* Works with "-I" Snort option to specify an interface and add report
* Work now with "-e" Snort option (Display the second layer header info)
* Use a specific plugin for generate your owns reference's SNORT rules
* Can specify order (ascending or descending)
* Can specify the number of occurrences to view
* Can resolve IP addresses and domains
* Add colors for a best visibility
* Possibility to do filtering (if you only want a specific IP source or high severity snort logs)
* Works with Fw-1 (4.1 and NG) in syslog and fw logexport command
* Works now with Fw-1 SmartDefense
* Works with Netfilter and IPFilter syslog logs
* Works now with syslog PIX log(Thanks to Edwin)
* Possibility to use DBM
* Works on Windows box (basic option: no graph)

Download Information:
To obtain the latest version of the tool see: http://jeremy.chartier.free.fr/snortalog/#download

SnortSnarf

Unix Review Survey of Log Analysis Tools for Snort

SnortSnarf: SnortSnarf is a Perl program that will read your log files and generate HTML output. This lets you look into your Snort log to find possible problems. The output contains links to WHOIS lookup and DNS lookup. You can also choose to find out what attacks are associated with certain IP addresses.

SnortSnarf comes with other functions besides log analysis. It contains nmap2html that will turn the result of a nmap scan into a HTML page. The latest addition is SISR (SnortSnarf Incident Storage and Reporting mechanism), which will help you to create a report for an incident and store that report or send the report by email efficiently.

To install SnortSnarf, first download it from the Web site (the latest version is 041501.1):

http://www.silicondefense.com/software/snortsnarf/index.htm

Make sure you have the Perl module you need.

• The Time::JulianDay module is included in the tarball. Prepare a directory for the HTML output, and then issue the following command:

Usage:

snortsnarf.pl -d /where/is/html/output /var/log/messages /var/log/snort/alert

This command will parse /var/log/messages and /var/log/snort/alert files, and then generate the HTML output in /where/is/html/output directory. There are other possible options listed in the snortsnarf.pl file.

The example output of SnortSnarf can be found at:

http://www.silicondefense.com/software/snortsnarf/example/index.html

Snort-sort

Unix Review Survey of Log Analysis Tools for Snort

Andrew R. Baker contributed this script that produces a sorted list of Snort alerts from a Snort alert format log. You can find the script under the contrib directory within the Snort distribution. This script can do reverse DNS lookups for you and also put a link on each IP address to do the whois lookup. This script currently does not handle 1.8 alerts well.

To run it, simply use:

snort-sort.pl /your/alert/file > result.html

It will generate the HTML output in the result.html file.

Snort2html

snort2html: Daniel Swan wrote this Perl script to turn Snort alerts in your syslog files into HTML format, with links to Max Vision's ArchNIDS database, from which you can get more information about the signature. You can also click on the source host and look up its information in the ARIN database. The script does not perform any further analysis on the log. The script does not handle the new format of Snort 1.8 well at this point.

To install it, simply change $logfile and $outputfile in the Perl script, then run it in your crontab every day or periodically by executing snort2html.pl. No command switch is needed.

Pig sentry

Protos Tools Pig Sentry

Pig Sentry is a lightweight script which is run against a Snort alert log. This tool is also available at snort.org. ACID, Snort Snarf and other reporting utilities are very nice for after the fact perusal, but not for up-to-the minute information. Pig Sentry is for real-time alerts, without getting spammed. It is intended for a high volume of alerts (the site it was implemented for, sees 200,000 to 300,000 alerts a day).

The way Pig Sentry works is it maintains a state table of recent alerts. It will send a notice if there is a new alert, or if there is an increase in the general trend or pattern of existing alerts. The trend notification is fairly configurable. It also expires alerts after there has been no activity with them, for a while.

Pig Sentry also checkpoints its state information to a file periodically and when it exits, and this same state store file is loaded when it starts up, so it can easilly be rolled into a lot rotation mechanism. The store file is not horribly readable, but is somewhat useful to peek at if you are interested to know the current profile of alerts on your site.

Tutorials

Scripting and Security (Part 1)

... we will be looking at PERL and how to use it to in an effort to help ease your life by leveraging the power of the PERL scripting language. The example shown will be but one small instance of what can be done with PERL, and the script will be small in size for ease of learning. Should you not have PERL installed on your computer then simply surf over to ActiveState and download the MSI package. Once installed you are good to go. Installation is a breeze as it is an MSI. The beauty of PERL is that it can also be used on other operating systems as well, as long as you have the PERL libraries installed.

What I have done is create a small file that contains the partial output of an "alert.ids" file from Snort. We will simply pick something in that file to pull out. In the instance used here we will be looking for all examples of the IP address 192.168.1.102 occurring in the input file called "articlefile". Much like I mentioned above, this is but a simplistic example of what can be done with PERL. That said though, it may be a small example but it is also a useful one. You may, at some point in the future, want to parse through some logs to see how often a certain IP address shows up. The script that I have pasted below will do that quite well, as long as you modify the relative paths to the input file to match your own file paths.

Behold the PERL script!

Pasted below is the fully functional PERL script that I wrote up to illustrate how handy PERL can be.

#!/usr/bin/perl -w
$pattern = "(192.168.1.102)";
open(INPUT, 'c:\articlefile') ||die "can't open the file called article_file: $1";
open(OUTPUT, '>c:\articleoutput') ||die "can't write to the file called script_output: $1";
while(<INPUT>) {
if ($_ =~(m/$pattern/)) {
print OUTPUT "$_\n";

}
}

close(INPUT);
close(OUTPUT);

Now I also pasted the test input file seen below I created called "articlefile" so that you can see exactly what is parsed out by the PERL script above.

[**] [1:2351:8] NETBIOS DCERPC ISystemActivator path overflow attempt little endian [**]

[Classification: Attempted Administrator Privilege Gain] [Priority: 1]

01/28-08:49:36.011482 192.168.1.102:1040 -> 192.168.1.101:135
TCP TTL:64 TOS:0x0 ID:31304 IpLen:20 DgmLen:1500 DF
***A**** Seq: 0xFFBD6980 Ack: 0x1071DF86 Win: 0x5B4 TcpLen: 32
TCP Options (3) => NOP NOP TS: 3937142 27317

[Xref => http://cgi.nessus.org/plugins/dump.php3?id=11808][Xref =>
http://www.microsoft.com/technet/security/bulletin/MS03-026.mspx]
[Xref => http://cve.mitre.org/cgi-bin/cvename.cgi?name=2003-0352][Xref =>
http://www.securityfocus.com/bid/8205]

[**] [1:2351:8] NETBIOS DCERPC ISystemActivator path overflow attempt little endian [**]

[Classification: Attempted Administrator Privilege Gain] [Priority: 1]

01/28-08:49:36.011482 192.168.1.103:1040 -> 192.168.1.101:135 #changed ip address here
TCP TTL:64 TOS:0x0 ID:31304 IpLen:20 DgmLen:1500 DF
***A**** Seq: 0xFFBD6980 Ack: 0x1071DF86 Win: 0x5B4 TcpLen: 32
TCP Options (3) => NOP NOP TS: 3937142 27317

[Xref => http://cgi.nessus.org/plugins/dump.php3?id=11808][Xref =>
http://www.microsoft.com/technet/security/bulletin/MS03-026.mspx][Xref =>
http://cve.mitre.org/cgi-bin/cvename.cgi?name=2003-0352][Xref =>
http://www.securityfocus.com/bid/8205]

Random Findings

Snort Simulator ASPN Mail Archive perl-win32-users

I have the following perl script that generates CheckPoint Alerts, and I want to transform it so it will generate ALL Types of Snort Alerts. Can someone guide me in the right direction ? I am a beginner in perl, and you are the experts.

Thanks in advance,

-Kamal.

#!/usr/bin/perl -w

my $TMPDIR = "/tmp";

use Socket;

my $nextArg = shift;
my ($host, $port, $targetRate);
if ($nextArg eq "-socket") {
$host = shift;
$port = shift;
$targetRate = shift;
} else {
$targetRate = $nextArg;
}

## Error injection: rates are in # / 100000
my $errRateBase = 100000;
my $ipErrorRate = 200;
my $actionErrRate = 100;
my $portErrRate = 100;
my $dateErrRate = 200;

$targetRate = 50 if ! defined $targetRate;
my $count = shift;
$count = 0 if ! defined $count;

my $rate = $targetRate;

my $recNum = intRand(675) * 10000;
my $rateCheck = 0;
my $maxRatio = 1.9; ## Do not have more than $maxRatio * $rate in any
bucket
my $maxPerSec = int($rate * $maxRatio);

my $SPREAD = 20;
my $BUCKETS = $SPREAD + 5;

my @str1 = ('udp', 'tcp', 'icmp');
my @months = ('Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug',
'Sept', 'Oct', 'Nov', 'Dec');

my @columns = (
'LogFileID',
'LogRecNum',
'time',
'action',
'orig',
'i/f_dir',
'i/f_name',
'proto',
'src',
'dst',
'service',
's_port',
'len',
'rule',
'has_accounting',
);

my $index1 = int(rand() * 1000000);
my $starttime = time;

sub event {
my $ref = shift;
my %data = %$ref;
my $res;
foreach my $key (@columns) {
my $val = $data{$key};
$val = "" if (! defined $val);
$res .= " " if defined ($res);
$res .= "$key=\"$val\"";
}
return $res;
}

sub intRand {
my $range = shift;
return int(rand() * $range);
}

sub needError {
my $errRate = shift;
return intRand($errRateBase) < $errRate;
}

sub badString {
return "bad'Str" if (intRand(20) < 3);
return "bad|S{tr" if (intRand(20) < 3);
return "ba}|S{tr" if (intRand(20) < 3);
return "ba}|S{tr" if (intRand(20) < 3);
return "ba}(S{tr" if (intRand(20) < 3);
return "ba`S{tr" if (intRand(20) < 3);
return "bacS{tr" . chr(3) if (intRand(20) < 3);
return "bacS{tr" . chr(203) if (intRand(20) < 3);
return chr(203) . "str" . chr(203) if (intRand(20) < 3);
return "a|b|'|c|d'::d{d}dd[f]s[]";
}

sub getIp {
my $num = shift;
my @a;
for (my $i=0; $i<4; $i++) {
$a[$i] = $num % 100;
$a[$i] = "0" . $a[$i] if (int($a[$i]) < 10);
$num = int($num / 100);
}
if (needError($ipErrorRate)) {
return "$a[3].$a[2]..$a[0]" if (intRand(10) < 5);
return "$a[3]:$a[2]:$a[1]:$a[0]" if (intRand(10) < 5);
return "$a[3]'$a[2]|$a[1]''$a[0]" if (intRand(10) < 5);
return "This is a test";
} else {
return "$a[3].$a[2].$a[1].$a[0]";
}
}

sub sendToStdout {
my $msg = shift;
printf "%s\n", $msg;
}

sub initSocket {
printf "Init socket $host -- $port\n";
my $hostname = shift;
my $lport = shift;
my ($a1, $a2, $prototype) = getprotobyname ("tcp");
if (0) {
my ($b1, $b2, $b3, $b4, $rawserver) = gethostbyname ($hostname);
my $serveraddr = pack ("Sna4x8", 2, $lport, $rawserver);
socket (SSOCKET, 2, 1, $prototype) || die ("No socket ($prototype)
$!");
printf "Binding\n";
bind (SSOCKET, $serveraddr) || die ("Can't bind");
printf "Listening\n";
listen (SSOCKET, 1) || die ("Can't listen");
printf "Accepting\n";
(my $clientaddr = accept (SOCKET, SSOCKET)) || die ("Can't accept:
$!");
printf "Selecting\n";
select (SOCKET);
$| = 1;
} else {
my $proto = getprotobyname('tcp');

socket(Server, PF_INET, SOCK_STREAM, $proto) || die "socket:
$!";
setsockopt(Server, SOL_SOCKET, SO_REUSEADDR,
pack("l", 1)) || die "setsockopt:
$!";
bind(Server, sockaddr_in($lport, INADDR_ANY)) || die "bind:
$!";
printf "Listening\n";
listen(Server,SOMAXCONN) || die "listen:
$!";
printf "Accepting\n";
my $paddr = accept(SOCKET,Server);
my($port,$iaddr) = sockaddr_in($paddr);
my $name = gethostbyaddr($iaddr,AF_INET);

printf "%s\n", "connection from $name [" .
inet_ntoa($iaddr) . "] at port $lport";

select(SOCKET);
$| = 1;
}
}

sub sendToSocket {
my $msg = shift;
print SOCKET ("$msg\n");
## printf STDOUT "%s\n", $msg;
}

sub sendEvent {
my $msg = shift;
if (defined $host) {
sendToSocket($msg);
} else {
sendToStdout($msg);
}
}

sub getDateStr {
my $year = 1997 + intRand(5);
my $mon = intRand(12);
my $day = intRand(28) + 1;

my $date = "";
$date .= " " if (int($day) < 10);
$date .= $day;
$date .= $months[$mon] . $year;
if (needError($dateErrRate)) {
return "Sunday Jan 15, 14:01" if (intRand(20) < 5);
return intRand(10) . $months[$mon] . $year;
}
return $date;
}

sub genOne {
my $index = shift;

my %data;
$data{'name'} = "test";

my $num = int(rand() * 100);
my $date = getDateStr();

$data{'LogFileID'} = '1044611703';
$data{'LogRecNum'} = $recNum++;
$data{'time'} = "$date";
$data{'action'} = (intRand(12) < 11) ? 'accept' : 'drop';
my @action = qw/
accept drop allow authcrypt
authorize ctl deauthorize decrypt
encrypt keyinst reject
/;

$data{action} = $action[intRand(scalar @action)];

if (needError($actionErrRate)) {
if (intRand(20) < 3) {
$data{'action'} = 'deny';
} elsif (intRand(20) < 5) {
$data{'action'} = ' accept ';
} elsif (intRand(20) < 5) {
$data{'action'} = 'a c c e p t';
} elsif (intRand(20) < 5) {
$data{'action'} = 'acc\'eptdeny';
} else {
$data{'action'} = badString();
}
}
$data{'orig'} = 'bedm-edn-650-pri';
$data{'i/f_dir'} = (intRand(6) < 4) ? 'inbound' : 'outbound';
$data{'i/f_name'} = "eth-s" . intRand(3) . 'p' . intRand(3) . 'c' .
intRand(4);
$data{'proto'} = intRand(3) < 2 ? 'udp' : 'tcp';
$data{'src'} = getIp($index);
$data{'dst'} = "172.16.0.65";
$data{'service'} = 21;
#$data{'service'} = intRand(2) + 10;
$data{'s_port'} = intRand(2);
if (needError($portErrRate)) {
if (intRand(20) < 5) {
$data{'s_port'} += 1000000;
} else {
$data{'s_port'} = badString();
}
}
$data{'len'} = intRand(44);
$data{'rule'} = intRand(20);
$data{'has_accounting'} = 0;

my $event = event(\%data);
return $event;
}

my $logFile = "$TMPDIR/perl_$$.log";
open (LOG, "> $logFile") or die ("cannot open logfile $logFile");
select(LOG);
$| = 1;
select(STDOUT);
$| = 1;

initSocket($host, $port) if (defined $host);

for (my $i=0; $i<$count or $count == 0; $i++) {
my $time = time();
my $timeIdx = $time % $BUCKETS;
$totals[$timeIdx]++;
$totals[($time+1) % $BUCKETS] = 0;
$totals[($time+2) % $BUCKETS] = 0;
my $spreadSecTotal = 0;
my @spreads = ();
my $num = 0;
for (my $j=0; $j<$SPREAD; $j++) {
my $idx = ($time - $j) % $BUCKETS;
if (defined $totals[$idx]) {
$spreadSecTotal += $totals[$idx];
$spreads[$j] = $totals[$idx];
$num++;
} else {
$spreads[$j] = 0;
}
}
my $lastOne = $spreads[0];
my $lastTwo = $spreads[1] + $lastOne;

my $needsToSleep = 0;
$needsToSleep = 1 if ($spreadSecTotal > = $num * $rate); ## There are
enough
$needsToSleep = 1 if ($lastOne > = $maxPerSec); ## Not too spikey
$needsToSleep = 1 if ($lastTwo > = ($maxPerSec + $rate));

if ($needsToSleep) {
my $sleepTime = 1 + (intRand(20) > 18 ? 1 : 0); ## Occasionally skip a
second
sleep $sleepTime;
if ($rateCheck) {
for (my $i=0; $i<$SPREAD; $i++) {
my $idx = ($time - $i) % $BUCKETS;
if (defined $totals[$idx]) {
printf("%d - ", $totals[$idx]);
}
}
printf("%.2f/sec\n", $spreadSecTotal / $num);
}
}
my $event = genOne($i);
if ($i % int($rate * 60) == 0) {
printf LOG "$i @ $time -- $event\n";
}
if ($timeIdx == 0 and $totals[$timeIdx] == 1) { ## Report the rates
every once in a while
printf LOG "$time: ";
for (my $i=0; $i<$SPREAD; $i++) {
my $idx = ($time - $i) % $BUCKETS;
if (defined $totals[$idx]) {
printf LOG ("%d - ", $totals[$idx]);
}
}
printf LOG ("%.2f/sec\n", $spreadSecTotal / $num);
}
sendEvent($event) if (!$rateCheck);
}

fwlogsum: A Perl script that produces digests of Checkpoint FireWall-1 logs. It parses other logfiles via converters.

loggrep: a tool for scanning and manipulating packet log entries from ipchains and iptables.

Pix2ss.pl is a Perl script that reads in PIX syslog denied connection logs and parses them into a file that can be read and processed by SnortSnarf. It's available at ActiveWorx and is supported by [email protected].

pixlog: a tool for summarizing PIX firewall traffic and keeping track of failed logins and attempts to access the PIX enable function.

pix-summarize -- Perl-based Cisco PIX log summarizer.

named-report v1.4 - BIND 9 log summary and report tool. It is written in Perl and is essentially a bunch of regex's comparing log messages fed to it and and those known by the program in order to create the various "Top 10 lists" in the report output. Contact John Kristoff for more information.

fromto-1.5.pl: Yet another Perl-based sendmail statistics summarizer.

sm.logger.pl : A Perl script that produces different details from a sendmail log, useful for determining the amount of mail usage per user.

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last updated: March 12, 2019