# $Id: README,v 1.49 2003/08/05 14:13:49 andreaso Exp $ #

README for Oinkmaster v0.8
~~~~~~~~~~~~~~~~~~~~~~~~~~

Written by Andreas stling <andreaso@it.su.se>.
Feedback is of course appreciated.

Homepage:
http://nitzer.dhs.org/oinkmaster/

Browsable CVS repository:
http://devel.it.su.se/cgi-bin/local/cvsweb.cgi/oinkmaster/

Or to do a CVS checkout via command line, set the CVSROOT environment
variable to ":pserver:cvs@cvs.it.su.se:/afs/su.se/services/cvs/public/cvsroot"
and then do "cvs login" with "cvs" as password. Then you can use the 
regular CVS commands, like "cvs co oinkmaster" (remember that this will 
get you the current development snapshot).



Intro
~~~~~
Oinkmaster is simple Perl script released under the BSD license to help you
update your Snort 2.0+ rules and comment out the unwanted ones after each
update. It also has a few other useful features regarding rules management. 
Oinkmaster will tell you exactly what had changed since the last update, 
hence giving you good control of your rules. It is most often used to 
update the official rules from www.snort.org, but can just as well be used 
for managing/distributing your homemade rules. Oinkmaster may be useful 
in concuntion with any program that can use Snort rules, like Snort (doh!)
or Prelude-NIDS.

It's a variant of the (now extremely obsolete) arachNIDS snort rules updater
(http://nitzer.dhs.org/arachnids_upd/) which was pretty much the same
thing but for the arachNIDS Snort rules exports at http://www.whitehats.com/,
which have not been maintained for a very long time.

Use Oinkmaster with care and at your own risk.
Check the INSTALL file for quick installation instructions.

Check out http://www.snort.org/ for more information about Snort and its
rules.



Requirements
~~~~~~~~~~~~
It should work on most UNIX-like systems containing Perl 5, tar, gzip and 
wget (ftp://ftp.gnu.org/gnu/wget/).
So far it has been successfully tested on OpenBSD, FreeBSD, NetBSD, Linux,
AIX, Solaris and Mac OS X. You can also run it under Windows with
either Cygwin or ActivePerl (see README.win32 for more information about 
that). Let me know if you manage to run it on an unlisted platform.

Again, use it at your own risk!



What problem does Oinkmaster solve?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We've all been there - we download and use the Snort rules and realize
that many of them aren't suitable for our environment.
So we disable (or modify) the ones we don't like, and everything is fine.
Since we always want to run the latest and greates rules, we download
the new rules from www.snort.org as soon as they have been updated.
Oops - all our customized rules are now overwritten, and we have 
to do it all over again. This is where Oinkmaster comes in - it will 
automatically do those boring modifications to the rules that you
would usually have to do manually after each update.



How it works
~~~~~~~~~~~~
It fetches the Snort rules archive from the specified URL, unpacks it, 
looks for a directory called "rules" in it, and comments out/modifies 
those rules as specified in the Oinkmaster configuration file.
It then compares all those rules to the ones in your local rules directory 
and prints possible changes. Your local rules are then overwritten by the 
new ones.

Unwanted rules are specified in the file oinkmaster.conf. In this file you 
can also tell Oinkmaster to skip entire files that you don't care about 
(i.e. don't want to update or check for changes), or that some rules 
should be modified after each update etc. See oinkmaster.conf for more 
information. 

This script should probably be run manually but can also be run as a cron job.
But of course, be very careful when doing the latter. Think about what you're
doing. It usually doesn't take much for these automatic updating scripts
to fail, so things could easily get messed up from one update to another
because of different layout in the rules archive, typo in a rule, bugs in
the script etc etc (at least run snort -T on the new rules before restarting
your running Snort). And as always when updating the Snort rules, there 
may be some changes that you really don't like (or at least want to know 
about before using), so you should always manually verify the new rules 
before using them. Since Oinkmaster will print all changes in the new 
rules to the screen (in an easy-to-read format), this becomes a fairly 
simple task. 



Usage information
~~~~~~~~~~~~~~~~~
Try ./oinkmaster.pl -h for more usage information and available options.

As a first test, you can create an empty rules directory, for example
/tmp/rules/. Then try executing "oinkmaster.pl -o /tmp/rules".
Since your /tmp/rules/ directory is empty, all rules files in the downloaded
archive will be regarded as added, and copied to that directory.
Then try "oinkmaster.pl -o /tmp/rules" one more time.
This time the files in the downloaded archive will be compared to the ones
in /tmp/rules/ and Oinkmaster will tell you if something has been
updated.

If you already have several rules commented out (or removed) in your current
local rules, you need to add the SIDs of those to oinkmaster.conf manually 
(or check out the contrib directory for a help script) before running 
Oinkmaster, or they will be re-enabled after each update. 

You disable rules by adding "disablesid x" to the file oinkmaster.conf, where
x is the Snort rule ID (the "sid") of the rule in question.
So if you want the rule with sid 12345 to be commented out after each update,
you just append the line "disablesid 12345" to oinkmaster.conf.
When you update the rules the next time, this rule will be commented out
and Oinkmaster will notify you about that change.

You can also add entire files to be totally ignored by adding
"skipfile filename" options to oinkmaster.conf where "filename" is a file
in the distribution archive you don't care about at all. These files will
not be checked for changes and they will not be added or updated. For
example if you don't include the file icmp-info.rules from your snort.conf
and don't care about keeping it up to date and don't want to be notified
of changes in it, you can to add the line "skipfile icmp-info.rules" to
oinkmaster.conf. Although it may be a good idea to track changes even for
rules files you don't use (i.e. that you don't include from your snort.conf).
Who knows, you might find something interesting some day.

The skilled/brave/stupid people can also use the "modifysid" keyword in
oinkmaster.conf to make arbitrary changes to certain rules after each download.
This is an excellent way to screw up your rules, so don't use it unless you
really have to. See oinkmaster.conf for more information and examples.



Usage examples
~~~~~~~~~~~~~~
To automagically update the rules every night at 02:30 and make backup
of the old ones into /snort/backup/ if there were any updates, and sending
difference notification to syslog, you could use something like this
in your crontab (always consider adding the -c flag (careful mode) when
running as a cron job though):

30 2 * * * oinkmaster.pl -o /snort/snort.org-rules/ -b /snort/backup 2>&1 | logger -t oinkmaster

When you check your syslog the next time and see that a new rule has been
added that you don't want to use, just add its SID to oinkmaster.conf and
it will always be commented out in future updates. Or if a rule you have
disabled has changed and you now decide that you want to use it again,
just remove its SID from oinkmaster.conf and it will be reactivated in
the next update. If you want the output to be sent to you in a mail
instead, you could use something like:
oinkmaster.pl ... 2>&1 | mail -s "subject" you@somewhere.foo

If you just want to check for changes in the rules but not actually update
your existing rules, you can use the -c flag for "careful" mode.
It's nice to combine -c with -q and run as a cron job to silently
check for updates (perhaps you just want to use this script to check for
updates but still want to do the actual rules updating manually).
To silently check for updates as a cron job and not sending any e-mail unless
there were updates available, you could use something like this:

30 2 * * * TMP=`mktemp /tmp/oinkmaster.XXXXXX` && (oinkmaster.pl -o /snort/snort.org-rules/ -q -c > $TMP 2>&1; if [ -s $TMP ]; then mail -s "subject" you@somewhere.foo < $TMP; fi; rm $TMP)

This example only works on systems with the mktemp command but could easily be
rewritten for others. You get the point. The important thing here is to use the
-q argument (quiet mode) so Oinkmaster doesn't give any output unless there
were updates available or if there were warning/error messages.

You should of course change directories and subject/e-mail addresses in the
examples above as appropriate.



Oinkmaster output
~~~~~~~~~~~~~~~~~
If there were any changes, Oinkmaster will tell you about them.
Here are what the different changes in the rules mean:

o Added:
  - New rule (the SID did not exist in the old rules file).

o Enabled:
  - The rule (with this SID) was commented out in the old rules file, but is
    now activated (probably because the SID was removed from oinkmaster.conf).

o Enabled and modified:
  -  The rule (with this SID) was commented out in the old rules file, but
     is now activated. The actual rule had also been modified.

o Removed:
  - The rule (with this SID) does no longer exist in the new rules file.

o Disabled:
  - The rule (with this SID) still exists but has now been commented out
    (probably because the SID was added to oinkmaster.conf).

o Disabled and modified:
  - The rule (with this SID) still exists but has now been commented out.
    The actual rule had also been modified.

o Modified active:
  - The rule (with this SID) has been modified and is an active rule.

o Modified inactive:
  - The rule (with this SID) has been modified but is currently an
    inactive (commented out) rule.


Here comes some typical example output.
We see that a rule has been removed from web-cgi.rules and that a rule
in dos.rules has been modified.



[***] Results from Oinkmaster started Tue Dec 25 23:36:07 2001 [***]

[*] Rules modifications: [*]

  [---]         Removed rules:        [---]

 -> File: web-cgi.rules:
    alert tcp $EXTERNAL_NET any -> $HTTP_SERVERS 80 (msg:"WEB-CGI infosearch fname"; flags: A+; uricontent:
"fname=|7c|";reference:arachnids,290;classtype:attempted-recon; sid:822; rev:1;)

  [///]          Modified active rules:         [///]

 -> File: dos.rules:
    Old: alert tcp $EXTERNAL_NET any -> $HOME_NET 7070 (msg:"DOS Real Server template.html"; flags: A+;
content:"/viewsource/template.html?"; nocase;reference:bugtraq,1288; classtype:attempted-dos; sid:277; rev:1;)
    New: alert tcp $EXTERNAL_NET any -> $HOME_NET 7070 (msg:"DOS Real Server template.html"; flags: A+;
content:"/viewsource/template.html?"; nocase; reference:cve,CVE-2000-0474; reference:bugtraq,1288;
classtype:attempted-dos; sid:277; rev:2;)

[*] Non-rule modifications: [*]
    None.

[*] Added files: [*]
    None.



If there were any added files (i.e. files that have not been included in
the rules archive before and that have now been added to your output directory),
you should probably consider including them from your Snort configuration
file. If there were any files removed from the archive, you should probably
consider excluding them from your Snort configuration file (you must use -r
to check for removed files).
Note that if a SID is moved from one file to another, it will be regarded
as removed from the first one and added to the new one.
This is good, because if a rule we like is moved from file A to file B
and we only include file A from our snort.conf, we sure would like to be
informed of that.



Misc important notes
~~~~~~~~~~~~~~~~~~~~
o It's a really good idea to have a separate directory for your snort.org rules
  on your snort machine. Why? Because if you have a local rules file called
  "coolstuff.rules" in the output directory and a file called "coolstuff.rules"
  is one day added to the rules distribution archive, your local file will be
  overwritten by that one. Same thing goes for snort.conf - we don't want 
  our snort.conf to be overwritten after each update. It's a good idea to keep
  an extra (unused) copy of snort.conf in the rules directory but specify 
  another snort.conf when starting Snort. This way you can keep track of 
  changes in the snort.conf from the rules archive while keeping your 
  customized copy intact. If there are added/removed rules files in the 
  archive or added/modified variables in snort.conf, you will have to edit 
  your production copy of snort.conf manually and update as needed. 
  Oinkmaster does not do that kind of stuff. Yes, this means that if 
  there is a new variable added the distribution snort.conf and that is used
  in the new rules, your new rules will not even load until you define that
  variable in your snort.conf manually. 
  See, auto-updating with auto-restart is bad unless you're careful.

  As of oinkmaster 0.8, you can run with -U <file> though, which means that
  Oinkmaster will search snort.conf in the downloaded rules archive for all
  variables (the "var somevar something" lines) and then verify against 
  <file> (which probably is your local production copy of snort.conf). 
  Variables that exist in the distribution snort.conf but not in <file> 
  will be added to <file> directly after any other variables it may contain. 
  Variables that have been modified are NOT updated, the -U argument only 
  cares about totally new ones. This is of course not guaranteed to solve 
  the problem, but it may be convenient on home networks and such if you 
  think it's a good idea.

o Oinkmaster also never deletes any rules files from your system.
  So if a file called foo.rules is usually included in the rules distribution
  archive but is one day removed from it, the old foo.rules will still be left
  in your rules directory. Oinkmaster will notify you of this but only if you
  use the "-r" argument, which means that each interesting file
  (*.rules, *.conf ...) in your output directory will be checked to see if it
  also exists in the downloaded rules archive. If it doesn't, it will be
  regarded as removed from the archive and a message will be printed because
  you may want to edit your Snort configuration file to exclude this file and
  possibly remove it from your system as well.

  So if you use a separate directory for your Snort rules from snort.org
  (i.e. you have possible local rules files in another directory) you
  probably want to use the "-r" argument. Otherwise it will be very
  missleading since your local rules files obviously don't exist in the
  rules distribution, and will therefore be regarded as removed every time.
  Note that the rules in your output directory will be overwritten after each
  update (if there were any changes, that is) which means that all possible
  local modifications will be lost. Therefore it's better to move rules you
  need to modify (other than just enabling/disabling) to some local file
  in another directory instead. That's my opinion anyway.

o Don't depend too much on this script for the rules updating process.
  It's just intended to be a small help and you should always keep an eye
  on it. Manual reviewing of all your rules and configuration should always be
  done every now and then.
