Print-friendly Version

Analyzing Snort Data With the Basic Analysis and Security Engine (BASE)

Amy Rich, October 2005

Abstract: This article describes storing Snort alert output in a MySQL database and using the web front end BASE to analyze the data.

Contents

• Introduction to BASE
• Installing and Configuring the Necessary Prerequisites
  • MySQL
  • Snort
  • PHP
  • ADOdb
  • PEAR Modules
• Installing and Configuring BASE
  • Downloading and Installing BASE
  • Configuring BASE
• Using BASE
  • Navigating the Main Screen
  • Creating Alert Groups
  • The Search Function
  • Generating Graphs
• Resources

In the article Introduction to Intrusion Detection With Snort, I covered basic concepts of intrusion detection and the installation and use of Snort, a network-based intrusion detection system (NIDS). In this article, I'll detail storing Snort alert output in a MySQL database and using the web front end BASE to analyze the data. BASE is the successor to ACID, the Analysis Console for Intrusion Databases, developed by Roman Danyliw at the CERT Coordination Center as a part of the AirCERT (Automated Incident Reporting) project. BASE is actively maintained and supported by a team of volunteers led by Kevin Johnson and Joel Esler.

Introduction to BASE, the Basic Analysis and Security Engine

BASE searches and processes databases containing security events logged by assorted network monitoring tools such as firewalls and IDS programs. BASE is written in the PHP programming language and displays information from a database in a user friendly web front end. When used with Snort, BASE reads both tcpdump binary log formats and Snort alert formats. Once data is logged and processed, BASE has the ability to graphically display both layer-3 and layer-4 packet information. It also generates graphs and statistics based on time, sensor, signature, protocol, IP address, TCP/UDP port, or classification. The BASE search interface can query based on alert meta information such as sensor, alert group, signature, classification, and detection time, as well as packet data such as source/destination addresses, ports, packet payload, or packet flags.

BASE allows for the easy management of alert data. The administrator can categorize data into alert groups, delete false positives or previously handled alerts, and archive and export alert data to an email address for administrative notification or further processing. Support for user logins and roles, allowing an administrator to control what is seen through the web interface, is also expected in an upcoming release of BASE. As of the current release of BASE (1.1.3), the hooks are there, but the code is not yet functional.

In the case we'll examine, Snort will log alert data to a MySQL database which will then be read by BASE and displayed via an Apache web server. BASE also supports other database back ends and can display information via any web server that supports PHP.

Installing and Configuring the Necessary Prerequisites

In order for BASE to function, we must first install and configure a back end database, in this case MySQL, to store the Snort alerts. In addition, we'll need Apache and Snort compiled with MySQL support. We also need to install PHP and a couple of PHP add-ons. ADOdb is an object-oriented PHP library used to interface to the database. You may already have some of these necessary tools on your system as part of the default distribution, depending on what version of the operating system you're running. The instructions below assume you are using the GNU tool chain (tar, make, gcc, and so on).

MySQL

We first start by obtaining and installing the MySQL package from MySQL. When unpacking, be sure to use GNU tar, since tar in the Solaris OS has issues with long file names. To avoid dependencies, we'll configure MySQL to build without libgcc and without zlib, but we'll still compile against openssl. (This assumes you've previously installed gcc and openssl.)

wget \
  http://dev.mysql.com/get/Downloads/MySQL-4.1/mysql-4.1.13.tar.gz/\
  from/http://mysql.mirrors.pair.com/

tar zxf mysql-4.1.13.tar.gz
cd mysql-4.1.13

LDFLAGS="-R/usr/local/lib" ./configure --prefix=/usr/local \
     --with-openssl \
     --without-docs \
     --without-libgcc \
     --with-named-z-libs=z
make
make install

If you run into issues compiling or installing MySQL, take a look at the Solaris OS section of the MySQL Reference Manual.

Snort

Now that we have MySQL installed, we can compile Snort with MySQL support. Slightly modify the installation directions from the previous article on Snort:

../configure --with-mysql=/usr/local --with-openssl=/usr/local

Then follow the rest of the installation instructions provided there.

Now set up the Snort database in MySQL. First create the snort user and grant the appropriate permissions:

mysqladmin -u root -p create snort

Next, run the MySQL script included in the Snort source directory to create the appropriate tables:

mysql -u root -p < snort-2.3.3/schemas/create_mysql snort

Now add the snort user and set the permissions:

mysql -u root -p snort

mysql> set PASSWORD FOR snort@localhost=PASSWORD('snort_user_password');
mysql> grant CREATE,INSERT,SELECT,DELETE,UPDATE on snort.* to snort@localhost;
mysql> flush privileges;
mysql> exit

Finally, edit the snort.conf file and modify the output plug-in:

output database: log, mysql, dbname=snort user=snort password=snort host=localhost
output database: alert, mysql, dbname=snort user=snort password=snort host=localhost

This will cause both log and alert data to be written to the database.

To verify that Snort is able to write to MySQL, make sure MySQL is running, then start Snort with the following options:

snort -c /etc/snort.conf -g snort

Once Snort and MySQL are running, wait a few moments until it collects some alert data. Then run the following command:

echo "SELECT count(*) FROM event" | mysql -u root -p snort

Your output should look similar to the following, where the number is the number of alerts you've received:

count(*)
1

If the number is zero, then you haven't seen any traffic that will trigger an alert, or you need to revisit your Snort/MySQL configurations.

PHP

This article assumes that you're running Apache as your web server, and that you've installed it with the GNU layout. If you're using a different web server or have installed Apache in a different location, these directions will need modification. First, download PHP from a nearby mirror. I've chosen us2.php.net:

wget http://us2.php.net/get/php-4.3.11.tar.gz/from/this/mirror

Now configure PHP to install into /usr/local/php and use apxs to add the libphp4.so module to Apache. The PHP configure lines below also tell PHP where to find MySQL, GNU gettext, OpenSSL, zlib, libjpeg, and libpng:

LDFLAGS="-R/usr/local/lib"  ./configure --prefix=/usr/local/php \
     --enable-memory-limit=yes \
     --with-apxs=/usr/local/sbin/apxs \
     --with-gettext=/usr/local \
     --with-exif \
     --without-mm \
     --with-mysql=/usr/local \
     --with-openssl=/usr/local \
     --with-zlib \
     --with-jpeg-dir=/usr/local \
     --with-png-dir=/usr/local \
     --with-exec-dir=/usr/local/php/libexec \
     --enable-cli \
     --enable-sockets
make
make install

In a production environment, you'll want to edit /usr/local/php/lib/php.ini and set the display_errors variable to off so that debugging messages will not be inlined in the HTML. If you prefer to have inline debugging messages, then it's recommended to at least set the error_reporting variable to E_ALL & ~E_NOTICE.

Obtain further information about PHP from the PHP web site, and further information about Apache from the Apache HTTP Server Project site.

ADOdb

ADOdb is a performance-conscious database abstraction layer for PHP. BASE requires ADOdb to talk to MySQL on the back end. First, obtain the source:

wget http://unc.dl.sourceforge.net/sourceforge/adodb/adodb465.tgz

Then unpack the source and place ADOdb where it can be accessed by BASE. The documentation recommends placing it in the Apache document root, but you can also configure BASE with ADOdb outside of Apache's tree (such as /usr/local/share/) if desired.

PEAR Modules

BASE documentation also recommends installing several PEAR modules. PEAR, the PHP Extension and Application Repository, is installed as part of PHP and is to PHP what CPAN is to Perl. If PEAR::Image_Graph is not already installed, obtain it by running the following commands:

/usr/local/php/bin/pear install Image_Color
/usr/local/php/bin/pear install Log
/usr/local/php/bin/pear install Numbers_Roman
/usr/local/php/bin/pear install http://pear.php.net/get/Numbers_Words-0.13.1.tgz
/usr/local/php/bin/pear install http://pear.php.net/get/Image_Graph-0.3.0dev4.tgz

Installing and Configuring BASE

Now that all of the prerequisites are in place, we can install and configure BASE itself.

Downloading and Installing BASE

First go to http://prdownloads.sourceforge.net/secureideas/base-1.1.3.tar.gz?download and pick a mirror from which to download the source code. Next, unpack the source tarball into your Apache DocumentRoot:

cd /usr/local/apache/htdocs
tar zxf /path/to/base-1.1.3.tar.gz
mv base-1.1.3 base

Use the supplied SQL script to create the BASE database:

mysql -u root -p < base/sql/create_base_tbls_mysql.sql snort

If you're using a database other than MySQL or upgrading to BASE from ACID, there are different scripts available in the base/sql directory.

Configuring BASE

Once you create the database, configure BASE by copying the base_conf.php.dist file to base_conf.php and customizing it to fit your environment:

cd base
cp base_conf.php.dist base_conf.php

Options in the config file are all well commented, but those listed in the table below are the minimum that must be set.

Until the authentication portion of BASE is working properly, protect the directory where you installed BASE. Apache can be configured to deny access based on IP address, as well as to require a user to enter a password. Modify /usr/local/apache/etc/httpd.conf and add something like the following to allow users from the host 192.168.1.100 to authenticate:

<Directory /usr/local/apache/htdocs/base/> 
Order Deny, Allow
Deny from All
Allow from 192.168.1.100
AuthType Basic
AuthName Access is restricted.
AuthUserFile /path/to/htpasswd/file
require valid-user
</Directory> 

Populate the .htpasswd file with username and encrypted password data. Please refer to the documentation on the Apache web site for more help on configuring access restriction.

Using BASE

You should now have a functional BASE install accessible at http://www.your.domain/base, and you're ready to begin using the GUI to view and manage alerts.

Navigating the Main Screen

Once you log in, the main page shows a summary of currently logged alerts as well as various alert summary breakdowns and links to graphs (see Figure 1).

Figure 1: Main Page of BASE
(Click to Enlarge)

Drilling down into any of the summaries will present a list of events. Depending on the list, it is possible to drill further down and gain more detail. For example, following the link Today's alerts: unique, brings up a new screen with a summary of alerts which begin at the previous midnight. A link labeled snort, located to the left of each signature, attempts to connect to the signature database at the Snort web site and provide more detailed information about that particular signature.

Drilling down on a source or destination IP address on any of the screens brings up a summary that includes how many times that IP was logged as a source or destination address. It also indicates the first and last time the IP was logged. Additionally, the summary page contains links to external web-based tools that provide DNS and Whois lookup services.

Drilling down on the source or destination port's links displays a summary of ports, number of occurrences, time first seen and time last seen. Each listed port number is a hyperlink to the SANS Internet Storm Center page for that port number.

Creating Alert Groups

Alert groups can be created to group event information into user-defined categories for easy perusal. In order to create a new alert group or modify existing groups, click on the Alert Group Maintenance link at the bottom of the main page.

Next, click the Create link and fill out the name and description fields for the new group. For this example I'll create an alert group named test2 based on an alert signature. To do so, I return to the main page and select the Unique alerts link, then decide to use the signature named IIS UNICODE CODEPOINT ENCODING.

I check the box next to that signature, then scroll to Action box at the bottom of the page. From the drop down menu labeled {action} I select the option to ADD to AG by Name, type in test2, and click on the Selected button. Returning to the Alert Group Maintenance screen I see that the group test2 now shows two alerts.

The Search Function

BASE has a search function that can be used to quickly search through the database for certain criteria and present it in an ordered fashion.

Figure 2: Search Function in BASE
(Click to Enlarge)

The allowable search criteria include Alert Group, Signature, and Alert Time. The results can be ordered by timestamp, signature, source IP, or destination IP. Unfortunately, there is no option to use an IP address as one of the criteria.

Generating Graphs

Graphs can be created from Alert Data or Alert Detection Time.

The Alert Data can be graphed and charted based on a variety of options to create easily readable reports. Figure 3 below shows a screen shot of a simple pie chart.

Figure 3: Pie Graph of Time vs. Number of Alerts
(Click to Enlarge)

This next screen shot shows a bar graph based on Alert Detection Time which can be used to identify periods of heavy activity.

Figure 4: Bar Graph of Time vs. Number of Alerts
(Click to Enlarge)

These charts and graphs allow the system administrator to visually pinpoint periods of attacks. The images created by BASE are also a valuable resource for inclusion in managerial reports and departmental presentations dealing with site security.

Resources

BASE Resources

• BASE main page
• BASE FAQ
• BASE forums
• secureideas-base-devel mailing list
• secureideas-base-users mailing list

Resources for Additional Software

• MySQL main site and reference manual
• PHP
• PEAR
• Snort
• Apache HTTP Server Project
• ADOdb

Other Resources

• Introduction to Intrusion Detection With Snort
• AirCERT project
• SANS Internet Storm Center

Unless otherwise licensed, code in all technical manuals herein (including articles, FAQs, samples) is provided under this License.