Syslog-NG Installation

Revision as of 11:41, 6 June 2008

The Syslog-NG, or System Log Next-Generation, service is an updated version of the default Syslog service found on FreeBSD and other UNIX and Unix-like systems (a paid-for version exists for Microsoft operating systems). Enhancements include the ability to filter content, various methods of storing information including separate files-per-device or MySQL databases and, a key feature, the addition of TCP for transportation of events (typically UDP is used on basic Syslog services).

The source code for Syslog-NG was used to create a commercial product maintained by | Balabit under the name Syslog-NG Premium Edition. This is a paid-for-support version of the free Syslog-NG code and revenue from it is used to fund development of the free edition.

Purpose

For further information on the default Syslog service installed on FreeBSD see the main Syslog article. The following is an explanation of the purpose of Syslog-NG specifically.

The purpose of installing and utilising Syslog-NG becomes apparent when you begin to monitor Syslog events from multiple servers (and other network-attached Syslog-capable devices) and require a centralised method in which to do so. Furthermore the ability to easily track, view and manage Syslog events are important. The enhancements made to Syslog-NG permit this functionality.

Examples

There are a number of reasons why upgrading to Syslog-NG is beneficial:

• a data-centre that houses numerous network devices all capable of sending Syslog event notifications;
• a medium-to-large organisation that has a wide-spread presence in different locations;
• fault detection of unmanned systems;
• other Syslog-capable devices that may not have simplified means to view events.

They all run along similar lines and many medium and larger organisations may benefit from utilising Syslog-NG somewhere on the corporate network.

Installation

The following is a guide to installing (or upgrading to) Syslog-NG on FreeBSD using the MySQL functionality. It also covers the installation and configuration of PHP-Syslog-NG. This is a web-based utility, essentially a set of PHP-based scripts for use within Apache to enable browser access to the Syslog-NG stored events.

Pre-requisites

For reference the following was undertaken by the author on a HP ML570 G2 server using FreeBSD 7.0. The particular release of FreeBSD should not matter too much. In addition the author selected Apache 2.2, MySQL 5.x and PHP 5.x and assigned a second (RAID5) drive entirely to an '/svr' volume (all regular FreeBSD volumes were allocated on the first (RAID1) drive).

For guidance on installing FreeBSD review this article. It is highly recommended, if using MySQL, to allocate either a larger than normal '/var' volume or create a separate volume to store the MySQL database in - for example a secondary drive allocated as '/svr'.

This installation guide will require the following ports to be installed:

• Apache - a full-featured web server, but any PHP-compatible web service should be acceptable;
• MySQL-Server - a full-featured database server;
• PHP - a popular scripting language used of web servers;
• PHP-Extensions - a set of extended features for the main PHP language;
• Syslog-NG - the updated Syslog service.

Ensure that the Apache module, MySQL and optionally GD and zlib dependencies are selected under the PHP port options page. The options page is displayed when running make from the ports system. This will install the 'mysql-client' automatically (not to be mistaken with the 'mysql-server' port).

It is recommended that the NTP service is configured to ensure accurate time is maintained against logged events.

The 'PHP-Syslog-NG' web-based utility is not available from the ports system however it can be downloaded from the Google Code website. Extract this into a new folder called '/svr/php-syslog-ng', for example:

# cd /svr
# fetch http://php-syslog-ng.googlecode.com/files/php-syslog-ng-<version>.tgz
# tar zxvf php-syslog-ng-<version>.tgz
# rm php-syslog-ng-<version>.tgz

Configuration

As with all services that run on FreeBSD they must first be enabled within the rc.conf file. Add the following lines as appropriate:

syslogd_enable="NO"
syslog_ng_enable="YES"
syslogd_program="/usr/local/sbin/syslog-ng"
syslogd_flags=""
mysql_enable="YES"
apache22_enable="YES"
syslog_pipe_enable="YES"

The 'syslog_pipe' entry is a script that is required if using the MySQL functionality and is therefore not part of the other 'syslog' entries.

MySQL

If utilising a larger, separate volume for the MySQL database store the following should be done:

# /usr/local/etc/rc.d/mysql-server stop
# cd /var/db
# mv mysql /svr
# ln -s /svr/mysql mysql

This has the effect of moving the physical 'mysql' database store from the default '/var' volume to the larger '/svr' volume and then creates a symbolic link to the original location. This 'trick' allows MySQL to function without needing to re-configure all of its data store locations individually. For more information on the concepts of symbolic links review this article.

If the MySQL directory does not exist first enable it (see above 'rc.conf' entries) and start MySQL as follows:

# /usr/local/etc/rc.d/mysql-server start

This will create the directory as part of the initial loading phase.

The MySQL server requires a password to be assigned to the 'root' user account (not to be confused with the FreeBSD 'root' user account) which is done using the following command:

# mysqladmin -u root password newpassword

Substitute the 'newpassword' part with a password of your choice.

PHP

The default installation of PHP creates two configuration files one of which requires copying as follows:

# cd /usr/local/etc
# cp php.ini-dist php.ini

The other configuration file is called 'php.ini-recommended' and is more secure for public-facing web server use. Either file can be copied as appropriate for the company security policy, etc.

Open the file 'php.ini' for editing and add the following entry highlighted in bold type:

; The file storage module creates files using mode 600 by default.
; You can change that by using
;
;     session.save_path = "N;MODE;/path"
;
; where MODE is the octal representation of the mode. Note that this
; does not overwrite the process's umask.
;session.save_path = "/tmp"
session.save_path = "/tmp"

; Whether to use cookies.
session.use_cookies = 1

Change the path location as appropriate.

Apache

The following configuration changes need to be made to Apache. In addition the installation of PHP does not automatically integrate itself into Apache. This is done by manually editing the Apache configuration files. Under Apache 2.2 this has changed from the way Apache 1.3 used to do it.

Under '/usr/loca/etc/apache22/' open 'httpd.conf' for editing and make the following changes highlighted in bold type:

Add the PHP Apache module:

LoadModule userdir_module libexec/apache22/mod_userdir.so
LoadModule alias_module libexec/apache22/mod_alias.so
LoadModule rewrite_module libexec/apache22/mod_rewrite.so
LoadModule php5_module        libexec/apache22/libphp5.so

Also add the default file name for PHP files:

#
# DirectoryIndex: sets the file that Apache will serve if a directory
# is requested.
#
<IfModule dir_module>
    DirectoryIndex index.html index.php
</IfModule>

Near the end of the file add:

NameVirtualHost *:80

Include etc/apache22/Includes/*.conf

Create a configuration file for what will become the PHP-Syslog-NG web interface called 'syslogweb.conf' in the '/usr/local/etc/apache22/Includes/' directory (note the capitalised letter 'i' in 'Includes') containing the following:

<VirtualHost *:80>
   ServerName syslog.example.com
   ServerAlias syslog
   ServerAdmin YourAdminName 

   DocumentRoot /svr/php-syslog-ng/html
   <Directory />
       Options FollowSymLinks
       AllowOverride All
   </Directory>
   <Directory /svr/php-syslog-ng/html>
       # pcw No directory listings
       # Options Indexes FollowSymLinks MultiViews
       Options -Indexes FollowSymLinks MultiViews
       AllowOverride All
       Order allow,deny
       allow from all
   </Directory>

   ErrorLog /var/log/php-syslog-ng-error.log

   # Possible values include: debug, info, notice, warn, error, crit,
   # alert, emerg.
   LogLevel warn

   CustomLog /var/log/php-syslog-ng-access.log combined
   ServerSignature On

</VirtualHost>

Taking note to change the various 'ServerXxx' entries as appropriate for your network.

Start (or restart) Apache to make the above changes take effect:

# apache start     (or restart)

Ensuring no error messages appear.

PHP-Syslog-NG

The PHP-Syslog-NG web-based utility requires write-access to its directory, which in turn uses the default user and group that Apache uses. This by default is www and www. The following sets the access rights to the PHP-Syslog-NG directory:

# cd /svr
# chown -R www:www php-syslog-ng

This sets the user and group ownership of the 'php-syslog-ng' directory, and the '-R' option (for "recursive") sets all sub-directories as well.

Providing all the above steps have successfully executed the PHP-Syslog-NG web front-end should be available to view. Open a web browser of choice and navigate to the DNS name assigned within the Apache web configuration, such as 'http://syslog.example.com'.

This should result in a page entitled 'pre-installation check' with a series of checks with successful test results in green and unsuccessful (either errors or incorrectly configured options) in red. Some red results may simply be optional components missing, such as the PHP GD component or more severe (as in 'show-stopping') problems with write-access to directories. Correct any issues that have been highlighted (remembering changes to Apache and PHP require an 'apachectl restart' command to be issued).

Click Next and accept the GNU/GPL license, then click Next once more.

The next page entitled 'MySQL database configuration' contains various fields which should be filled in as appropriate. The default passwords provided on the screen are simply 'sysloguser' for the 'sysloguser' user account and 'syslogadmin' for the 'syslogadmin' user account. The user names and passwords should be changed as required. Remember that the 'root' user and password refers to the MySQL user specified above.

Do not specify a 'MySQL Table Prefix', de-select the 'Install Sample Data' option. The 'Install CEMDB Data' option is only necessary if monitoring CISCO Syslog-enabled network equipment. Click Next to save and test these settings.

The next page entitled 'Enter the name of your Php-Syslog-NG site' simply requires a title to be specified. This could be the company name or other appropriate entry. Click Next to continue.

The next page entitled 'Confirm the site URL, path, admin e-mail and file/directory chmods' contains a number of options that have been determined by the installation process. These should largely be correct and require only an email address and password to be specified however specify the options as required. Click Next to continue.

The final page entitled 'Final Step: CEMDB Install' shows a hint to remove the installation directory and option to 'Install CEMDB' if it was selected during the installation process.

Syslog-NG Piping

The key component to joining the Syslog-NG service to the MySQL database store are a few scripts that continuously runs in the background.

This script is called '040.syslog-pipe' and is saved in '/usr/local/etc/rc.d' containing the following:

#!/bin/sh

# PROVIDE: syslog_pipe
# REQUIRE: DAEMON
# BEFORE:  LOGIN
# KEYWORD: nojail

# Code based on documentation found at:
# http://www.freebsd.org/doc/en_US.ISO8859-1/articles/rc-scripting/rcng-daemon-adv.html
#
# Written by DrModiford, www.modiford.com

.  /etc/rc.subr

name="syslog_pipe"
rcvar=`set_rcvar`

command="/usr/local/bin/mysql"
command_args="-u logger --password=l0gg3r syslog < /var/log/mysql.pipe 2>&1 &"

pidfile="/var/run/${name}.pid"

#load_rc_config $name
start_precmd="${name}_precmd"
stop_postcmd="echo Stopped ${name}."

#command="exec /usr/local/bin/syslog-pipe.sh 2>&1 &"
#command_interpreter="/bin/sh"

syslog_pipe_precmd()
{
if [ ! -e /var/log/mysql.pipe ]
then
        mkfifo /var/log/mysql.pipe
fi
return 0
} 

load_rc_config $name
run_rc_command "$1"

This script is called 'syslog-pipe.sh' and is saved in '/usr/local/bin/' containing the following:

#!/bin/sh

# Loop MySQL pipe function
while [ -e /var/log/mysql.pipe ]
do
        /usr/local/bin/mysql -u logger --password=l0gg3r syslog < /var/log/mysql
.pipe
done

Ensure the scripts are executable with the following commands:

# chmod 555 /usr/local/etc/rc.d/040.syslog-pipe
# chmod 555 /usr/local/bin/syslog-pipe.sh

The above contains the user name 'logger' and the password 'l0gg3r' which must be granted access to the syslog database on MySQL. This is done using the following commands:

# mysql -u root -p
Enter password:       (enter the MySQL root user password)

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 8
Server version: 5.0.51a FreeBSD port: mysql-server-5.0.51a

grant insert, select on syslog.* to logger@localhost identified by 'l0gg3r';

Query OK, 0 rows affected (0.01 sec)

mysql> exit

Final Configuration