Configuring LoggerFS

As you may recall, there are only two configuration files for LoggerFS: logs.xml and schemas.xml. These configuration files are stored in /etc/loggerfs; the binaries are stored in /usr/local/bin.

Let's create these files from scratch for practice. Either delete or zero the existing configuration files:

# rm -f /etc/loggerfs/*
or
# echo "" > /etc/loggerfs/logs/xml; echo "" > /etc/loggerfs/schemas.xml



We'll start with logs.xml. RHEL5 stores Apache log files in /var/log/httpd and all Log paths in httpd.conf are relative to that path, so we'll set the log file location to simply access_log. Apache by default (at least in RHEL5) uses combined format, so we'll call our schema apache_combined. Since the apache server will need permission to write to these log files, we'll trust that the user apache and anyone in the apache group with write permissions by settings <permissions> to 0220. Obviously we're using MySQL for our LAMP server, so we'll say MySQL is listening on localhost and on the default port (3306). Finally, we set the database, username, and password to match what we created when setting up MySQL:


<logs>
     <log>
          <location>access_log</location>
          <uid>apache</uid>
          <gid>apache</gid>
          <permissions>0220</permissions>
          <schemas>apache_combined</schemas>
          <database-software>mysql</database-software>
          <database>myApacheLogs</database>
          <table>accessLogs</table>
          <server>localhost</server>
          <port>3306</port>
          <username>apachelogger</username>
          <password>s3cr3tp4zzw0rd</password>
     </log>
</logs>

Example 1: logs.xml for Apache Combined Format.


Onto schemas.xml. When creating this file, it's important to consider the data you're trying to log; for example, here's an apache line in Combined Format:

67.202.12.183 - - [07/Oct/2007:12:09:52 -0700] "GET / HTTP/1.0" 403 272 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"

First, we identify what parts of the line are important to us; what do we want to log? Then we write our regular expression out and give each match a column name. For our LAMP server, the following will suffice as a regular expression:

(\S+) (\S+) (\S+) \[.+\] "(.+)"+ (\S+) (\S+) "(\S+)" "(.+)"$

Now we'll name each match of the regular expression. These names will become the column names of the accessLogs table (specified above in logs.xml).

dst_ip,logname,authname,request,http_status,bytes,referer,user_agent

We've already named this schema apache_combined in the logs.xml, so we have everything we need to create our schemas.xml file:

<schemas>
  <schema>
    <name>apache_combined</name>
    <regex>(\S+) (\S+) (\S+) \[.+\] "(.+)"+ (\S+) (\S+) "(\S+)" "(.+)"$</regex>
    <columns>dst_ip,logname,authname,request,http_status,bytes,referer,user_agent</columns>
  </schema>
</schemas>

Example 2: schemas.xml for Apache Combined Format.

Post new comment

CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.
Copy the characters (respecting upper/lower case) from the image.