HAAGE&PARTNER Computer GmbH  HAAGE&PARTNER

Sawmill Analytics

Analyse und Reporting
für Web | Netzwerk | Sicherheit

Zugriffs- und Datenanalyse von Server-Logs (Proxy, Mailserver, Firewall, Webserver) und Überwachung der Sicherheit & Performance, Schwachstellenanalyse.

Sawmill Analytics 8 | Loganalyse

Sawmill-Tutorial

Using CFGA Files Incrementally Override CFG Files


Important Note: The feature described in this article was introduced in Sawmill 7.2.11, and will not work in earlier versions.

Sawmill uses its CFG format (CFG stands for "ConFiguration Group," as it is usually a group of configuration options or values) in most situations where it needs to store formatted, hierarchical, textual data on disk. These include profiles (in LogAnalysisInfo\profiles), log format plug-ins (in LogAnalysisInfo\log_formats); the files used to match spiders, search engines, and worms (spiders.cfg, worms.cfg, and search_engines.cfg, all in LogAnalysisInfo); preferences; language modules (LogAnalysisInfo\languages), and more.

Simple uses of Sawmill do not require direct editing or viewing of CFG files at all, but advanced uses often require editing profiles, creating or editing log format plug-ins, and performing other CFG file edits.

Editing CFG files works fine until you upgrade to a new release of Sawmill. At that point, if you've edited search_engines.cfg to add a new search engine to Sawmill's detection, you'll have a choice: keep your edited search_engines.cfg, or use the latest one. If you use the latest one, you'll get whatever is new in Sawmill; but you'll lose your own edits, and have to re-do them in the new version of search_engines.cfg. If you keep yours, you'll keep your edits, but you won't get anything that's new, including new search engines added, or bugs fixed, as part of the Sawmill upgrade.


CFGA Files Are Patches Applied To CFG Files

The solution is to use CFGA files, instead of editing CFG files. CFGA files are "ConFiguration Group Addition," so called because they add new information (or modify existing information) to a CFG file, from Sawmill's perspective. They allow you to edit the contents of a CFG file as Sawmill sees it, without actually editing the original text file. This lets you add a new search engine, for instance, without actually editing search_engines.cfg.

To use CFGA files, create a file next to a CFG file, with the same name except for the CFGA extension. In that file, use the same structure as the CFG file (the same internal groups), and any options you select will automatically be patched into the CFG file whenever Sawmill uses it.


Example: Adding A Search Engine To search_engines.cfg

For example, suppose search_engines.cfg looks like this (this is a simplified version of the usual search_engines.cfg, which contains many more entries):


   search_engines = {
 
    yahoo = {
      name = "Yahoo"
      substring = "yahoo."
      regexp = "yahoo\\.[^/]+/.*[&?]p=([^&]*)"
    } # yahoo
 
    lycos = {
      name = "Lycos"
      substring = "lycos.com"
      regexp = "lycos\\.[^/]+/.*[&?]query=([^&]*)"
    } # lycos

  
    google2 = {
      name = "Google"
      substring = "google."
      regexp = "google\\.[^/]*/.*[&?]q=([^&]*)"
    } # google
  
  } # search_engines


search_engines.cfg


Now, suppose your favorite search engine isn't there. You want to add MSN Search. You could do it by adding it to search_engines.cfg directly, like this:


   search_engines = {
 
    yahoo = {
      name = "Yahoo"
      substring = "yahoo."
      regexp = "yahoo\\.[^/]+/.*[&?]p=([^&]*)"
    } # yahoo
 
    lycos = {
      name = "Lycos"
      substring = "lycos.com"
      regexp = "lycos\\.[^/]+/.*[&?]query=([^&]*)"
    } # lycos

  
    google2 = {
      name = "Google"
      substring = "google."
      regexp = "google\\.[^/]*/.*[&?]q=([^&]*)"
    } # google

    msn_search = {
      name = "MSN Search"
      substring = "search.msn."
      regexp = "search\\.msn\\.[^/]*/.*[&?]q=([^&]*)"
    } # msn_search

  } # search_engines

search_engines.cfg (with MSN Search added)


That would work--you'd see "MSN Search" in your Search Engines report. But the next time you updated to the newest Sawmill, your change would be overwritten by the newest search_engines.cfg. A better solution is to create a file in LogAnalysisInfo (next to search_engines.cfg) called search_engines.cfga:


   search_engines = {
 

    msn_search = {
      name = "MSN Search"
      substring = "search.msn."
      regexp = "search\\.msn\\.[^/]*/.*[&?]q=([^&]*)"
    } # msn_search

  } # search_engines

search_engines.cfga


When Sawmill goes to look at search_engines.cfg, it will automatically add the information from search_engines.cfga, as it reads it, and the effect will be the same as if you had modified search_engines.cfg. On the next update, search_engines.cfg will be overwritten, but your modifications will not be lost, because they are in search_engines.cfga, which is not overwritten (because it is not part of the standard distribution of Sawmill).


Example 2: Adding A Log Filter To A Log Format Plug-in

Suppose you always want to discard spider traffic from your Apache logs. You could do this by adding a log filter like this one, each time you create a profile, to the log_filters section (log.filters):


  (... beginning of file omitted ...)
    reject_spiders = `if (spider ne '(not a spider)') then 'reject';`
  (
... end of file omitted ...)


Modification to profile CFG (partial)


But that would affect only that profile; it would have to be re-done for any future profiles. So a better solution is to add that filter to the apache_combined.cfg plug-in, in the same place as you would add it in the profile (log.filters). That would add the filter to all future profiles created for that log format. But the plug-in change would be overwritten when you upgrade to a new release of Sawmill, so you would have to re-do the plug-in edit after each upgrade. So the best solution is to create a new file, in LogAnalysis\log_formats, called apache_combined.cfga (next to the apache_combined.cfg log format plug-in file), which contains this:


  apache_combined = {
    log.filters.reject_spiders = `if (spider ne '(not a spider)') then 'reject';`
  } # apache_combined


apache_combined.cfga


This has the same effect as adding that line to apache_combined.cfg--it adds the filter to the bottom of the list of any new profile created from that plug-in--but when you upgrade to a new version, the plug-in change will not be overwritten. So this has to be done only once, and will add that filter to all future profiles for that format, for all future upgrades of Sawmill.


Other Uses Of CFGA Files

This can also be used to:
All these changes will survive upgrading, and can be moved to different installations to apply the changes there, without editing CFG files.


[Article revision v1.0]


Professionelle Dienstleistungen

Sollten Sie die Anpassung von Sawmill Analytics nicht selbst vornehmen wollen, können wir Ihnen dies als Dienstleisung anbieten. Unsere Experten setzen sich gerne mit Ihnen in Verbindung, um die Reports oder sonstige Aspekte von Sawmill Analytics an Ihre Gegebenheiten und Wünsche anzupassen. Kontakt

Zur Tutorial-Übersicht

Weitere Informationen

      Live-Demonstrationen »    
© 1995-2011 HAAGE & PARTNER Computer GmbH · Impressum · Datenschutz · www.haage-partner.de