HAAGE&PARTNER Computer GmbH  HAAGE&PARTNER

Sawmill Analytics

Analyse und Reporting
für Web | Netzwerk | Sicherheit

Zugriffs- und Datenanalyse von Server-Logs (Proxy, Mailserver, Firewall, Webserver) und Überwachung der Sicherheit & Performance, Schwachstellenanalyse.

Sawmill Analytics 8 | Loganalyse

Sawmill-Tutorial

Excluding Your Own Traffic With Log Filters


If you have a web site, you probably visit it yourself, frequently. You might want to ensure that it is functioning properly (through an automated system, or manually), or you might be looking for information on your own site, or there might be portions of your own site that are private and for your use only. When you analyze your web site with Sawmill, your own traffic will normally appear in the reports. If the purpose of the analysis is to determine how other people use your site, your own traffic can be a distraction, or can skew the statistics. Therefore, it is often best to exclude your own traffic, when doing an analysis with Sawmill, so the reports show only external traffic.

At first, it might seem that the way to do this is using a Report filter, by excluding your own IP address using the Filters icon at the top of the reports. That will work, but it is usually not the best way. Report filters do their work while the reports are being generated, after the logs have been imported into the database, and therefore slow down report generation. Reports with complex filters usually require a full table scan, which is a slower report generating method, whereas unfiltered reports can be generated much faster, because they can be generated from cross-reference tables. So if you intend always to discard your own traffic, you should do it with a Log Filter, so you can use unfiltered reports, and get better performance. Log Filters run while the log data is being imported, so they do not affect reporting speed; and a simple log filter doesn't hurt import speed much.

Note: because this technique requires Log Filters, this cannot be done with Sawmill Lite.


Using A Log Filter To Discard Your IP Address

For now, we'll assume that your traffic always comes from a particular IP address. Maybe it's the IP address of your firewall, or maybe it's the IP address of your own computer. The first step is to find out what IP address is yours. If you're behind a firewall, the IP address of your computer will not be the IP address that web servers see. If you're not sure, one way is to go to whatsmyip.com, which will display your IP address as it appears to web servers on the Internet. Let's assume that your IP address is 12.34.56.78. Now we'll create a log filter to reject all events from that IP. First, click View Config next to the profile name, in the Admin page, to go to the Config section of your profile: There, click Log Data and then Log Filters, in the left menu, to see the current list of log filters. Click New Log Filter, in the upper right of the log filters list, to create a new log filter, and type a name for it, like "Reject my IP address":

new log filter
Now, click the Filter tab, and click New Condition to set up the condition for the filter. The condition is met when the IP address is 12.34.56.78, so choose the IP address field from the Log field menu (this will vary depending on log format; it will be "Hostname" for Apache logs, and "Client IP" for IIS logs), choose "is equal" from the Operator menu, and enter 12.34.56.78 in the Value field:

condition IP equal

Click OK, and click Add Action to add the action this filter is to take when the condition is met. The action should be to reject this entry (we're rejecting all entries from your IP address), so choose "Reject log entry" from the Action menu:

action reject

Click OK. At this point, you could click Save and Close--the log filter is done. But for performance reasons, and to ensure no earlier filters explicitly accept the events you're trying to reject, it's best to have rejecting filters at the top of the list of log filters, so before you save this, move it to the top by clicking Sort Filters, and clicking Up [ + ] until the new filter is at the top:

sort
Now click Save and Close, and the log filter will be saved permanently to the profile.

If you view reports now, you won't see any change, because log filters only have an effect while importing log data. So it is now necessary to rebuild the database, which you can do by clicking Rebuild Database at the top of the Config page. After you rebuild the database, all reports will show only hits from IP addresses other than 12.34.56.78.


Advanced Topic: More Sophisticated Filters

The simple example above rejects only a single IP address. That is sufficient for some purposes, but if your internal traffic is not always from the same IP, you will need a more complex filter. Log Filters of any complexity can be created in the Log Filter Editor. For instance, if your internal traffic is from any of 12.34.56.76, 12.34.56.77 or 12.34.56.78, you can create a filter with three conditions (by clicking New Condition three times) to reject hits from all three IPs:

reject 3 ips

This same filter could be done using a regular expression, by choosing "Matches regular expression" from the Operator menu in the Condition page, creating a filter which uses a regular expression to determine which IP addresses to reject:

regexp filter
By customizing the regular expression, you can reject any class of IP address which can be described by regular expressions (which are very flexible).

The same filter could also be implemented with even more flexibility, by choosing "Advanced expression syntax" from the Filter Type menu. This allows you to enter an arbitrary Salang expression. Salang is the "Sawmill Language," and allows fully general programming language syntax, including conditions, loops, variables, subroutines, and more. It's overkill for this example, but is useful for complex ranges, and for conditions which involve more than just the IP address. The three-IP filter could be implemented as an IP range in Salang using this expression:
advanced expression



Professionelle Dienstleistungen

Sollten Sie die Anpassung von Sawmill Analytics nicht selbst vornehmen wollen, können wir Ihnen dies als Dienstleisung anbieten. Unsere Experten setzen sich gerne mit Ihnen in Verbindung, um die Reports oder sonstige Aspekte von Sawmill Analytics an Ihre Gegebenheiten und Wünsche anzupassen. Kontakt

Zur Tutorial-Übersicht

Weitere Informationen

      Live-Demonstrationen »    
© 1995-2011 HAAGE & PARTNER Computer GmbH · Impressum · Datenschutz · www.haage-partner.de