Analyse und Reporting
für Web | Netzwerk | Sicherheit
Zugriffs- und Datenanalyse von Server-Logs (Proxy, Mailserver, Firewall, Webserver) und Überwachung der Sicherheit & Performance, Schwachstellenanalyse.
Sawmill Analytics 8 | Loganalyse
Sawmill-Tutorial
Using "Create Many Profiles" to Create and Maintain Many Similar Profiles
In large multi-user environments, like web hosting companies, Sawmill is used to provide one profile per customer. In other situations, Sawmill is used to manage one profile per server, or one profile per device. In these and similar situations, the Sawmill installation can have hundreds or thousands of profiles, all of them very similar. Creating all these profiles manually, using the Create Profile wizard, can be very time-consuming. When a change needs to be made to all of them, it can take a very long time if it is done separately for each profile through the Config interface.
The solution is to use Create Many Profiles, a feature of Sawmill which lets you create as many similar profiles as you like, in a single step. Create Many Profiles works in three stages: create a template profile, create the create_many_profiles.cfg file, and run the command to generate or regenerate the profiles.
Stage 1: Create a Template Profile
The first step is to create the template profile. For this example, we'll assume that we're creating only three profiles for the web sites, site1.com, site2.com and site3.com, on a Windows server where the logs are at C:\logs\site1, C:\logs\site2 and C:\logs\site3. The template profile is usually a separate profile from any of the final profiles, so we'll call it "template". Start by creating this "template" profile, and pointing it to the log data for site1.com. This first profile is created using the Create Profile wizard (the usual way, through the web interface). Enter C:\logs\site1 as the log source for the profile. This template profile won't actually process logs or generate reports, but it needs to see some log data so it knows what the format is, so all the format-related options can be propagated to the other profiles. Finish creating the profile, and call it "template." Do not build the database or view reports; this template exists only to be a model for other profiles.
Stage 2: Set up create_many_profiles.cfg
Now, using a text editor like Notepad, edit the file LogAnalysisInfo\miscellaneous\create_many_profiles.cfg . This is the file which describes the profiles you want to create from the template. In this case, it might look like this:
create_many_profiles = { template_profile_name = "template" profiles = { site1 = { changes = { label = "Site 1" log.source.0.pathname = "c:\\logs\\site1" } } # site1 site2 = { changes = { label = "Site 2" log.source.0.pathname = "c:\\logs\\site2" } } # site2 site3 = { changes = { label = "Site 3" log.source.0.pathname = "c:\\logs\\site3" } } # site3 } # profiles } # create_many_profiles |
The parts of this file are the following: First, the whole thing is enclosed in the create_many_profiles group, which just wraps the rest of the data into a CGF file.
create_many_profiles = { ... } # create_many_profiles |
The following line means that the profile called "template" (the internal profile name, as it appears when you look in LogAnalysisInfo\profiles, without the .cfg extension) is used as the template to create all the other profiles.
... template_profile_name = "template" ... |
Within the create_many_profiles section, there is a "profiles" section, which contains information about the profiles to create:
... profiles = { ... } # profiles ... |
And within the "profiles" section, there are three sections, one for each profile, like this one:
... site1 = { changes = { label = "Site 1" log.source.0.pathname = "c:\\logs\\site1" } } # site1 ... |
The section above (site1) means that the Create Many Profiles operation should create a profile whose internal name is "site1" (e.g., it will be site1.cfg in the profiles folder), and whose label is "Site 1" (i.e., it will appear as "Site 1" in the web interface). All settings will be copied from the template profile--it will be an exact clone of the template profile, except for the settings specified here, which are the internal name, the label, and the log.source.0.pathname option. That option (log.source.0.pathname) refers to the "pathname" option in the "0" group of the "source" group of the "log" group of the profile, which, for a profile with one log source, is the pathname of the log data. (If you look in the LogAnalysisInfo\profiles\templates.cfg file, you can see this structure, with the curly-bracketed "log" group containing the "source" group, etc.) So this will create a profile based on "template", but with internal name "site1", with label "Site 1", and which reads its log data from C:\logs\site1 (the single \'s must be replaced by \\'s within CFG file string options).
Similarly, the site2 and site3 sections create the Site 2 and Site 3 profiles.
Stage 3: Create the profiles
To create the profiles, you need to run the following command line on Windows. Open Command Prompt, use the "cd" command to change to the Sawmill installation directory, and run Sawmill with the "-dp templates.admin.profiles.create_many_profiles" option to create the profiles, like this:
cd C:\Program Files\Sawmill 7
SawmillCL -dp templates.admin.profiles.create_many_profiles
Or on non-Windows:
cd <sawmill directory>
./sawmill -dp templates.admin.profiles.create_many_profiles
When the command completes, all the specified profiles will have been created, just as though they had all been created separate using the Create Profile wizard. Now you can proceed with building databases, viewing reports, etc., for each profile.
Modifying all the profiles
Now maintenance of the profiles becomes very easy. Suppose you want to add a log filter to all profiles. Just open the "template" profile in Config, edit it to add the log filter, and go to Stage 3 again, above, to recreate the profiles. Recreating the profiles does not delete or rebuild the database, so it can be safely done at any time. It can even be scheduled using an external scheduler like cron or Windows Scheduler, to recreate all profiles every night, to pick up the previous day's changes to "template", or to create_many_profiles.cfg .
Adding another profile
Adding another profile is also very easy. Just add a new section (e.g., site4) to create_many_profiles.cfg, and repeat Stage 3 to recreate all profiles. It won't affect the other profiles, and it will create the new profile.
Automation
Because the Create Many Profiles feature uses a text file and the command line, it is very easy to automate from a scripting environment. Just have your script edit, or rewrite, the create_many_profiles.cfg file to list all profiles and their modifications, and have the script run the command line to regenerate all the profiles any time something changes.
Professionelle Dienstleistungen
Sollten Sie die Anpassung von Sawmill Analytics nicht selbst vornehmen wollen, können wir Ihnen dies als Dienstleisung anbieten. Unsere Experten setzen sich gerne mit Ihnen in Verbindung, um die Reports oder sonstige Aspekte von Sawmill Analytics an Ihre Gegebenheiten und Wünsche anzupassen. KontaktZur Tutorial-Übersicht