Sawmill Analytics 7

Übersicht
Neuigkeiten
Informationen
Versionen
Referenzen
Pressestimmen
Download
Support
Supportforum
Preise | Lizenzen

Support



New features in 6.4.0:

NOTE: These are *all* features added since 6.3.16; i.e. they are all the new 6.4 features.

  1. Added "don't autodetect log format" checkbox to configuration creator, so if the autodetector is getting it wrong, it can be overridden.
  2. Added "numerical ascending" and "numerical descending" table sorts.
  3. Added "reverse alphabetical" as a statistics table sort order.
  4. Added a "safe update" feature (on by default) which backs up the database before updating it. This all but eliminates the possibility of an update corrupting the database on an error, but it requires twice the disk space and some additional time for an update.
  5. Added a feature to save the contents of the "Create/Update Many Configurations" text box, so when you go back, it's how you last submitted it.
  6. Added a GeoIP database to Sawmill, so it can compute countries from IP addresses. This used to include a "Top countries" view in most configurations.
  7. Added a new "paths through a page" view, which shows all paths through a particular page by showing the immediate predecessors and successors in the clickstream.
  8. Added a new "Support email address" field in the Preferences (Miscellaneous tab) which specifies the address where bug reports should be sent. This should be set by administrators of multi-user installations, because the bug reports are usually configuration issues that can be corrected by the Sawmill administrator, and cannot be corrected by us (the software vendor), since we don't have access to the installation. By default, support mail will continue to come to us until this option is changed.
  9. Added a new concatenate2 log filter type, which puts the result in field2 instead of field1.
  10. Added a new option to make graph bars non-clickable (by default, they are clickable). This is useful in cases where there are so many bars that the overheard of making them clickable bogs down the network, Sawmill server, and/or browser.
  11. Added a new type of session filter, which lets you zoom in on particular session using its ID.
  12. Added a number_thousands_separator option, which is a comma (,) by default, and which is used to separate thousands in large numbers.
  13. Added a recentminutes:N filter that shows hits in the past N minutes.
  14. Added a WebTrends Extended Log Format plug-in that can work with any syslog.
  15. Added an "allow viewers to rebuild" option. When this option is checked, there will be rebuild/update links in the info bar of the configuration, even if the user isn't administrator, and the user will be able to manually update/rebuild the database by clicking the links. No other administrative operations will be available.
  16. Added an "Average bytes transferred per second" item to the Overview.
  17. Added an "expand all" link to the "paths through the site" toolbar, to fully expand the entire paths tree.
  18. Added an "extra options" field to Scheduler items, so arbitrary command line options can be sent to Sawmill as part of the scheduled command line task. This allows powerful options like Scheduling different views to be emailed to different people, with different filters, which used to require the use of an external scheduler.
  19. Added an "If A, then do B followed by C" filter type.
  20. Added an accept_collected_entry_regexp_carryover filter, which works like accept_collected_entry_regexp except that it carries the collected values over rather than resetting them. So a single field value can be collected early on, and later acceptances will all use that value, even if it does not appear again.
  21. Added autodetection of RADIUS format strings, so RADIUS logs with arbitrary fields are now supported.
  22. Added checkbox to view/subview filter editor so "incremental" filters can be specified from the GUI.
  23. Added command-line CSV export of the Sessions (summary) and Overview views.
  24. Added date range filtering-- the current shown date range now appears at the top of the statistics, in the Filters bar, and you can choose the starting and ending year/month/day to zoom in on a particular date range.
  25. Added date support for bpftTraflog
  26. Added expansion of path segment larger than a certain size, automatically in offline pages and manually through an "expand all" link in dynamic stats.
  27. Added filter numbers to the Log Filters table.
  28. Added ignore_quotes option. This option, which is generally of use only if you're creating a custom format, tells Sawmill to treat quotes (" or ') the same as any other character. Some formats require quotes to be treated specially for optimal processing; some require they they *not* be treated specially; this lets you choose.
  29. Added IIS/CGI installation chapter to documentation
  30. Added red arrow icons to *all* URLs in tables (so you can go to that URL by clicking), not just referrer URLs.
  31. Added red-arrow icons to entry/exit page views, when "Server Root" is set.
  32. Added rekey_collected_entry and reverse_rekey_collected_entry log filter actions. These filters are of interest only to people creating log format plug-ins for strange formats. They allow Sawmill to process log formats where the key field changes as the entry goes through stages, but where there is some value that ties the old and new keys together.
  33. Added support CacheFlow RealMedia logs
  34. Added support for "m/d/yy h:mm" date/times.
  35. Added support for %{%Y-%m-%d}t and %{%T}t dates and times in Apache custom log format strings.
  36. Added support for a LOGANALYSISINFODIR environment variable which, when set, determines the location and name of the LogAnalysisInfo folder.
  37. Added support for a new Sidewinder log format
  38. Added support for Apache Combined with Server Name after Agent
  39. Added support for Atom Mesge log format.
  40. Added support for bpftTraflog
  41. Added support for Bulletproof/G6 log format with mm/dd/yy dates.
  42. Added support for BulletproofG6FTDDMMYYYY log format
  43. Added support for CacheFlowRealMedia & CacheFlowerWindowsMedia logs
  44. Added support for Centrinity FirstClass logs
  45. Added support for CentrinityFirstClass (mm/dd/yyyy) log format.
  46. Added support for Checkpoint Alternate log format
  47. Added support for Cisco CE Common logs
  48. Added support for Cisco IDS IOS log format
  49. Added support for Cisco IOS (Unix syslogd) as well as created language variable IOS_PARSING_FILTERS logs
  50. Added support for Cisco IOS Alt3 logs
  51. Added support for Cisco PIX (NT Syslog, With Hostname) log format.
  52. Added support for Cisco PIX KIWI Alt2 logs
  53. Added support for Cisco PIX KIWI ISO.
  54. Added support for Cisco PIX SL4NT logs
  55. Added support for Cisco Router log format.
  56. Added support for Cisco SOHO77 logs
  57. Added support for Cisco VPN Concentrator (Comma-separated) log format.
  58. Added support for CiscoVoiceRouter (KIWI) logs
  59. Added support for ColdFusion Application log format.
  60. Added support for ColdFusion Web Server log format.
  61. Added support for Declude log format.
  62. Added support for DecludeSPAM log format
  63. Added support for Eims Mail SMTP log format
  64. Added support for FastHosts log format.
  65. Added support for FedEx tracking log format.
  66. Added support for files (like CSV files) which can have line breaks in the middle of a field, as long as the field is quoted.
  67. Added support for Firebox log format
  68. Added support for FlashFXP log format
  69. Added support for GENERIC_IOS Debug IP Packet Detailed Logs.
  70. Added support for GNATBox Kiwi yyyy-mm-dd logs
  71. Added support for Groupwise Web log format.
  72. Added support for GroupwiseWebmmddyy log format
  73. Added support for Hosting.com log format.
  74. Added support for IISFTP log format
  75. Added support for Interscan Email Logformat
  76. Added support for Interscan Proxy log format.
  77. Added support for Interscan Viruswall Virus log format
  78. Added support for InterscanEmailViruswall log format
  79. Added support for IST logs
  80. Added support for LSMTPAccess logs
  81. Added support for MailerDaemon log
  82. Added support for MailMax SE POP log format.
  83. Added support for MailMax SE SMTP log format.
  84. Added support for Merak IMAP/POP3 Log Format.
  85. Added support for Merak SMTP Log Format.
  86. Added support for Microsoft Proxy logs with m/d/yyyy dates.
  87. Added support for Microsoft SQL Profiler log format.
  88. Added support for Mod Gzip log format
  89. Added support for MonitorWare logs
  90. Added support for MonitorWareAlt logs
  91. Added support for N2H2 log format
  92. Added support for Nescape Directory Server log format.
  93. Added support for Net-Acct log format
  94. Added support for NetCache NetApp log format.
  95. Added support for NetScreen log format.
  96. Added support for Network syslog format.
  97. Added support for Novell Border Manager logs
  98. Added support for OpenwaveIntermail logs
  99. Added support for PostOfficeMailServer log format
  100. Added support for ProFTP log format.
  101. Added support for qmail (syslog required) log format
  102. Added support for Quicktime Streaming Server logs
  103. Added support for Radware Load Balancing log format
  104. Added support for RaidenFTP log format
  105. Added support for RealProxy log format.
  106. Added support for Samba Server log format
  107. Added support for Sambar Server log format
  108. Added support for Sidewinder log format
  109. Added support for Siteminder Netegrity beyond 4.x logs
  110. Added support for Snort Alternate logs
  111. Added support for SNORT Portscan logs
  112. Added support for SonicWallKiwi (yyyy-mm-dd) log format
  113. Added support for SquidGuard log
  114. Added support for SymantecWebSecurity logs
  115. Added support for syslog-only log format plugins, and "generic" log format plugins. Previous versions of Sawmill supported separately each combination of logging device and syslog server. This could get very ugly, with Cisco PIX as the classic example-- 6.3b3 supported 15 versions of Cisco PIX, all similar but using different Syslog servers. Now, there is a "generic" Cisco PIX plugin, and a simple Kiwi Syslog plugin that is not tied to any particular device, and Sawmill lets you choose both of them, and combined them to support Cisco PIX logged through Kiwi (if only one "generic" and only one "syslog" format match, it will be chosen for you automatically, comparably to how it's done now). We will continue to add new "generic" plug-ins and "syslog" plug-ins until all previous syslog-based plugins have been obsoleted, but the will be a period of overlap. When we're done, the total number of formats supported by Sawmill will be dramatically higher than before, because every one of 20-some syslog formats will work with every of the 20-some generic formats, for 400 new effective supported formats.
  116. Added support for Sysreset Mirc log format
  117. Added support for tcpdump With Interface Alternate
  118. Added support for TomcatAlt log format, also Greg modified Sawmill so that it can collect and carryover
  119. Added support for URLScan log format
  120. Added support for URLScan log format.
  121. Added support for War FTP logs
  122. Added support for WarFTPAlt log format
  123. Added support for WebShield SMTP log format.
  124. Added support for Webtrends Extended log format.
  125. Added support for whatsUp syslog
  126. Added support for Whistle Blower Performance Metrics Log.
  127. Added support for WhistleBlower log format.
  128. Added support for Windows Event log format.
  129. Added support for X-Stop log format.
  130. Added support for zyXEL Communications log format
  131. Added support for Zyxel/Kiwi log format.
  132. Added support Watchguard WELF log format
  133. Added update/rebuild links to the info bar in the config menu, to allow for easy updating/rebuilding of database from the statistics (admin only).
  134. Changed language module version numbers to match Sawmill version numbers. For instance, where the language module version for Sawmill 6.3.10 used to be v88, it is now v6.3.10. This makes it easier to find the language module for a particular version, and eliminates certain types of uncommon errors where a pre-release version would not be able to find recent language module variables.
  135. Changed sort order to list directories first in the Browse... window.
  136. Changed the "page view vs. non-page view" log filters, which categorize hits based on whether they are page views or not, so it uses the "file type" field to categorize, rather than the "page" (or equivalent) field. This is a better method because the "file type" field extracts the extension properly even when there are URL parameters, so for instance it will correctly categorize a hit like /images/some.gif?param which would have been categorized as a page view previously.
  137. Changed the way Sawmill handles GUI requests. Previous versions would handle requests by creating a separate thread within the main web server process; Sawmill now creates a separate process for each new request. This has some major advantages. Most importantly, it improves stability by making it impossible for separate tasks to stomp on each others' memory. This was a particularly big problem on Solaris, where there has been a bug which causes crashes when using Sawmill in web server mode; this bug is now fixed. Other lesser bugs (including some unknown ones) may also be fixed by this change. This change also makes other features possible, including the improved progress feature and the web server administration improvements in this version.
  138. Created generic Unix Syslog log format plugin.
  139. Extended support for Bind Query Log Format With Timestamp log format
  140. Extended support for Cisco PIX KIWI logs
  141. Extended support for generic Cisco logs
  142. Extended support for iMail logs
  143. Extended support for Netegrity SiteMinder 4.X logs
  144. Extended support for Netegrity Siteminder logs
  145. Extended support for Snort Alt log format and made it into a syslog required log data type
  146. Extended support for Snort logs
  147. Extended support for SonicWallKiwi logs
  148. Extended support in CiscoIOS log format to support AUDIT_TRAIL information
  149. Extended support of Cisco PIX KIWI IOS logs
  150. Extended support of Cisco PIX UTC logs
  151. Extended support of OpenwaveIntermail Server Logs to capture bandwidth information
  152. Extended support of Postfix Mail Server Logs
  153. Extended support of ServU FTP logs.
  154. Extended support of TinyPersonalFirewall to include a new variety of log
  155. Fixed a bug where errors were not reported properly, especially on Windows (a blank page would appear instead).
  156. Fixed a bug where if the log format matched one or more syslog formats, but didn't match any other formats, Sawmill would show an empty menu of available formats.
  157. Fixed a bug where images did not appear correctly in the progress page generated during a "send view by email" operation.
  158. Fixed a bug where progress pages would reload forever during a "send view by email" operation, never actually showing the final result page, even though the email had been sent.
  159. Implemented a generic Cisco VPN Concentrator log format, which works with any syslog server.
  160. Implemented ColdFusion Application (CSV) log format.
  161. Improve ipchains format to accept a slight variant.
  162. Improved "log out" so it takes you back to the configuration login page, if appropriate.
  163. Improved Cisco Generic plug-in to correctly detect files that consisted entirely of %FW lines.
  164. Improved Cisco VPN Concentrator format to extract user bandwidth from "disconnect" lines.
  165. Improved Declude format to handle more fields.
  166. Improved Declude SPAM format to track message and action.
  167. Improved error handling in CGI mode in the event that the specified "temporary directory" cannot be created, or is a file.
  168. Improved error messages to eliminate the ugly traceback information; on a related topic, you can now click Report It to email any error message to our support email address.
  169. Improved format screen so the custom format string fields are only shown when the associated custom format plug-in is selected.
  170. Improved FTP log source entry so if you enter *both* a URL and other fields (like username), the values entered in the fields will override those in the URL. Previous versions ignored all fields except the URL, if a URL was specified, so if you entered ftp://mysite.com/ as the URL, and entered the username/password/pathname below, they would be ignored, and anonymous FTP would be attempted.
  171. Improved full-Sawmill-URL generation (used to generate bookmark URLs) to look at the SERVER_NAME and SCRIPT_NAME environment variables and use them intelligently in the URL.
  172. Improved generic Cisco format to handle FW-3 and FW-4 lines better.
  173. Improved GeoIP country lookup to work when DNS lookup is on. Previously, DNS lookup would resolve the IPs, preventing GeoIP from computing countries. Now, Sawmill preserves the IP for GeoIP's use, even if it's resolved, so countries are computed the same regardless of DNS settings.
  174. Improved handling of DNS server failures; Sawmill now switches to the secondary as soon as it sees the first primary failure. If the secondary also fails at some point, it tries the primary again, etc.
  175. Improved handling of situation where the user enters something like logs/* as the log filename, and checks "pattern is a regular expression." * isn't a valid regular expression, but many users don't know that, and confuse regular expressions with wildcard expressions. Sawmill used to give a long-winded error message in this case, talking about wildcard expressions and regular expressions, but we frequently get "bug reports" containing this error message, so that wasn't working either. So Sawmill now tries the "regular expression" as a wildcard expression if it isn't a valid regular expression. In other words, entering logs/* and clicking "pattern is a regular expression" will now work-- it will analyze all files in logs/.
  176. Improved iMail log format to track local domains and "ldeliver" messages.
  177. Improved IPTables format to handle a slight variant.
  178. Improved log format detection so "Generic" formats like W3C Generic are only offered as a possibility when no non-Generic formats match.
  179. Improved Netscreen format to handle a slightly variant of the format.
  180. Improved NetScreen log format to track event types.
  181. Improved Performance for Cisco PIX KIWI ISO
  182. Improved PIX/IOS plugin to track message code field.
  183. Improved Postfix log format to handle some different types of log entries.
  184. Improved progress reporting in CGI mode and in single-threaded mode. Sawmill now uses the full progress page in all cases. Previous versions used a similar version in CGI/single-threaded most with Netscape, but the feature used was not available in other browsers (like IE), so it was forced to fall back on a "stacked lines" progress indicator, where the progress information appeared as a series of tiny lines at the top of the page. That is gone-- Sawmill uses reloading progress in all cases now. It even uses reloading progress pages when it's used from the command line, and the new Web Server Administration task list lets you tap into a running command-line build to see full progress information.
  185. Improved Raptor format to track visitors, allow session information, track hosts individually, and report IPs and ports separately.
  186. Improved session IDs so they contain the visitor id, for easier identification of the source of the session.
  187. Improved Squid format plug-in to allow headers at the end of each line
  188. Improved Squid log format to accept resolved hostnames as well as IPs. Improved export so that the statistics are shown after the export, and the export is done with a .csv file extension and Content-type text/csv. Previously, exports long enough to require progress pages would be left dangling at the progress page after the export completed; now the statistics are shown at the end of the progress cycle. Also, the exported filename was previously the same as the name of the Sawmill executable (e.g. "sawmill" or a derivative), which made it difficult to figure out what format the export was in. Now the file ends with .csv, so the format is clearer (and the browser will be able to open it with an appropriate program, like Excel).
  189. Improved TaskLog messages to include lines processed and bytes processed
  190. Improved the "collected log entries" feature of Sawmill so it would recycle collected log entries after a certain number of lines without activity. Previous versions would continue to keep the collected entries in memory for the entire log processing run, which is theoretically necessary in case a related log entry at the very end of the data, but in practice is not necessary, because associated entries tend to clump very close together. In cases of very large datasets, an infinity growth option was impractical, requiring several gigabytes of RAM in some examples. By default, Sawmill now recycles log entries after 10,000 lines of inactivity; this can be customized, and recycling can be disabled to get the old behavior.
  191. Improved the "Data Not Available" error message and page. The error message has been simplified in the event that the viewer is not administrator, and no longer talks about how the data can be made available. When the view is an administrator, the message is similar to before, except that it also includes a "Make Data Available" button at the bottom. Clicking this button adds the necessary xrefs, rebuilds the database, and shows the view again (now with data available).
  192. Improved the generic Unix Syslog plug-in to handle a slight variant.
  193. Improved the log formats list so "General" log format (like General W3C, previously called Generic W3C) only appear if there are no other matches.
  194. Improved the Session Filters editor. The filters no longer display arbitrarily-long menus-- instead, they display the first 1000 options, and you can enter options manually. The rest of the menus can be shown with a click, and the limit (1000) is customizable in the Stats Sizes tab.
  195. Improved the Sessons view (now called "Individual Session(s)" so the session IDs are clickable, and apply a session filter to show only that session.
  196. Improved the Sessons view (now called "Individual Session(s)" to show a click-by-click listing of the session, if there is only one session selected.
  197. Improved the syslog/nonsyslog split so the syslog format menu is only shown when a "require syslog" format is selected.
  198. Improved the Web Server Administration page so it shows a table of active tasks. You can click a task to see progress information for that task. This works even if the tasks are CGI-based, Scheduled, or command-line-- any task will show up in the list, and you can get full progress information for any task by clicking it. The Web Server Administration was not available in CGI mode in earlier versions, but now that it has a useful purpose, it has been enabled.
  199. Improved W3C formats to support x-datestamp fields.
  200. Improved W3C support to handle #Fields: lines with a tab after the :. Also, added support for #Start-Date: headers in W3C logs.
  201. Made filters clearer when active, and added a more prominent brief blurb about filters and views in the Overview, which appears when Filters are active. This is an attempt to make the usage of Filters and Views clearer to beginning users, in particular to make it clear that after clicking an item to apply a Filter, you can click a View down the left to see more information about that item.
  202. Moved the "single-page summary" to the bottom of the views list, so first-time Sawmill uses won't be discouraged by the long time it takes to generate that view, when they're first clicking through the views.
  203. Optimized database updating in the case where the is no new data in the log source. Previous versions would re-consolidate the database and re-write the index, even though there was no change. This version skips that step, resulting in a much faster update in this situation.
  204. Renamed URLScan to URLScanW3C log format
  205. Reworked the PIX/IOS filter to be more flexible. Rather than having a huge list of pre-defined line formats, it now has a huge list of pre-defined log *segment* formats. So if new data is present that isn't expected, or if segments are put together in unexpected ways, or if some of the parts are missing, it will still grab whatever it can from each line. Since there are apparently an infinite number of PIX/IOS formats out there, this is necessary to provide something like universal compatibility.
  206. Split Firebox format apart from its syslog-dependent section, so it will work with any supported syslog now.
  207. Split Netscreen format apart from its syslog-dependent section, so it will work with any supported syslog now.

Die komplette Versionen-History von Sawmill 6



  © 1995-2011 HAAGE & PARTNER Computer GmbH · Impressum · Datenschutz · www.haage-partner.de