Table of Contents
Lire's intend to be the universal log reporting tool. It should be able to process logs from any products and generate useful reports from it. To be useful in the heterogeneous networks that are common nowadays, reports from different products accomplishing similiar functionalities should be comparable. To this end, Lire is designed around a three processes architecture.
Lire's architecture contains three processes:
Log Abstraction. The first process abstracts logs from different products into a generic format (DLF) that can be shared by all products that have similar functionality. For example, log files from products as different as Apache and Microsoft Internet Information Server will be transformed into an identical format.
Report Generation. The second process generates a report from the generic log. This report is based on the user's configuration. This process is a generic report engine that compute the report based on specifications that describes the operations that are need to create the report. The report is generated in a generic XML format.
Report Formatting. The last process converts the generic XML report's format into a more standard output format for human reading like HTML, text or PDF.
The mapping of those three processes can be seen in the Figure 1.1 figure which present the intermediary products of those processes and their mapping to the Lire's component that implement them. Each of these processes is detailed in a later chapter.
This section defines more precisely some terms that will be used often in the rest of this manual.
1010912574 10.0.0.2 121.68.134.195.in-addr.arpa PTR recurs 1010912574 10.0.0.2 121.68.134.195.in-addr.arpa PTR recurs 1010912592 10.0.0.2 120.67.123.212.in-addr.arpa PTR recurs 1010912600 10.0.0.2 207.7.178.212.in-addr.arpa PTR recurs 1010912600 10.0.0.2 tr16.kennisnet.nl A recurs 1010912616 10.0.0.2 120.67.123.212.in-addr.arpa PTR recurs 1010912630 10.0.0.2 207.7.178.212.rbl.maps.vix.com ANY recurs 1010912630 10.0.0.2 NLnet.nl ANY recurs
DLF stands for “Distilled Log Format”. This is the generic log format used by Lire to abstract the different products log files. This is a really simple ASCII format where each event is represented by one line. The information about the event is represented by fields separated by spaces. All non-printable ASCII characters are replaced by ?. Spaces in field's value are replaced by _ (the underscore). Each lines must have the same number of fields. A DLF file doesn't contains any header information. Example 1.1 shows an excerpt of a DNS DLF file.
Information about the order of the fields in a DLF file, their types and what they represent is specified in the DLF's schema. Schemas are defined in XML files using the Lire DLF Schema Markup Language (LDSML). Lire's offers an API (only in Perl for now) to programmaticaly access the information of the schema.
It's the fact that several different products' log files can share a common DLF schema that makes Lire's reports easily comparable.
A report is what is generated by Lire. It is made of several subreports. Those subreports can be grouped into sections. The report is computed from the DLF file (and not the native log file) based on a configuration file which describes the subreports that make up the report along with their parameters. (Consult the Lire User's Manual section Customizing Lire for more information.)
Put simply, a service is a specific application that produce log. Altough it is usually the case, one application will be equivalent to one service. For example, the mysql service is used to process MySQL's log files.
But more precisely, a service is a specific log format. For example, the common service can be used for all web servers that supports the Common Log Format. Similarly, the welf service can be used to process the firewalls' log files written using WebTrends Enhanced Log Format.
In order to generate a report on it, the native log will be converted to the appropriate superservice's DLF schema
A subreport is a particular view on the DLF log's data. Subreports are defined in XML files using the Lire Report Specification Markup Language (LRSML). (Altough it defines subreports, it is called a Report Specification because several subreports makes up the report.) Example of a subreport would be Requests by Hours of the Day.
Subreport are defined for a particular DLF schema.
A superservice is a collection of service that shares the same DLF schema and report. It is used to group together applications (services) that offers the same kind of functionalities.
Lire currently supports 8 superservices: database, dns, email, firewall, ftp, print, proxy, and www.