Re: Log files monitoring

[Jan-Henrik Haukeland]

On 5. mai. 2005, at 17.40, Christian Hopp wrote:

> On Thu, 5 May 2005, Sorin S. wrote:
>
> Hi!
>
>
> > 1) The implementation can be similar with the logtail
> > (the one written by Craig Rowland, which I am
> > attaching for convenience)  - that is we care about
> > the inode and not the file, so we would not be
> > sensitive at logrotation. Also we can keep an offset
> > (logtail keeps it in a file on the same directory with
> > the logfile, we might want to keep it in the state
> > file) so we can catch only the last changes in the
> > file. We run logtail against some large logfiles
> > (8-12G) with no problems whatsoever.
>
> Some distributions use logrotations by default (some work well, some
> don't).  And actually I don't wanna miss it (I find too large logs
> inconvinient, chunkier ones are easier to handle!). (: That's why we
> always have to check if the inode changes. Then we have to read the
> rest of the old one and continue with the new one.
>
> The position should be saved in the monit.state.  We don't want to
> leave our mess, where we are not supposed to write in. (:

Good idea. However since monit is a daemon we can maintain the files  
and offset in a struct in-memory. We should use the monit-state file  
to update this struct in case of restart or crash so we can start  
reading from where we left.

> > 2) Multiline parsing is needed. As monit checks the
> > logs every N seconds it is probable that we get
> > multiple lines on the logs.
>
> Well, what I ment are so called multiline regexps.  Is it necessary
> catch multiple lines with one regexp?

I think we should try to parse in a line-by-line fashion or in chunks  
of e.g. 512 bytes, whichever comes first. Judging from my humble Perl  
and Awk experience, handling multiple new line records with a regular  
expression language is not easy, or am I wrong?

--
Jan-Henrik Haukeland
Mobil +47 97141255