Re: [Logs-devel] why two options in rule?

[Vijay Lakshminarayanan]

> > Since we're giving functions as actions, the "no" part can be easily
> > (and better) expressed in the action itself, providing speedup if both
> > of them are used.
>
> I would put the "no" part into the match, myself; but I agree the "no"s
> don't need to exist.
>
> In other words: Since the no-match specifies exceptions to the match, it
> could easily be expressed within the match itself.
>
>  (if (and (equal (message msg) "foo")
>           (not (is-thursday-p)))
>      ... )
>
> > There is one possible use for keeping the rules: Logsurfers who make
> > the switch will find it easier :-)

Actually, that's what I'd meant too :-)

> Wouldn't it be fundamentally better to instead have some sort of macro
> mechanism that would essentially mash the match and no-match parts of a
> rule (as it exists now) into a form, something like the following?
>
> (if
>     `(and ,match (not ,no-match))
>      t
>      NIL)

exactly what I had in mind!

> That could:
> 1. cure your (just) criticisms of the "no"s
> 2. be (slightly) more efficient than the current implementation
> 3. allow the logsurfer converts to keep with their "no" notions
> 4. match and no-match portions may both contribute to the return values
> for the action functions to use (a /very/ minor win, but... )
>
> That sounds to me like everyone wins.  Getting a little speed increase
> here is pretty motivating on its own.  My goal is to handle the log stream
> from 10,000 computers in real-time.

Wow!

> > Here's a thought: we have macros for each level of specifying rules.
> > That way lisp users can use lisp whenever they want to and others can
> > just use the macros.
>
> yes :)
>
> I've always thought about it more as "the lisp users can always use lisp
> however they want".  They will (hopefully) write their own
> macros/functions/packages.

And maybe come up with better ideas than I have.  As of now I've given
up trying to make LoGS unlispy.  I'll take it the way it is and worry
about non-lispers later.

> There's a great thread on the SEC mailing list right now.  They want to be
> able to count the number of times a log message is seen, essentially.  The
> problem is that the author says "just use Perl code" and the users say
> "shouldn't this be a real object on par with a Rule"?
I read somewhere (I think in a python tutorial) that if when faced
with a problem and you think "I should use regexps", you now have two
problems.  Regular expressions to count lines?  Ninja techniques
indeed.

> I believe this is one of the first issues I addressed with LoGS :)  We
> solved it by picking Lisp :)  A user make make her own functions, rule
> types, macros, whatever she needs to get her job done.
>
> > I looked at the rules for Logsurfer and they're both ugly and simple.
>
> yes :)  very very ugly :)
>
> > Most people by now know regexps very well so they'll be able to write
> > that first line, no matter how complicated it is, even if they have to
> > test it several times.
>
> maybe :)
>
> they won't be able to write this unless they're total ninjas:
> a rule that creates a rule that creates a rule that creates a rule
>
> the escaping becomes a total pain.  Its incredibly hard to keep track of.
> This is the reason we started working on LoGS and its why we chose to
> represent the rules with s-expressions (nesting rules is quite simple, and
> its not a matter of how many levels of escapes you're using).

Indeed.  Rules creating rules was something I hadn't thought of.  In
fact, the Logsurfer man pages lists this very problem under BUGS.

> > There are only two things we can do with message: (message msg) and
> > (length msg), so we probably let them say message and length to refer
> > to them respectively.  quoting them would be preferrable if we could
> > get away with it.
>
> I don't understand.

I just read that and I don't either.  I was just typing stuff off the
top of my head.  Please ignore.

> One thing is is that I've not really explored the idea of what in the heck
> a message *IS* yet.  I have a vague notion that a message should be able
> to be either ASCII data (or some "text" format, unicode maybe) or "binary"
> log data, something like a M$ Windoze binary log.

True.  binary logs completely slipped my mind.  I think what I'd meant
previously had been that we could just refer to (message message) by
message and (length (message message)) by length.  But it isn't a big
save and only causes ambiguity.  Stupid Idea.

Cheers
Vijay