New Monit user here, I'm really just kind of kicking the tires.
I've got a several things that I'm monitoring on a small server that I have, but I've also get it set up to monitor services on another box. The relevant portion of monitrc looks like this:
if failed port 80 protocol http 3 times within 5 cycles then alert
if failed port 587 protocol smtp then alert
The vast majority of the time this works just fine. However, periodically I'll get a failure on one (or very occasionally on both) of these tests, which clear up on the next test cycle (60 seconds later). A few times I've been connected to the machine running monit and as soon as I get the failure, I'll try to manually telnet to the other machine on the appropriate port and it's always worked. I'm trying to figure out why it's failing.
The problem is that this doesn't happen terribly frequently, so I'm thinking that just running with -Iv might not be practical, since I'd get tons of output. (And to be honest, I'm not quite sure if I'd even see anything there.)
Can anyone suggest the best way to figure out why these tests are actually failing? Maybe run with verbose mode then tail & filter the output? (Filter for what?)
Thanks.
David