Calling an external program from monit [was: Re: multiple pids]

[Jan-Henrik Haukeland]

On 20. apr. 2006, at 16.46, Martin Pala wrote:

> Jan-Henrik Haukeland wrote:
> > On 20. apr. 2006, at 12.19, Marc Petitmermet wrote:
> > > I guess then that there is also no
> > >
> > >    "check status with script /etc/init.d/serviceX"


This is actually a good syntax and I suggest that we use this with  
some slight modifications. I suggest that we support two levels of  
running an external script. 1) as a full 'check status' service and  
2) as an if test. Here are the syntax defined more formal (keywords  
in uppercase) for those two cases:

1) CHECK STATUS OF name WITH PATH "/path/to/script"
        [[AND] TIMEOUT AFTER X sec]
        IF FAILED THEN
                {ALERT|MONITOR|UNMONITOR|START|STOP|RESTART|EXEC}
        [ELSE
                {ALERT|MONITOR|UNMONITOR|START|STOP|RESTART|EXEC}
        ]
       [ALERT ..]
       [EVERY ..]
       [DEPENDS ON ..]
       [GROUP ..]

2) check X ...
        IF FAILED STATUS OF [SCRIPT|PROGRAM] "/path/to/script"
          [AND TIMEOUT AFTER X sec] THEN
                {ALERT|MONITOR|UNMONITOR|START|STOP|RESTART|EXEC}
        ELSE
                {ALERT|MONITOR|UNMONITOR|START|STOP|RESTART|EXEC}
        ...

Detailed discussion:
====================

The script is executed by monit and the return value is used to  
decide the success. That is, if the script returns 0 it succeded and  
if it returns anything else it failed.

The new sub-statement [TIMEOUT AFTER X sec] is used to timeout  
execution. I.e if the script did not return after X seconds, monit  
aborts the execution and the test failed. This statement is optional  
and if not used, defaults to 5 seconds.

We should _not_ use the popen(3) function. It is considered unsafe  
and is only a variant of the system(3) call. Instead we should do our  
own plumbing and use fork(2), execv(3) and pipe(2) to read output  
from the script. The output will be logged if and only if an error  
occured and also sent in any alert message.

I think that this covers Martins input below. Please provide any  
feedback or changes to this suggestion. We start by adding this  
feature to our TODO list at http://www.tildeslash.com/monit/doc/ 
next.php.


> I agree - we can provide some interface which can smoothly  
> integrate monit with external check scripts.
>
> The script should return to monit (for example):
> - the status (error level)
> - detailed description of the situation (message)
>
> The script should probably not perform the action itself - this  
> should be monit responsibility. Monit can map the error level  
> (status) to the appropriate action and call the methods (such as  
> alert, stop, restart, unmonitor, etc.)
>
> Monit can encapsulate the script:
>
> - set the timelimit on script execution (in the case that the  
> script will not return in some time limit, monit can perform the  
> actions related to unresponsive fault monitor (for example send  
> alert, etc.)
>
> - make sure that there is just one instance running