Re: Apache, rotatelogs & chroot environment

[David Fletcher]

Hi everyone,

Re: Apache, rotatelogs & chroot environment 

Following some more thought about the problem I described in my last post I
have found a solution. Since there have been no replies, perhaps nobody is
interested in this (very specialised) issue with Apache in a chroot, but I
wanted to report my solution for future use.

Monit can easily restart apache, therefore restarting any failed piped
logging process. However, Monit has to know that something is wrong, and it
is nice if it can find this out *before* the web server begins to lock up.

When apache tries to re-start the piped logging process (e.g. rotatelogs)
it looks for the binary it was previously running, but it looks on the
inside of the chroot, when the binary lies outside if the chroot is created
using mod_security.

Actually putting a copy of the binary inside the chroot jail is a
nuisance, and in any case the log files would start going inside the chroot
rather than the normal place, which is messy. Much better is to put a
simple statically linked binary as a 'fake' rotatelogs on the inside of the
chroot. When apache runs this fake rotatelogs, this binary updates a status
file(e.g./tmp/log_fail.txt), which Monit notices by a timestamp change, and
the server is automatically re-started.

Code for the fake fakerotatelogs.c:
-----------------------------------------------------------
#include <stdio.h>
#include <stdlib.h>

//Fake rotatelogs: compile with
//gcc -static -o rotatelogs fakerotatelogs.c

int
main(int argc, char *argv[])
{

  FILE * out;
  time_t timer;

  timer=time(NULL);

  /*Open logging state file and write time of failure*/
  out = fopen("/tmp/log_fail.txt", "w");
  
  fprintf(out, "Logging failed on %s", asctime(localtime(&timer)));

  fclose(out);

  return 0;
}
-----------------------------------------------------------

Compile it and place the output inside the chroot as (in my case)

/chroot/apache/usr/local/apache/bin/rotatelogs

Add to the /etc/monitrc file an additional entry for watching the timestamp
on the state file at /chroot/apache/tmp/log_fail.txt.

/etc/monitrc entry
-----------------------------------------------------------
check file logging_indicator with path /chroot/apache/tmp/log_fail.txt  
group www   
start program = "/usr/bin/touch /chroot/apache/tmp/log_fail.txt"   
if changed timestamp then exec"/usr/local/bin/monit restart apache"
alert address@hidden
-----------------------------------------------------------

This Monit entry also contains a "start" entry, which will create an
initial empty state file if one doesn't already exist.

Hopefully, with this in place the server will be restarted if a logging
process dies, rather than locking up. Restarting is more drastic than
simply creating a new logging process, but it should be a rare event. 

One worry is that the state file inside the chroot provides a way for
someone to maliciously cause the server to re-start from inside the
chroot. Perhaps a timeout of some sort on the number of times the "exec"
process runs would be useful?

An alternative approach might be to use Monit to spot the growing number of
locked child httpd processes, and just assume a problem rather than getting
into all these details.

I hope this is of use to someone.

Regards,

David.

-- 
-------------------------------------------------
Email: address@hidden
-------------------------------------------------