[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: PID file being removed
|
From: |
Martin Pala |
|
Subject: |
Re: PID file being removed |
|
Date: |
Mon, 08 Sep 2003 19:55:00 +0200 |
|
User-agent: |
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4) Gecko/20030827 Debian/1.4-3 |
Hi,
i tried to replicate your problem - i have currently access to debian
and redhat linux machines only. I tried it on Debian unstable (sid) with
MySQL 4.0.14 and monit 3.2 and to have as similar configuration as
possible (see attachment for mysql rc script and config and for monitrc).
MySQL was running before i started Monit by:
unicorn:~/cvs/monit-3.2# ./monit -vc /root/cvs/monitrc.shannon
Runtime constants:
Control file = /root/cvs/monitrc.shannon
Log file = /var/log/monit
Pid file = /var/run/monit.pid
Debug = True
Log = True
Use syslog = False
Is Daemon = False
Use process engine = True
Poll time = 0 seconds
Mail server = (not defined)
Mail from = (not defined)
Mail subject = (not defined)
Mail message = (not defined)
Start monit httpd = False
The process list contains the following entries:
Process Name = mysql
Group = (not defined)
Pid file = /usr/local/mysql/var/sun07.pid
Monitoring mode = active
Start program = /etc/init.d/mysql
Stop program = /etc/init.d/mysql
Host:Port = (not defined)
Resource Limits = (not defined)
Every = (not defined)
Timeout = Do timeout if 3 restart within 3 cycles
Alert mail to = address@hidden
alert from = (default)
alert subject = (default)
alert message = (default)
alert on timeout = no
alert on restart = yes
alert on checksum = no
alert on resource = no
alert on stop = no
alert on timestamp = no
Alert mail to = address@hidden
alert from = (default)
alert subject = (default)
alert message = (default)
alert on timeout = yes
alert on restart = no
alert on checksum = no
alert on resource = no
alert on stop = no
alert on timestamp = no
-------------------------------------------------------------------------------
'mysql' is running with pid 6074
'mysql' zombie check passed [status_flag=0000]
'mysql' check_process_state() passed.
As you can see, everything works well => i was not able to replicate
your problem.
The mysql pidfile removal is strange - it seems that it is caused by
mysqld itself => this line of your rc script will involve pidfile removal:
su mysql -c "exec /usr/local/mysql/libexec/mysqld" &
Monit is not able to unlink monitored process's pidfile (you can look on
monit's code - the only file which is remove by monit is its own pidfile).
The main problem is that in your case monit have identified running
process as non running. It could be configuration problem (permissions,
keying mistake, etc.) or monit bug related to your environment (in such
case it is will be very interesting, because process monitoring is one
of main monit functions and it was tested on lot of operating systems,
including solaris). Then the start method is called and as i noted
mysqld will remove its pidfile probably.
As which user are you running monit?
From your prompt it seems that you are using Sun => probably Solaris,
rigth? If it is true, then you can use truss to watch what happend:
truss -leaf -r all -w all -o /tmp/monit.truss /usr/local/bin/monit -v
Please send us the truss's output (/tmp/monit.truss).
In addition you can try Monit-4.0 beta if you want to (you can get it at
http://www.tildeslash.com/monit/beta/monit-4.0.tar.gz). Following
configuration should work for you:
--SNIP--
set logfile /var/log/monit
check process mysql with pidfile /usr/local/mysql/var/sun07.pid
if 3 restarts within 3 cycles then timeout
start program = "/etc/init.d/mysql start"
stop program = "/etc/init.d/mysql stop"
alert address@hidden on { restart, timeout }
--SNIP--
We can see then whether monit-4.0 will report that running process is
not running in your case too (this is valueable feedback for us in beta
phase - in the case that there is some problem in monit we can fix it
before stable 4.0 will be released).
Thanks,
Martin
Shannon E. Reall wrote:
Here is the rc script for mysql:
#!/sbin/sh
#
case "$1" in
'start')
su mysql -c "exec /usr/local/mysql/libexec/mysqld" &
;;
'stop')
ps -ef | grep mysqld | grep -v grep | awk '{print $2}' | xargs
kill -9
;;
*)
echo "Usage: $0 { start | stop }"
exit 1
;;
esac
**The pid file is definately there after mysql is started.
Here is the output from running monit in verbose mode:
Runtime constants:
Control file = /etc/monitrc
Log file = /var/log/monit
Pid file = /var/run/monit.pid
Debug = True
Log = True
Use syslog = False
Is Daemon = False
Use process engine = True
Poll time = 0 seconds
Mail server = (not defined)
Mail from = (not defined)
Mail subject = (not defined)
Mail message = (not defined)
Start monit httpd = False
The process list contains the following entries:
Process Name = mysql
Group = (not defined)
Pid file = /usr/local/mysql/var/sun07.pid
Monitoring mode = active
Start program = /etc/init.d/mysql
Stop program = /etc/init.d/mysql
Host:Port = (not defined)
Resource Limits = (not defined)
Every = (not defined)
Timeout = Do timeout if 3 restart within 3 cycles
Alert mail to = address@hidden
alert from = (default)
alert subject = (default)
alert message = (default)
alert on timeout = no
alert on restart = yes
alert on checksum = no
alert on resource = no
alert on stop = no
alert on timestamp = no
Alert mail to = address@hidden
alert from = (default)
alert subject = (default)
alert message = (default)
alert on timeout = yes
alert on restart = no
alert on checksum = no
alert on resource = no
alert on stop = no
alert on timestamp = no
-------------------------------------------------------------------------------
start: (mysql) /etc/init.d/mysql
monit: Warning process 'mysql' was not started
Monitoring enabled -- process mysql
Restart notification is sent to address@hidden
Thanks,
Shannon
address@hidden wrote:
------------------------------
Message: 3
Date: Thu, 28 Aug 2003 13:12:41 +0200
From: Martin Pala <address@hidden>
Subject: Re: PID file being removed
To: This is the general mailing list for monit
<address@hidden>
Message-ID: <address@hidden>
Content-Type: text/plain; charset=ISO-8859-2; format=flowed
Hi,
the problem is probably caused in your mysql startup script, which
probably removes it. Monit is not able to remove the monitored service's
pidfile (until instructed to do so via exec statement which is not this
case).
Please:
1.) attach your mysql rc script - we can figure out where and why it
removed the pidfile.
2.) optionaly trace the process as described in FAQ.txt distributed with
monit, it could help to see the actions/environment which preceded to
start method execution and the cause why monit decided to start mysql
though it was running before monit started (which is not normal - there
must be some reason for it - it works in 3.2 well).
3.) run monit in verbose mode ('-v' command line option)
If it is problem for you to do any of above hints, please send kindly
just some of these informations.
Thanks for feedback
Martin
Message: 2
Date: Fri, 22 Aug 2003 07:49:19 -0400
From: "Shannon E. Reall" <address@hidden>
Subject: PID file being removed
To: address@hidden
Message-ID: <address@hidden>
Content-Type: text/plain; charset=us-ascii; format=flowed
I recently upgraded to 3.2 and am now having a problem monitoring
mysqld. Here is that portion of the conf file:
check mysql with pidfile /usr/local/mysql/var/sun07.pid
timeout(3,3)
start program = "/etc/init.d/mysql start"
stop program = "/etc/init.d/mysql stop"
alert address@hidden on { timeout }
alert address@hidden on { restart }
The pid file exists before I start monit but then it disappears. Here
is what I did for testing:
address@hidden:/# ps -ef |grep mysql
mysql 455 1 0 18:05:55 ? 0:01
/usr/local/mysql/libexec/mysqld
root 23227 19786 0 07:47:11 pts/3 0:00 grep mysql
address@hidden:/# echo 455 > /usr/local/mysql/var/sun07.pid
address@hidden:/# ls -l /usr/local/mysql/var/sun07.pid
-rw-r--r-- 1 root other 4 Aug 22 07:47
/usr/local/mysql/var/sun07.pid
address@hidden:/# /usr/local/bin/monit -d 60
Starting monit daemon
address@hidden:/# tail /var/log/monit
[EDT Aug 22 07:37:20] Starting monit daemon
[EDT Aug 22 07:39:44] Stopping monit HTTP server
[EDT Aug 22 07:39:44] monit daemon with pid [22123] killed
[EDT Aug 22 07:43:28] Starting monit daemon
[EDT Aug 22 07:43:28] start: (mysql) /etc/init.d/mysql
[EDT Aug 22 07:44:28] monit: Warning process 'mysql' was not started
[EDT Aug 22 07:44:29] Stopping monit HTTP server
[EDT Aug 22 07:44:29] monit daemon with pid [23002] killed
[EDT Aug 22 07:47:47] Starting monit daemon
[EDT Aug 22 07:47:47] start: (mysql) /etc/init.d/mysql
[EDT Aug 22 07:48:47] monit: Warning process 'mysql' was not started
address@hidden:/# ls -l /usr/local/mysql/var/sun07.pid
/usr/local/mysql/var/sun07.pid: No such file or directory
Am I missing something? Thanks for any help you can provide.
--
Shannon
------------------------------------------------------------------------
--
To unsubscribe:
http://mail.nongnu.org/mailman/listinfo/monit-general
set logfile /var/log/monit
check mysql with pidfile /usr/local/mysql/var/sun07.pid
timeout(3,3)
start program = "/etc/init.d/mysql start"
stop program = "/etc/init.d/mysql stop"
alert address@hidden on { timeout }
alert address@hidden on { restart }
#
# The MySQL database server configuration file.
#
# You can copy this to one of:
# - "/etc/mysql/my.cnf" to set global options,
# - "/var/lib/mysql/my.cnf" to set server-specific options or
# - "~/.my.cnf" to set user-specific options.
#
# One can use all long options that the program supports.
# Run program with --help to get a list of available options and with
# --print-defaults to see which it would actually understand and use.
#
# This will be passed to all mysql clients
[client]
#password = my_password
port = 3306
socket = /var/run/mysqld/mysqld.sock
# Here is entries for some specific programs
# The following values assume you have at least 32M ram
# This was formally known as [safe_mysqld]. Both versions are currently parsed.
[mysqld_safe]
err-log = /var/log/mysql/mysql.err
socket = /var/run/mysqld/mysqld.sock
[mysqld]
user = mysql
#pid-file = /var/run/mysqld/mysqld.pid
pid-file = /usr/local/mysql/var/sun07.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
#
# You can also put it into /var/log/mysql/mysql.log but I leave it in /var/log
# for backward compatibility. Both location gets rotated by the cronjob.
#log = /var/log/mysql/mysql.log
log = /var/log/mysql.log
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
language = /usr/share/mysql/english
skip-locking
#
# The skip-networkin option will no longer be set via debconf menu.
# You have to manually change it if you want networking i.e. the server
# listening on port 3306. The default is "disable" - for security reasons.
skip-networking
set-variable = key_buffer=16M
set-variable = max_allowed_packet=1M
set-variable = thread_stack=128K
#
# Here you can see queries with especially long duration
#log-slow-queries = /var/log/mysql/mysql-slow.log
#
# The following can be used as easy to replay backup logs or for replication
#server-id = 1
#log-bin = /var/log/mysql/mysql-bin.log
#binlog-do-db = include_database_name
#binlog-ignore-db = include_database_name
#
# Read the manual if you want to enable InnoDB!
skip-innodb
#
# Read the manual, too, if you want chroot!
# chroot = /var/lib/mysql/
#
# If you want to enable SSL support (recommended) read the manual or my
# HOWTO in /usr/share/doc/mysql-server/SSL-MINI-HOWTO.txt
# ssl-ca=/etc/mysql/cacert.pem
# ssl-cert=/etc/mysql/server-cert.pem
# ssl-key=/etc/mysql/server-key.pem
[mysqldump]
quick
set-variable = max_allowed_packet=1M
[mysql]
#no-auto-rehash # faster start of mysql but no tab completition
[isamchk]
set-variable = key_buffer=16M
#!/bin/sh
#
# MySQL daemon start/stop script.
#
# Debian version. Based on the original by TcX.
#
set -e
set -u
${DEBIAN_SCRIPT_DEBUG:+ set -v -x}
test -x /usr/sbin/mysqld || exit 0
SELF=$(cd $(dirname $0); pwd -P)/$(basename $0)
CONF=/etc/mysql/my.cnf
MYADMIN="/usr/bin/mysqladmin --defaults-extra-file=/etc/mysql/debian.cnf"
set +e # for deinstalling with deleted configs
ERRLOG=`perl -n -e '/^err-log[^=]*=\s*(.*)/ && { print $1 }' < $CONF`
set -e
ERRLOG=${ERRLOG:-/var/log/mysql/mysql.err}
# Safeguard (relative paths, core dumps..)
cd /
umask 077
export PATH=/bin:/usr/bin
# mysqladmin likes to read /root/.my.cnf. This is usually not what I want
# as many admins e.g. only store a password without a username there and
# so break my scripts.
export HOME=/etc/mysql/
# "if" considers exit(0) as true so we can just pass mysqladmins exit code here.
# If mysqladmin does not exist the return code will be silently set to !0, too.
is_mysqld_alive () {
s=`$MYADMIN ping 2>&1`; rc=$?
if [ $rc != 0 ]; then
(echo "`date +'%y%m%d %H:%M:%S'` Error at $0\n$s") >> $ERRLOG
fi
return $rc
}
case "${1:-''}" in
'start')
# check for config file
if [ ! -r /etc/mysql/my.cnf ]; then
echo "WARNING: /etc/mysql/my.cnf cannot be read. See README.Debian."
fi
# Start daemon
echo -n "Starting MySQL database server: mysqld"
/usr/bin/mysqld_safe > /dev/null 2>&1 &
for i in 1 2 3 4 5 6; do
if is_mysqld_alive; then break; fi
sleep 1
done
if is_mysqld_alive; then
echo "."
else
echo "...failed."
/bin/echo -e "\tPlease take a look at $ERRLOG"
fi
;;
'stop')
# * As a passwordless mysqladmin (e.g. via ~/.my.cnf) must be possible
# at least for cron, we can rely on it here, too. (although we have
# to specify it explicit as e.g. sudo environments points to the normal
# users home and not /root)
echo -n "Stopping MySQL database server: mysqld"
if is_mysqld_alive; then
set +e
$MYADMIN shutdown
r=$?
set -e
if [ "$r" -ne 0 ]; then
echo "...failed"
echo -n "Killing MySQL database server by signal: mysqld"
for i in 1 2 3 4 5 6; do
killall -15 mysqld
sleep 1
killall -9 mysqld
if is_mysqld_alive; then break; fi
done
fi
fi
if is_mysqld_alive; then
echo "...failed."
echo "Please stop the daemon yourself!"
exit -1
else
echo "."
fi
;;
'restart')
set +e; $SELF stop; set -e
$SELF start
;;
'reload'|'force-reload')
echo -n "Reloading MySQL database server: mysqld"
$MYADMIN reload
echo "."
;;
'status')
$MYADMIN version
;;
*)
echo "Usage: $SELF start|stop|restart|reload|force-reload"
exit 1
;;
esac