connection failed, INET[www.example.com:80] via TCP is not ready for i|o -- Interrupted system call

[Marcus Mülbüsch]

Hello all,

   hoping that this helps someone somewhere in the future. Skip the 
text and jump to the solution at the bottom if you are not interested in 
my adventure. I write this purely because googling for the error message 
did not show up with a single useful entry.

   Since yesterday I received the message "connection failed, 
INET[www.example.com:80] via TCP is not ready for i|o -- Interrupted 
system call" every 5 minutes from a monit instance.

   Sometimes it skipped the five minutes and came back after ten 
minutes or more (but always a multiple of five), sometimes it jumped one 
minute ahead and faulted after six minutes, to continue then again after 
five.

   The server, however, *was* reachable and working (as it tested 
successfully from another server).

   When I investigated further I discovered that the firewall did open 
all necessary connections, so it wasn't a fault there.

   When I tcpdumped the traffic on the monit instance everything worked 
fine: Monit opens a TCP connection and the whole process of sending SYN, 
SYN/ACK, ACK, traffic and finally FIN, FIN/ACK etc. went smoothly.

   However, every five minutes monit sent a SYNC, received a SYNC/ACK, 
but instead of acknowledging with an ACK the very next package was a RST.

   Duh!

   Replaying a configuration from the backup from the day before 
yesterday didn't help.

   Finally I checked /var/log/messages (should have done that at first, 
but you know how it goes), where I discovered the wonderful "init: Id 
"mo" respawning too fast: disabled for 5 minutes".

   Yes, I start monit from inittab.

   Yesterday the switch where this server was connected to died. After 
reconnecting to another switch everything was fine again, but somehow 
the init process must have thought that monit was not running, tried to 
start it only to find out it was already running...

   So the solution was very simple:

SOLUTION:
--------

a) comment out the "mo:2345:respawn:/usr/bin/monit \-I" from /etc/inittab.
b) restart the init process ("init q")
c) stop monit ("monit quit")
d) uncomment again the monit entry from inittab.
e) Restart init again ("init q")
d) watch your logfile. No more errors

Marcus