Re: MySQL monitoring - Host 'linux.local' is blocked because of many connection errors

[Martin Pala]

Hi Srini,

we have reproduced the issue and prepared fix, you can get the development 
snapshot here: https://mmonit.com/tmp/monit-5.14_devel.tar.gz

Compilation:

        tar -xzf monit-5.14_devel.tar.gz
        cd monit-5.14_devel
        ./configure
        make

You can then run monit either from the build directory (e.g. ./monit) or 
install it (make install)

Regards,
Martin


> On 25 May 2015, at 09:57, Srinivasan Thandapani <address@hidden> wrote:
> 
> Martin,
> 
> Thanks for looking into this and for the analysis.
> 
> From this link 
> http://serverfault.com/questions/578682/monit-mysql-server-blocked-because-of-too-many-bad-connections
>  , I was able to do a workaround by doing a real connection to MySQL to do 
> test status instead of doing a port test.
> 
> I will reproduce the original issue & will share the details later that you 
> have mentioned. Thanks a lot.
> 
> Regards,
> Srini
> 
> -----Original Message-----
> From: address@hidden [mailto:address@hidden On Behalf Of Martin Pala
> Sent: Friday, May 22, 2015 6:00 PM
> To: This is the general mailing list for monit
> Subject: Re: MySQL monitoring - Host 'linux.local' is blocked because of many 
> connection errors
> 
> Yet one thing … according to mysql documentation the connecton errors counter 
> is incremented in the case the the connection is *interrupted*. It is very 
> likely the MySQL protocol test failed 10 times before your host was blocked 
> for some other reason, which will point to the root cause.
> 
> Please can you enable Monit logging (add “set logfile <path>|syslog” 
> statement if it’s missing + reload monit), unblock the host (using for 
> example “mysqladmin flush-hosts”) and when the host will be locked out again, 
> send the whole mysql-related monit log and mysql error log. The errors 
> preceding the blocked host error message are trigger the issue.
> 
> Regards,
> Martin
> 
> 
> 
>> On 21 May 2015, at 22:18, Martin Pala <address@hidden> wrote:
>> 
>> Hi,
>> 
>> i’m unable to replicate the issue with the mysql protocol check using the 
>> following Monit configuration (with 1s poll interval):
>> 
>> —8<—
>> check host mysql_server with address 1.2.3.4
>>  if failed port 3306 protocol mysql then alert —8<—
>> 
>> Tested with following MySQL servers:
>> 
>> 1.) MySQL 5.1.73 on CentOS 6.6 with default max_connect_errors
>> settings (max_connect_errors=10)
>> 
>> 2.) MySQL 5.5.43 on Ubuntu 14.04.2 with default max_connect_errors
>> settings (max_connect_errors=10)
>> 
>> (created mysql user to allow connections from the host, from which
>> monit is running checks)
>> 
>> 
>> Please can you run Monit in verbose mode (using “-vI” options) and provide 
>> output for MySQL check?
>> 
>> The problem could be triggered by some other client connecting/testing your 
>> MySQL server (not Monit test) and Monit then just reports the real problem, 
>> where your host is locked out.
>> 
>> Please can you test with MySQL 5.1.73? Because we’re not able to replicate 
>> the problem with it, it is possible it was some MySQL bug fixed in some 
>> post-5.1.40 release.
>> 
>> Regards,
>> Martin
>> 
>> 
>>> On 21 May 2015, at 10:08, Srinivasan Thandapani <address@hidden> wrote:
>>> 
>>> Hi Team,
>>> 
>>> 
>>> 
>>> I’m monitoring a remote MySQL server using a protocol test to port
>>> 3306. Here is my configuration in monitrc file,
>>> 
>>> 
>>> 
>>> check host myserver with address xx.xx.xx.xx
>>> 
>>>               if failed port 3306 protocol mysql with timeout 15
>>> seconds then alert
>>> 
>>> 
>>> 
>>> After some point of time, I’m getting error the below error from
>>> MySQL Server,
>>> 
>>> 
>>> 
>>> Host 'linux.local' is blocked because of many connection errors
>>> 
>>> 
>>> 
>>> Can someone help on how to fix this? I don’t not want to increase the 
>>> max_connect_errors variable in MySQL.
>>> 
>>> 
>>> 
>>> Here is my details,
>>> 
>>> 
>>> 
>>> Monit version: 5.12.1
>>> 
>>> MySQL Server version: 5.1.40
>>> 
>>> 
>>> 
>>> Regards,
>>> 
>>> Srini
>>> 
>>> 
>>> 
>>> "DISCLAIMER: This message is proprietary to Aricent and is intended solely 
>>> for the use of the individual to whom it is addressed. It may contain 
>>> privileged or confidential information and should not be circulated or used 
>>> for any purpose other than for what it is intended. If you have received 
>>> this message in error, please notify the originator immediately. If you are 
>>> not the intended recipient, you are notified that you are strictly 
>>> prohibited from using, copying, altering, or disclosing the contents of 
>>> this message. Aricent accepts no responsibility for loss or damage arising 
>>> from the use of the information transmitted by this email including damage 
>>> from virus."
>>> --
>>> To unsubscribe:
>>> https://lists.nongnu.org/mailman/listinfo/monit-general
>> 
>> 
>> --
>> To unsubscribe:
>> https://lists.nongnu.org/mailman/listinfo/monit-general
> 
> 
> --
> To unsubscribe:
> https://lists.nongnu.org/mailman/listinfo/monit-general
> "DISCLAIMER: This message is proprietary to Aricent and is intended solely 
> for the use of the individual to whom it is addressed. It may contain 
> privileged or confidential information and should not be circulated or used 
> for any purpose other than for what it is intended. If you have received this 
> message in error, please notify the originator immediately. If you are not 
> the intended recipient, you are notified that you are strictly prohibited 
> from using, copying, altering, or disclosing the contents of this message. 
> Aricent accepts no responsibility for loss or damage arising from the use of 
> the information transmitted by this email including damage from virus."
> --
> To unsubscribe:
> https://lists.nongnu.org/mailman/listinfo/monit-general