Mod cluster proxy timeout in apache error logs

Question

In our environment we get these entries consistently in error logs.

[Sun Dec 22 20:29:43 2013] [error] (70007)The timeout specified has expired: ajp_ilink_receive() can't receive header
[Sun Dec 22 20:29:43 2013] [error] ajp_read_header: ajp_ilink_receive failed
[Sun Dec 22 20:29:43 2013] [error] (70007)The timeout specified has expired: proxy: read response failed from (null) (xx.xx.xx.xx)
[Sun Dec 22 20:30:01 2013] [error] (70007)The timeout specified has expired: ajp_ilink_receive() can't receive header

If these errors appear from all nodes at once the site will be throwing a 998 response code until a restart which makes these errors disappear. Apache is communicating to Back end jboss servers.

What might be causing these issues. Based on my observation it might due to delay in webservice calls causing termination of AJP connection.

Please correct me if I am wrong and how can this issue be fixed permanently.

Thanks.

Answer 1

(better late than never)

I had a similar issue;

[Mon Jan 20 13:39:57 2014] [error] (70007)The timeout specified has expired: ajp_ilink_receive() can't receive header
[Mon Jan 20 13:43:26 2014] [error] (70007)The timeout specified has expired: ajp_ilink_receive() can't receive header

Which led me to find this:

When nodeTimeout is not defined the ProxyTimeout directive Proxy is used. If ProxyTimeout is not defined the server timeout (Timeout) is used (default 300 seconds). nodeTimeout, ProxyTimeout or Timeout is set at the socket level.

I had neither a nodeTimeout nor a ProxyTimeout, which leaves us with the Timeout variable (which is set to 60 seconds). Under most situations, 60 seconds is sufficient for a webpage to return, however investigation into the app revealed a bad MySQL query was taking over 20 minutes (yes, minutes!) to run.

You could try increasing your timeouts, or more likely, figure out why the page is slow.

Answer 2

I fixed this error as folow:

 <Proxy *>
    AddDefaultCharset Off

    Order deny,allow

    Allow from all

 </Proxy>

ProxyPass / ajp://backend.example.com:8009/ retry=0 timeout=10 ttl=60

ProxyPassReverse / ajp://backend.example.com:8009/