Spring cloud Zuul retry when instance is down and forward to other available instance

匿名 (未验证) 提交于 2019-12-03 08:54:24

问题:

using 'Camden.SR5' for spring-cloud-dependencies, with spring boot '1.5.2.RELEASE'.

In my current setup, I have

  • eureka server
  • config server (running on random ports)
  • zuul gateway server
  • and 2 instances of a service (running on random ports)

All these instances are successfully register with Eureka.

When all the services are running, The load balancing is done properly through zuul without any issues.

when an instance is killed, Zuul is still trying to fulfil the request using the same service which is down. However if waited till the eureka registry is fetched after shutting down the instance, requests are fulfilled with the other instances which are 'UP'.

    2017-03-07 19:57:41.409 DEBUG 26658 --- [nio-5555-exec-3] c.n.l.reactive.LoadBalancerCommand       : Got error org.apache.http.conn.HttpHostConnectException: Connect to 10.99.4.151:64381 [/10.99.4.151] failed: Connection refused when executed on server 10.99.4.151:64381 2017-03-07 19:57:41.420 DEBUG 26658 --- [nio-5555-exec-3] com.netflix.hystrix.AbstractCommand      : Error executing HystrixCommand.run(). Proceeding to fallback logic ...  com.netflix.client.ClientException: null     at com.netflix.client.AbstractLoadBalancerAwareClient.executeWithLoadBalancer(AbstractLoadBalancerAwareClient.java:123) ~[ribbon-loadbalancer-2.2.0.jar:2.2.0]     at com.netflix.client.AbstractLoadBalancerAwareClient.executeWithLoadBalancer(AbstractLoadBalancerAwareClient.java:81) ~[ribbon-loadbalancer-2.2.0.jar:2.2.0]     at org.springframework.cloud.netflix.zuul.filters.route.support.AbstractRibbonCommand.run(AbstractRibbonCommand.java:96) ~[spring-cloud-netflix-core-1.2.5.RELEASE.jar:1.2.5.RELEASE]     at org.springframework.cloud.netflix.zuul.filters.route.support.AbstractRibbonCommand.run(AbstractRibbonCommand.java:42) ~[spring-cloud-netflix-core-1.2.5.RELEASE.jar:1.2.5.RELEASE] 

    at org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:75) ~[httpclient-4.5.3.jar:4.5.3]     at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ~[httpclient-4.5.3.jar:4.5.3]     ... 162 common frames omitted  2017-03-07 19:57:41.425 DEBUG 26658 --- [nio-5555-exec-3] com.netflix.hystrix.AbstractCommand      : No fallback for HystrixCommand.   java.lang.UnsupportedOperationException: No fallback available.     at com.netflix.hystrix.HystrixCommand.getFallback(HystrixCommand.java:292) [hystrix-core-1.5.6.jar:1.5.6]     at org.springframework.cloud.netflix.zuul.filters.route.support.AbstractRibbonCommand.getFallback(AbstractRibbonCommand.java:117) ~[spring-cloud-netflix-core-1.2.5.RELEASE.jar:1.2.5.RELEASE]     at org.springframework.cloud.netflix.zuul.filters.route.support.AbstractRibbonCommand.getFallback(AbstractRibbonCommand.java:42) ~[spring-cloud-netflix-core-1.2.5.RELEASE.jar:1.2.5.RELEASE] 

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_66]     at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.5.11.jar:8.5.11]     at java.lang.Thread.run(Thread.java:745) [na:1.8.0_66]  2017-03-07 19:57:41.428  WARN 26658 --- [nio-5555-exec-3] o.s.c.n.z.filters.post.SendErrorFilter   : Error during filtering  com.netflix.zuul.exception.ZuulException: Forwarding error     at org.springframework.cloud.netflix.zuul.filters.route.RibbonRoutingFilter.handleException(RibbonRoutingFilter.java:170) ~[spring-cloud-netflix-core-1.2.5.RELEASE.jar:1.2.5.RELEASE]     at org.springframework.cloud.netflix.zuul.filters.route.RibbonRoutingFilter.forward(RibbonRoutingFilter.java:145) ~[spring-cloud-netflix-core-1.2.5.RELEASE.jar:1.2.5.RELEASE]     at org.springframework.cloud.netflix.zuul.filters.route.RibbonRoutingFilter.run(RibbonRoutingFilter.java:88) ~[spring-cloud-netflix-core-1.2.5.RELEASE.jar:1.2.5.RELEASE] 

Following are the zuul configuration used with @EnableZuulProxy and @EnableEurekaClient

    server:   port: 5555  spring:   application:     name: gateway-server   cloud:     config:       discovery:         enabled: true         service-id: CONFIGSERVER       fail-fast: true       retry:         multiplier:  1.1         initial-interval: 1000         max-attempts: 6         max-interval: 2000  hystrix:   command:     default:       execution:         isolation:           thread:             timeoutInMilliseconds: 100000         timeout:           enabled: false  ribbon:   ReadTimeout: 5000   ConnectTimeout: 3000   maxAutoRetries: 1   MaxAutoRetriesNextServer: 2   OkToRetryOnAllOperations: true   logging:   level:     ROOT: DEBUG  zuul:   routes:     security-service:       retryable: true 

The 2 instances of service with are running with unique instance-ids

@EnableEurekaClient @EnableHystrix @SpringBootApplication public class SecurityServer implements HealthIndicator{      public static void main(String args[])     {         SpringApplication.run(SecurityServer.class,args);     }      @Override     public Health health() {         return Health.up().withDetail("STATUS", "SUCCESS").build();     } } 

instanceId: ${spring.cloud.client.hostname}:${spring.application.name}:${spring.application.instance_id:${random.uuid}} 

Can you help me with the zuul & instances configuration, so that request is automatically forwarded to the other available instances when an instance goes down.

回答1:

upon searching out more and looking into spring-cloud-netflix issue tracker, There is a wonderful discussion between william-tran and ryanjbaxter on the best practices. Thanks to both of you.

https://github.com/spring-cloud/spring-cloud-netflix/issues/1290#issuecomment-242204614

https://github.com/spring-cloud/spring-cloud-netflix/issues/1295

In summary, Camden doesn't use the Ribbon HTTP Client(deprecated) so none of the ribbon.* properties will help you control the retry logic. Camden uses Apache HTTP client.

So the solution would be to use Ribbon HTTP Client in camden version using below configuration

ribbon.restclient.enabled=true 

or

Move to Camden.BUILD-SNAPSHOT or Dalston.BUILD-SNAPSHOT for using spring-retry (https://github.com/spring-projects/spring-retry)



回答2:

A very good article is here by Ryan Baxter on the fixes for this issue in Brixton and Camden release.



易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!