Why is ZUUL forcing a SEMAPHORE isolation to execute its Hystrix commands?

Question

I noticed Spring-Cloud ZUUL forces the execution isolation to SEMAPHORE instead of the THREAD defaults (as recommended by Netflix).

A comment in org.springframework.cloud.netflix.zuul.filters.route.RibbonCommand says:

we want to default to semaphore-isolation since this wraps 2 others commands that are already thread isolated

But still I don't get it :-( What are those two other commands?

Configured this way, Zuul can only sched load but does not allow for timing out and let the client walk away. In short, even if the Hystrix timeout is set to 1000ms, clients will only be released once the call forwarded to the service down the chain returns (or timeouts because of a ReadTimeout for instance).

I tried to force THREAD isolation by overriding the configuration (per service unfortunately, since the default is forced in the code) and everything seems to work as expected. However, I'm not keen in doing this without proper understanding of the implications - certainly in regards of the comment found in the code and the defaults adopted by the Spring Cloud version of Zuul.

Can someone provide more information? Thx

Answer 1

The Hystrix documentation has a good example of why semaphore isolation is appropriate when wrapping commands that are thread isolated. Specifically, it says:

The façade HystrixCommand can use semaphore-isolation since all of the work it is doing is going through two other HystrixCommands that are already thread-isolated. It is unnecessary to have yet another layer of threading as long as the run() method of the façade is not doing any other network calls, retry logic, or other “error prone” things.

Update: The question mentions that thread isolation has to be configured for each service, but I found out that you can control the isolation of all Hystrix commands (including RibbonCommands) by setting the following property:

hystrix.command.default.execution.isolation.strategy = THREAD

Answer 2

This pattern is defined in the Hystrix documentation

The façade HystrixCommand can use semaphore-isolation since all of the work it is doing is going through two other HystrixCommands that are already thread-isolated. It is unnecessary to have yet another layer of threading as long as the run() method of the façade is not doing any other network calls, retry logic, or other “error prone” things.

The reason why we only use semaphore is because

1. We want to throttle number of requests to primary & secondary (may be more) combined
2. Since isolation is already achieved by the actual hystrix command thread pools (that this facade proxies), we dont need to need to worry about isolatation at this level using threadpools (each thread in the pool has a fixed overhead interms of resource consumption, unlike semaphore which is just a counter). So, lightweight semaphore is good enough

Reference: https://github.com/Netflix/Hystrix/wiki/How-To-Use#primary--secondary-with-fallback