Is Spring an option for you? Spring-MVC 3.2 has a class called DeferredResult, which will gracefully handle your "10,000 open connections/10 server pool threads" scenario.
Example: http://blog.springsource.org/2012/05/06/spring-mvc-3-2-preview-introducing-servlet-3-async-support/