For most things I'd say that HttpClient is the way to go. However there are some situations and edge cases where I'd fall back to a URLConnection. Examples of edge cases here and here
EDIT
A similar question has been asked before: httpclient vs httpurlconnection. I would assume that HttpUrlConnection is somewhat faster as the HttpClient is built on top of the standard Java libraries. However I would find HttpClient code much quicker and easier to write and maintain. According to a comments below, the core elements of HttpClient have been performance optimised.
If performance is a major concern your best bet is to write two clients, one using each method, then benchmark them both. If you do this, please let us know the results.