Are you accessing your site through a proxy? If so, you may be seeing better performance because the proxy is being bypassed or reduced to just using handling the initial CONNECT requests.
A proxy could be checking and caching content when you use HTTP - leading to reduced performance.