Reusing http connections in Golang

问题:

I'm currently struggling to find a way to reuse connections when making HTTP posts in Golang.

I've created a transport and client like so:

// Create a new transport and HTTP client tr := &http.Transport{} client := &http.Client{Transport: tr} 

I'm then passing this client pointer into a goroutine which is making multiple posts to the same endpoint like so:

r, err := client.Post(url, "application/json", post) 

Looking at netstat this appears to be resulting in a new connection for every post resulting in a large number of concurrent connections being open.

What is the correct way to reuse connections in this case?

回答1:

You should ensure that you read until the response is complete before calling Close().

e.g.

res, _ := client.Do(req) io.Copy(ioutil.Discard, res.Body) res.Body.Close() 

To ensure http.Client connection reuse be sure to do two things:

• Read until Response is complete (i.e. ioutil.ReadAll(resp.Body))
• Call Body.Close()

回答2:

If anyone is still finding answers on how to do it, this is how I am doing it.

package main  import (     "bytes"     "io/ioutil"     "log"     "net/http"     "time" )  var (     httpClient *http.Client )  const (     MaxIdleConnections int = 20     RequestTimeout     int = 5 )  // init HTTPClient func init() {     httpClient = createHTTPClient() }  // createHTTPClient for connection re-use func createHTTPClient() *http.Client {     client := &http.Client{         Transport: &http.Transport{             MaxIdleConnsPerHost: MaxIdleConnections,         },         Timeout: time.Duration(RequestTimeout) * time.Second,     }      return client }  func main() {     var endPoint string = "https://localhost:8080/doSomething"      req, err := http.NewRequest("POST", endPoint, bytes.NewBuffer([]byte("Post this data")))     if err != nil {         log.Fatalf("Error Occured. %+v", err)     }     req.Header.Set("Content-Type", "application/x-www-form-urlencoded")      // use httpClient to send request     response, err := httpClient.Do(req)     if err != nil && response == nil {         log.Fatalf("Error sending request to API endpoint. %+v", err)     } else {         // Close the connection to reuse it         defer response.Body.Close()          // Let's check if the work actually is done         // We have seen inconsistencies even when we get 200 OK response         body, err := ioutil.ReadAll(response.Body)         if err != nil {             log.Fatalf("Couldn't parse response body. %+v", err)         }          log.Println("Response Body:", string(body))     }  } 

Go Playground: http://play.golang.org/p/oliqHLmzSX

In summary, I am creating a different method to create a HTTP client and assigning it to global variable and then using it to make requests. Note the

defer response.Body.Close()  

This will close the connection and set it ready for reuse again.

Hope this will help someone.

回答3:

Edit: This is more of a note for people that construct a Transport and Client for every request.

Transport is the struct that holds connections for re-use:

http://golang.org/src/pkg/net/http/transport.go#L46

So if you create a new Transport for each request, it will create new connections each time. In this case the solution is to share the one Transport instance between clients.

回答4:

IIRC, the default client does reuse connections. Are you closing the response?

Callers should close resp.Body when done reading from it. If resp.Body is not closed, the Client's underlying RoundTripper (typically Transport) may not be able to re-use a persistent TCP connection to the server for a subsequent "keep-alive" request.

文章来源: Reusing http connections in Golang