Repeated POST request is causing error “socket.error: (99, 'Cannot assign requested address')”

Question

I have a web-service deployed in my box. I want to check the result of this service with various input. Here is the code I am using:

import sys
import httpli         


        

Answer 1

I too faced similar issue while executing multiple POST statements using python's request library in Spark. To make it worse, I used multiprocessing over each executor to post to a server. So thousands of connections created in seconds that took few seconds each to change the state from TIME_WAIT and release the ports for the next set of connections.

Out of all the available solutions available over the internet that speak of disabling keep-alive, using with request.Session() et al, I found this answer to be working which makes use of 'Connection' : 'close' configuration as header parameter. You may need to put the header content in a separte line outside the post command though.

headers = {
        'Connection': 'close'
}
with requests.Session() as session:
response = session.post('https://xx.xxx.xxx.x/xxxxxx/x', headers=headers, files=files, verify=False)
results = response.json()
print results

This is my answer to the similar issue using the above solution.