Paramiko Fails to download large files >1GB

Question

def download():
if os.path.exists( dst_dir_path ) == False:
    logger.error( \"Cannot access destination folder %s. Please check path and permissions. \" % ( dst_di         


        

Answer 1

I have try to trace the code into paramiko, now I'm sure it is the server problem.

1. What prefetch has done

In order to increase the download speed, paramiko try prefetch the file by fetch method.When SFTP_FILE.prefetch() method is called, a new thread is create and tons fetch request will send to server util the whole file is covered.
we can find this in file paramiko/sftp_file.py around line 464.

2. How to sure is the server problem

The request mention above is run in async mode. SFTP_FILE._async_response() is used to receive the response from the server async.And trace down the code, we can find this exception is created in method SFTP_FILE._async_response() which convert from the message sent from the server. Now, We can sure that is the exception from server.

3. How to solve the problem

Because I have no access to the server, so use sftp in command line is my best choice.But on the other hand, now we know that too many request makes the server crash, so we can make a sleep when sending the request to server.