Ruby Net::FTP Timeout Threads

Question

I was trying to speed up multiple FTP downloads by using threaded FTP connections. My problem is that I always have threads hang. I am looking for a clean way of either te

Answer 1

The trick for me that worked was to use ruby's Timeout.timeout to ensure the FTP connection was not hanging.

begin
    Timeout.timeout(10) do
        ftp.getbinaryfile(rmls_path, local_path)
    end
    # ...
rescue Timeout::Error
    errors << "#{thread_num}> File download timed out for: #{rmls_path}"
    puts errors.last
rescue
    errors << "unable to get file > ftp reponse: #{ftp.last_response}"
    # ...
end

Hanging FTP downloads were causing my threads to appear to hang. Now that the threads are no longer hanging, I can use the more proper way of dealing with threads:

threads.each { |t| t.join }

rather than the ugly:

# If @last_updated has not been updated on the server in over 20 seconds, wait 3 seconds and check again
while Time.now < @last_updated + 20 do
    sleep 3
end
# threads are hanging so joining the threads does not work.
threads.each { |t| t.kill }