Making multiple HTTP requests asynchronously

Question

require \'net/http\'

urls = [
  {\'link\' => \'http://www.google.com/\'},
  {\'link\' => \'http://www.yandex.ru/\'},
  {\'link\' => \'http://www.baidu.com/\'}
]

ur         


        

Answer 1

I have written an in-depth blog post about this topic which includes an answer that is somewhat similar to the one August posted - but with a few key differences: 1) Keeps track of all thread references in "thread" array. 2) Uses "join" method to tie up threads at the end of program.

require 'net/http'

# create an array of sites we wish to visit concurrently.
urls = ['link1','link2','link3']  
# Create an array to keep track of threads.
threads = []

urls.each do |u|  
  # spawn a new thread for each url
  threads << Thread.new do
  Net::HTTP.get(URI.parse(u))
    # DO SOMETHING WITH URL CONTENTS HERE
    # ...
    puts "Request Complete: #{u}\n"
  end
end

# wait for threads to finish before ending program.
threads.each { |t| t.join }

puts "All Done!"  

The full tutorial (and some performance information) is available here: https://zachalam.com/performing-multiple-http-requests-asynchronously-in-ruby/