Ruby on Rails Multiple HTTP request at the same time?

Question

I'm pulling multiple requests (its pulling one at a time), I was wondering if there's a way pull requests all at the same time if I have something like this:

client = Instagram.client(:access_token => session[:access_token])
@user = client.user
@recent_media_items = client.user_recent_media

@lv = client.tag_recent_media('lv', options = {:count => 60})
@lv1 = client.tag_recent_media('lv1', options = {:count => 60})
@lv2 = client.tag_recent_media('lv2', options = {:count => 60})
@lv3 = client.tag_recent_media('lv3', options = {:count => 60})

Each lv makes a request to client. I was wondering if there's a way to do such so it can do the request all at once (together), instead of one finishes the request, then goes on to the next request, and so on...

Thanks!

Answer 1

Yes! For a proof-of-concept, try

require 'thread'

client = Instagram.client(:access_token => session[:access_token])
@user = client.user
@recent_media_items = client.user_recent_media

threads = []
threads << Thread.new { @lv = client.tag_recent_media('lv', options = {:count => 60}) }
threads << Thread.new { @lv1 = client.tag_recent_media('lv1', options = {:count => 60}) }
threads << Thread.new { @lv2 = client.tag_recent_media('lv2', options = {:count => 60}) }
threads << Thread.new { @lv3 = client.tag_recent_media('lv3', options = {:count => 60}) }
threads.each(&:join) # this waits for all the threads to finish before proceeding
puts [@lv, @lv1, @lv2, @lv3]

In practice, you will want to set up some error handling and retry settings within the threads. Also, you may run into issues with thread-safety in the Instagram gem. If you are doing this on a large scale with hundreds or thousands of requests, you may want to try a concurrent HTTP client like Typhoeus or an evented HTTP client like EM-HTTP-Request. For these, you would have to manually implement the tag_recent_media method that is in the Instagram gem.

Answer 2

There are gems like spawn that do this in a forked process or as a new thread in your process.. Here is a link to another SO post that discusses it

What is the difference between forking and threading in a background process?

Here is the gem in question: https://github.com/tra/spawnling