sidekiq

My rails app have one sidekiq worker. The worker will do 2500 requests to a api external, the response is a xml. How get best performance for this worker? Inside the worker, spawn application level Threads. For example, Create 10 ruby threads to process the 2500 external api requests(means each ruby thread will process 250 requests). # threads will contain the threads threads = [] external_requests.each_slice(250) do |group| threads << Thread.new(group) do |tsrc| tsrc.each do |ex_request| # Do your external call here end end end threads.each(&:join) # wait for all threads to finish 来源： https:/

I am using Sidekiq for my background jobs: I have a worker app/workers/data_import_worker.rb class DataImportWorker include Sidekiq::Worker sidekiq_options retry: false def perform(job_id,file_name) begin #Some logic in it ..... end end Called from a file lib/parse_excel.rb def parse_raw_data #job_id and #filename are defined bfr DataImportWorker.perform_async(job_id,filename) end As soon as i trigger it from my action the worker is not getting called.. Redis is running on localhost:6379 Any idea why this must be happening. The Environment is Linux. basgys I had a similar problem where Sidekiq

I had huge issues with starting sidekiq on Heroku after updating my gems and putting everything into production. The problem was that Sidekiq tried to connect to Redis on a local connection and port, instead of using the REDISTOGO variable. After spending a few hours, I managed to fix it: Answer below. Remove everything from the if and below and run this: heroku config:set REDIS_PROVIDER=REDISTOGO_URL Sidekiq will automatically use it. I looked up the correct, new connection that RedisToGo provides and then inserted it into the variables. Some posts here on SO claimed that this wasn't

I'm trying to use a sidekiq worker , which more or less saves a image file to database (using carrierwave). There are few files to save, which are a keyframes extracted from a video file. That's what that worker is about. My image uploader has a few versions defined and looks as follows: class KeyframeUploader < CarrierWave::Uploader::Base # ... # Keyframe thumbnail sizes version :small do process resize_to_fill: [180, 180], if: :square? process resize_to_fill: [320, 180], if: :not_square? end version :medium do process resize_to_fill: [460, 460], if: :square? process resize_to_fill: [640, 460

I get the following warning with Rails 4 and Sidekiq on os x on development 10:13:39 worker.1 | 2014-09-22T07:13:39.857Z 86981 TID-oug0oog10 WARN: could not obtain a database connection within 5.000 seconds (waited 5.002 seconds) 10:13:39 worker.1 | 2014-09-22T07:13:39.857Z 86981 TID-oug0oog10 WARN: /Users/me/.rvm/gems/ruby-2.1.3/gems/activerecord-4.1.6/lib/active_record/connection_adapters/abstract/connection_pool.rb:190:in `block in wait_poll' I read other answers that say to reduce the concurrency I give to sidekiq to allow more for other things, but with worker: bundle exec sidekiq -c 10

I have a Rails 4 application that I use in conjunction with sidekiq to run asynchronous jobs. One of the jobs I normally run outside of my Rails application is a large set of complex SQL queries that cannot really be modeled by ActiveRecord. The connection this set of SQL queries has with my Rails app is that it should be executed anytime one of my controller actions is invoked. Ideally, I'd queue a job from my Rails application within the controller for Sidekiq to go ahead and run the queries. Right now they're stored in an external file, and I'm not entirely sure what the best way is to have

I have looked at multiple sources and tried various scenarios but couldn't resolve this hence the issue. Please point me in the right direction. Like everybody I have 3 env (development, staging and production). I have the following in my sidekiq.yml file # Options here can still be overridden by cmd line args. # sidekiq -C config.yml --- :verbose: false :namespace: xyz :logfile: log/sidekiq.log :concurrency: 25 :strict: false :pidfile: tmp/pids/sidekiq.pid :queues: - [stg_xyz_tests_queue, 10] - [stg_default_xyz_queue, 2] - [stg_xyz_default_queue, 3] development: :verbose: true :concurrency:

(See below for my detailed config, which is the result of Henley Chiu's answer). I've been trying to wrap my brain around Sidekiq deploys, and I am not really getting it. I have an app with a staging environment, and a production environment, on the same server. Everything I see about sidekiq deploys basically say "just add sidekiq/capistrano to your deploy file", so I did that. And then the instructions are "here's a yml file with options" but nothing seems to be explained. Do I need namespaces? I see that in an initialize file, but that seems to be to point outside the server. I deployed