Index the results of a method in ElasticSearch (Tire + ActiveRecord)

Question

I\'m indexing a data set for elasticsearch using Tire and ActiveRecord. I have an Artist model, which has_many :images. How can I index a method of the Artist model which re

Answer 1

So, to include your solution to the indexing problem here.

Indexing associations

One way to index a method is to include it in the to_json call:

def to_indexed_json
  to_json( 
    :only   => [ :id, :name, :normalized_name, :url ],
    :methods   => [ :primary_image_original, :primary_image_thumbnail, :account_balance ]
  )
end

Another one, and more preferable, is to use the :as option in the mapping block:

mapping do
  indexes :id, :index    => :not_analyzed
  indexes :name             
  # ...

  # Relationships
  indexes :primary_image_original, :as => 'primary_image_original'
  indexes :account_balance,        :as => 'account_balance'
end

Fighting n+1 queries when importing

The problem with slow indexing is most probably due to n+1 queries in the database: for every artist you index, you issue a query for images (original and thumbnail). A much more performant way would be to join the associated records in one query; see Eager Loading Associations in Rails Guides.

The Tire Index#import method, and the import Rake task, allow you to pass parameters which are then sent to the paginate method down the wire.

So let's compare the naive approach:

bundle exec rake environment tire:import CLASS=Article FORCE=true
Article Load (7.6ms)  SELECT "articles".* FROM "articles" LIMIT 1000 OFFSET 0
Comment Load (0.2ms)  SELECT "comments".* FROM "comments" WHERE ("comments".article_id = 1)
Comment Load (0.1ms)  SELECT "comments".* FROM "comments" WHERE ("comments".article_id = 2)
...
Comment Load (0.3ms)  SELECT "comments".* FROM "comments" WHERE ("comments".article_id = 100)

And when we pass the include fragment:

bundle exec rake environment tire:import PARAMS='{:include => ["comments"]}'  CLASS=Article FORCE=true 
Article Load (8.7ms)  SELECT "articles".* FROM "articles" LIMIT 1000 OFFSET 0
Comment Load (31.5ms) SELECT "comments".* FROM "comments" WHERE ("comments".article_id IN (1,2, ... ,100))

Much better :) Please try it out and let me know if it solves your issue.

---

You can also try it out in the Rails console: Article.import vs. Article.import(include: ['comments']). As a side note, this exact problem was the reason for supporting the params hash in the whole importing toolchain in Tire.