Rails: Improve performance of simple searching method

问题

I have been building an app these days. The functionality is nothing fancy at all, I have to connect to a client's SOAP webservice, fetch some data, save it into my pg database and build a search functionality based on this data.

The search has to be performed on two tables, both combined are like 80K rows. It needs to look for every word in the input text in several fields from these two tables, which have a classical assocation one to many.

Previous to get my hands dirty I was looking at the choices I had to get the functionality done (ransack, searchkick, scoped_search etc), but I ended up trying first just vanilla Active Record and I was very surprised to find that I could achieve the functionality way easier than I thought and with an acceptable response time, about to 400ms active record time for the most expensive queries in local.

So the problem is, the performance of this app in Heroku is way worse than in local (I'm developing using a vagrant box btw). On average, queries take 2-3 times longer than in local, so the user experience goes from acceptable to poor. I was wondering If someone could help to improve my query. I'm also worried about how the background job that fetchs the data is also way les performant than in local and about some issues with the memory, but that's a different story though.

The relevant snippets are these:

part_master.rb where the search method is implemented:

class PartMaster < ApplicationRecord
has_many :part_variants, foreign_key: 'sap_cod', primary_key: 'sap_cod'
has_many :locations, foreign_key: 'sap_cod', primary_key: 'sap_cod'

scope :con_stock, -> { where("stock > 0") }
scope :planta, -> (planta) { where planta_cod: planta}

def self.search(params)
  recordset = PartMaster.joins(:part_variants).all
  recordset = recordset.con_stock if params[:stock].present?
  recordset = recordset.planta(params[:planta]) if params[:planta].present?
  recordset = search_keywords(params[:search], recordset)
  recordset
end

private 

def self.search_keywords(query, recordset)
  keywords = query.to_s.strip.split
  if query
    keywords.each do |keyword|
      recordset = recordset.where('part_masters.sap_cod ILIKE :q OR 
                          unaccent(descripcion_maestro) ILIKE unaccent(:q)
                          OR fabricante ILIKE :q OR ref_fabricante ILIKE :q 
                          OR fabricante_prov ILIKE :q OR ref_prov ILIKE :q', 
                          q: "%#{keyword}%")
    end
    recordset.distinct.order(:sap_cod)
   end
 end
end

And this is the call to the method from the controller:

  def index
   parts = params[:search].present? ? PartMaster.search(params) : 
           PartMaster.none
   @parts = parts.page(params[:page]).per(50)
  end

I have an index in every searchable field.

EDIT: Finally I have tried a mix of the proposal in the answers. I have created one field in each table that is a concatenation of the relevant fields for the search, having so 2 OR statements instead of 5, and I also have put trigram GIN indexes in both new fields. I haven't seen any improvement though, the times corresponding to ActiveRecord are very similar, perhaps marginally better.

The thing is, the output for the query using EXPLAIN dones't show any info about the indexes being used.

Hash Join  (cost=2243.29..6067.41 rows=2697 width=132)
Hash Cond: ((part_variants.sap_cod)::text = (part_masters.sap_cod)::text)
Join Filter: ((part_masters.combinada_maestro ~~* '%rodamiento%'::text)         OR (part_variants.combinada_info ~~* '%rodamiento%'::text))
->  Seq Scan on part_variants  (cost=0.00..1264.96 rows=54896 width=18)
->  Hash  (cost=1128.13..1128.13 rows=34813 width=132)
     ->  Seq Scan on part_masters  (cost=0.00..1128.13 rows=34813             width=132)
(6 rows)

回答1:

Suggestions to improve the AR query speed use a direct Postgresql query in your model

Example for in you keywords loop

query = "SELECT * FROM part_masters WHERE......"

PartMaster.connection.execute(query, :skip_logging)

回答2:

I agree with spikermann. Also the multiple ORs in a loop is not helping neither.

If you only want to work on a vanilla solution vs adding SOLR or any other engine, you could have one separate field to hold copies of the strings that you would like to search. (ex. name, description, ...). The search this field only. You will need some method to keep the field updated when the name, description or other values change.

来源：https://stackoverflow.com/questions/49629107/rails-improve-performance-of-simple-searching-method