I'm curious if anyone noticed any scaling issues with Sunspot-Solr . Even if I remove all the searchable params, and its just counting against the raw class by itself; it still takes 5 to 8 seconds to load on my local, 4 to 5 seconds on production.
Has anyone else been able to scale Sunspot-Solr ? What are some common problems?
How can one look into this deeper?
Here's the Solr Logs for a single request :
Solr Select (208.1ms) {:rows=>20, :start=>0, :q=>"*:*", :sort=>"score desc", :fq=>["type:Organization", "published_b:true", "updated_at_d:[2009\\-02\\-03T16\\:11\\:55Z TO *]"]}
Solr Select (5.5ms) {:rows=>20, :start=>0, :q=>"*:*", :sort=>"score desc", :fq=>["type:Organization", "published_b:true", "updated_at_d:[2009\\-02\\-03T16\\:11\\:55Z TO *]"]}
Solr Update (12.6ms) <?xml version="1.0" encoding="UTF-8"?><add><doc><field name="type">User</field><field name="type">ActiveRecord::Base</field><field name="id">User 2</field><field name="class_name">User</field><field name="first_name_s">Bob</field><field name="created_at_d">2009-09-28T21:00:27Z</field><field name="last_name_s">Marley</field><field name="email_s">bob.marley@gmail.com</field><field name="name_s">Bob Marley</field><field name="last_name_text">Marley</field><field name="first_name_text">Bob</field><field name="email_text">bob.marley@gmail.com</field><field name="name_text">Bob Marley</field></doc></add>
Solr Update (487.7ms) <?xml version="1.0" encoding="UTF-8"?><commit/>
Completed in 12632ms (View: 11633, DB: 228) | 200 OK [http://localhost/organizations/search]
1000 objects is child's play for Solr, so there's something fishy going on here with the ~200ms Solr read. However, your most immediate problem is that you're writing to Solr during what appears to be a GET request -- what's up with that? Are you saving a searchable object, which is triggering Sunspot's auto-index? If you have the need to update models during the course of a GET request (which should probably be done in a background job if possible), you'll want to disable auto-indexing in Sunspot:
searchable :auto_index => false
# sunspot setup
end
Then you'd need to explicitly call my_model.index in your controllers when you actually do want to update them in Solr.
Finally, that big update at the end is a Solr commit, which tells Solr to write unstaged changes to disk and load up a new searcher that reflects those changes. Commits are expensive; Sunspot::Rails by default performs a commit at the end of any request that writes to Solr, but this behavior is targeted at the principle of least surprise for new users of Sunspot rather than a live app in production. You'll want to disable it in your config/sunspot.yml:
auto_commit_after_request: false
You'll then probably want to configure autoCommit in your solr/conf/solrconfig.xml -- it's commented out in the default Sunspot Solr distribution, and there is an explanation in there too. I've found that once a minute is a good place to start.
After making those changes, I'd see if your reads are still slow -- I think it's quite possible that the cause of that is that every time you search, your write/commit to Solr is causing it to have to load up a fresh searcher from disk. So, it can't allow any of its internal caches to warm, etc., and is generally under a tremendous amount of strain.
Hope that helps!
When I experienced long requests time during update commits I stumbled upon this blog
and it turned out that I had to change the following:
Performance killer:
<str name="buildOnCommit">true</str>
Way to go:
<str name="buildOnCommit">false</str>
in solrconfig.xml
来源：https://stackoverflow.com/questions/4887142/sunspot-solr-slowing-down-to-a-beast-once-my-application-climbed-to-1000-objec