Tweaking magento for performance

Question

i\'m looking on performance (server load time) of magento site and i\'m trying to tune search result pages. I realized that when I disabled all heavy things like top navigat

Answer 1

As Vinai said, Magento is all about extensibility and raw performance is secondary but remedied by things like indexing and caching. Significantly improving performance without caching is going to be very difficult. Short of full-page caching, enabling block caching is a good method of improving performance but proper cache invalidation is key. Many blocks are cacheable but not already configured to be cached by default so identify the slowest ones using profiling use Vinai's guide for enabling caching. Here are a few additional things to keep in mind with block caching:

• Any block that lists product info should have the product's tag which is 'catalog_product_'.$productId. Similarly, use 'catalog_category_'.$categoryId for categories. This will ensure the cache is invalidated when the product or category is saved (edited in backend). Don't set these in the constructor, set them in an overridden getCacheTags() so that they are only collected when the block is saved and not when it is loaded from cache (since that would defeat the purpose of caching it).
• If you use https and the block can appear on an https page and includes static resources, make sure the cache key includes Mage::app()->getRequest()->isSecure() or else you'll end up with http urls on https pages and vice versa.
• Make sure your cache backend has plenty of capacity and avoid needless cache flushes.
• Don't cache child blocks of a block that is itself cached unless the parent changes much more frequently than the child blocks or else you're just cluttering your cache backend.
• If you do cache tagging properly you should be able to use a very long default cache lifetime safely. I believe setting "false" as the lifetime actually uses the default, not infinite. The default is 7200 seconds but can be configured in local.xml.
• Using the redis backend in most cases will give you the best and most consistent performance. When using Redis you can monitor used memory size using this munin plugin.

Answer 2

Make sure mysql query cache is turned on. And set these variables in mysql (maybe need tweaking depending on your setup).

query_cache_type=1
query_cache_size=64M

Answer 3

First you need to audit and optimize time to first byte (TTFB).

Magento has profiler built-in that will help you identify unoptimized code blocks.

Examine your template files and make sure you DO NOT load product models inside a loop (common performance hog):

foreach($collection as $_product){
   $_product = Mage::getModel('catalog/product')->load($_product->getId()

I see this code often in product/list.phtml

I wrote a step-by-step article on how to optimize TTFB

Disclaimer: the link points to my own website

Answer 4

Magento is a extremely flexible ecommerce framework, but that flexibility comes with a price: performance. This answer is a collection of pointers and some details on caching (especially for blocks).

One thing to consider is the Magento environment, e.g. tuning the php, the web server (favor nginx over Apache), and MySQL. Also, set up a good caching backend for Magento. All these are covered e.g. in the Magento Performance Whitepaper that applies also to the CE.

After the environment is set up, the other side of things is the code.
Reducing the number of queries is possible for some pages by enabling the flat table catalog (System > Configuration > Catalog > Frontend), but you will always have a high number of queries.

You also can't really reduce the time spent creating the blocks except by tuning the environment (APC, memory, CPU). So as the other commenters said, your best choice is utilizing the caching functionality that Magento has built in.

Magento Block Caching

Because you specifically mentioned blocks in the question, I'll elaborate a bit more on block caching. Block caching is governed by three properties:

1. cache_lifetime
2. cache_key
3. cache_tags

All these properties can be set in the _construct() method of a block using setData() or magic setters, or by implementing the associated getter methods (getCacheLifetime(), getCacheKey(), getCacheTags()).

The cache_lifetime is specified in (integer) seconds. If it is set to false(boolean), the block will be cached for ever (no expiry). If it is set to nullthe block will not be cached (this is the default in Mage_Core_Block_Abstract).

The cache_key is the unique string that is used to identify the cache record in the cache pool. By default it is constructed from the array returned by the method getCacheKeyInfo().

// Mage_Core_Block_Abstract
public function getCacheKeyInfo()
{
    return array(
        $this->getNameInLayout()
    );
}

public function getCacheKey()
{
    if ($this->hasData('cache_key')) {
        return $this->getData('cache_key');
    }
    /**
     * don't prevent recalculation by saving generated cache key
     * because of ability to render single block instance with different data
     */
    $key = $this->getCacheKeyInfo();
    //ksort($key);  // ignore order
    $key = array_values($key);  // ignore array keys
    $key = implode('|', $key);
    $key = sha1($key);
    return $key;
}

The best way to customize the cache key in custom blocks is to override the getCacheKeyInfo() method and add the data that you need to uniquely identify the cached block as needed.

For example, in order to cache a different version of a block depending on the customer group you could do:

public function getCacheKeyInfo()
{
    $info = parent::getCacheKeyInfo();
    $info[] = Mage::getSingleton('customer/session')->getCustomerGroupId()
    return $info;
}

The cache_tags are an array that enable cache segmentation. You can delete sections of the cache matching one or more tags only.
In the admin interface under System > Cache Management you can see a couple of the default cache tags that are available (e.g. BLOCK_HTML, CONFIG, ...). You can use custom cache tags, too, simply by specifying them.
This is part of the Zend_Cache implementation, and needs to be customized far less frequently compared to the cache_lifetime and the cache_key.

Other Caching

Besides blocks Magento caches many other things (collection data, configuration, ...).
You can cache your own data using Mage::app()->saveCache(), Mage::app()->loadCache(), Mage::app()->cleanCache() and Mage::app()->removeCache(). Please look in Mage_Core_Model_App for details on these methods, they are rather straight forward.

You will also want to use a full page cache module. If you are using the Magento EE, you already have one. Otherwise search Magento Connect - there are many options (commercial).
Some of those modules also tune various parts of Magento for you beyond the full page caching aspect, e.g. Nitrogento (commercial).

Using a reverse proxy like Varnish is also very beneficial.

There are quite a number of blog posts on this subject. Here is one post by the publishers of the Nitrogento extension.
If you are running Magento on a more low-scale environment, check out my post on the optimization of the file cache backend on magebase.com.

Answer 5

Just to follow on from Mark... most of the tables in the Magento database are InnoDB. Whilst the query cache can be used in a few specific places, the following are more directly relevant...

innodb_buffer_pool_size
innodb_thread_concurrency
innodb_flush_method
innodb_flush_log_at_trx_commit

I also use

innodb_file_per_table

as this can be beneficial in reorganising specific tables.

If you give the database enough resource, (within reason) the amount of traffic really doesn't load the server up at all as the majority of queries are repeats anyway, and are delivered out of database cache.

In other words, you're probably worrying about nothing...

Answer 6

i found a very interesting blog post about Magento Performance optimization, there are many configuration settings for your server and your magento store, was very helpful for me.

http://www.mgt-commerce.com/blog/magento-on-steroids-best-practice-for-highest-performance/