The generic approach to solving Magento speed problems is well known: cache, cache, cache. But while debugging a standard Magento 1.5.1.0 installation, I’ve discovered some peculiar behaviour: some type of caching will actually slow down your site.

The problem occurs when you have configured a RAM based block cache, such as memcache or APC (you should, because this type is a magnitude faster than the standard disk or database caches!). You might have noticed, that even though you are using RAM, Magento will still write files to its var/cache/ folder. The reason for this is that Magento automatically engages the Two Level caching mechanism (Zend reference manual). Magento will write only parts of the cache fragments to the fast cache (APC) and write everything to the slow cache (default: file storage).

This is a good strategy if your fast cache has a limited amount of RAM. But since memory is very cheap nowadays, chances are that you can fit your whole shop many times in the fast cache and you would never have to rely on the slow cache. And writing to the slow cache only wastes unnecessary CPU cycles.

I found out by stracing the Magento server process and adding some debug output to the various cache handlers. Without slow_backend defined, you will get this:

Unfortunately, there is no configuration option to disable the Two Level caching mechanism. You could modify the code, in which case you should edit app/code/core/Mage/Core/Model/Cache.php and set  $enable2levels to false somewhere around line 158.

Luckily there’s an easier approach as well. An undocumented feature exist, called “slow_backend_store_data”. You can find its syntax in local.xml.additional and you should set it to “0″. As its name implies, it prevents the 2nd level cache from writing (effectively disabling the 2nd level cache).

But! This only works if you have set slow_backend to “database” (see line 254 of app/code/core/Mage/Core/Model/Cache.php). What is the logic of this? I don’t know. But it means that if you did not define slow_backend in the configuration, it will default to File and hence the slow_backend_store_data setting will be ignored.

To summarize: Two Level cache incurs unnecessary overhead if your primary RAM cache is large enough. You can disable it by pointing your slow backend to the database and disabling slow data store.

Here’s the relevant part from my local.xml config:

Note: if you’re hosted with Byte, you should substitute Apc with memcached.

More info on Two Level Cache and its problems can be found in this excellent post by Fabrizio

  • Cipriano Groenendal

    Hey Willem,

    Quite interesting to see something I’ve had a hunch about finally confirmed. Any chance for an update as to what happens if you set both the slow and fast backend to memcache? It might mean that everything gets written to memory twice, but does it increase the speed since they won’t need to write to the disk anymore?

  • http://www.customerparadigm.com/index/608/magento-ecommerce-programmer.php Magento Website

    Great post! We have actually had issues with slow cacheing in the past… thanks for the tips! Keep it up!

  • Chris Forscutt

    I have just turned on APC for my site and used this tip to turn off the 2 level caching, and it worked like a charm, thanks.

    Do you know how often I should flush the APC cache, and how I could automate this process if needed?

  • Mark van der Sanden

    I have read a lot about Magento caching in the last few days. From what I understand, it’s really not a good idea to disable part of the two level caching. It serves its purpose! This purpose is handling cache tags. APC and Memcached backends cannot support tags (as they just store key/value pairs), so all operations including tags (like retrieving or cleaning entries by tag) will be done on disk. Disabling this will most likely kill all tag related functionality in Magento.

    I’ve read some really promising results of using the Redis backend (also, Vinai explains the whole caching system and a system he has thought out to improve cache tag handling in the file system):
    http://magebase.com/magento-tutorials/improving-the-file-cache-backend/

  • Willem

    @Mark: Thanks for the link! As I understand it, the penalty of killing tag functionality is a slower cache purge when stuff changes. I would say that cache purges happen less often than regular hits, so it’s a trade-off between incidental slower pages and faster average loading times. Would be nice to measure the amount of cache flushes vs regular pageviews on a live magento shop.

  • Mark van der Sanden

    @Willem: Not really. The penalty of killing tag functionality is that when Magento asks to invalidate some part of the cache (for instance, after saving a product or clear ‘block html output’ from the backend), nothing happens. So you end up with old data on your webshop.

    For instance, at a client’s webshop we have a block that shows 2 products from a category. It caches the resulting html output for speeding up page generation. If the category gets saved, we’ll clean the block’s cached output so the contents get regenerated. Without using the file backend as slow backend (or some other backend facilitating tags functionality, but at the moment DB is the only alternative you have available), the cache entries matching the category’s tag cannot be looked up and will not be invalidated.

    I only see one alternative: cleaning the whole cache each time Magento requests to clean cache items associated with one or some particular tags. And I suppose that that happens too often. Imagine some ERP connector that saves some product data every 5 minutes. That means you have to rebuild your *whole* cache (thousands of files) every 5 minutes!

  • http://netzarbeiter.com/ Vinai

    Hi William,
    thanks for the post! I just found this article through http://www.fabrizio-branca.de/magento-zend-frameworks-twolevels-cache-backend-mess.html
    Fabricio does a great job at explaining the details and problems of the Zend Framework Two Level Cache.

    @Mark: The slow_backend_store_data option only disables the data storage in the database backend, the cache tags are still handled by it (reference Varien_Cache_Backend_Database->save(), the tags are saved regardless of the ‘store_data’ setting).

    This logic should be implemented in the TwoLevel cache backend which would be the proper place for it instead of the slow cache backend (which would require a change to the Zend_Cache_Backend_ExtendedInterface, e.g. saveTags() or saveMetaData())

  • http://www.facebook.com/aredamkrik Kirk Madera

    This did not work for me, but I did discover a solution. I am running Magento EE 1.11.0

    local.xml setting:

    apc
    MAGE_
    Zend_Cache_Backend_Apc

    The code expects the full class name “Zend_Cache_Backend_Apc” rather than just “apc”

  • Jason Porter

    Since RAM is cheap, as you said, why not just mount /var to tmpfs and run the whole var tree, cache included, in RAM? It works very well in our config, with both APC and the “slow” filesystem cache being served from RAM.

  • Maier

    What about what did you set that to?

  • shirtsofholland

    Great article. But now the biggie.

    What are the settings when you have both APC and MEMCACHED installed?

  • shirtsofholland

    This article is quite old now.

    How would this work with Redis installed? and how does that compare to tmpfs solution?

    Would one store all cache + session in Redis? and would this make memcache obsolete: I understand memcache also caches database data?