nginx return corrupted data from memcached

问题

On my web site i've made data caching with memcached. It stores fully generated html pages. Next step was to get this data from memcached by nginx and send back to user w\o starting apache process. First i tried to get data from cache by php backend and it worked. But when i try make this with nginx - i see hardly corrupted data. smth like

i'm asking for help with this problem.

p.s. here the part of nginx config if it can help

location / {
    #add_header    Content-Type  "text/html";
    set $cachable 1;

    if ($request_method = POST){
        set $cachable 0;
        break;
    }
    if ($http_cookie ~ "beauty_logged") {
        set $cachable 0;
        break;
    }

    if ($cachable = 1) {
        set             $memcached_key   'nginx_$host$uri';
        memcached_pass  127.0.0.1:11211;
    }

    default_type    text/html;
    error_page      404 502 504 405 = @php;
    #proxy_pass http://front_cluster;
}

location @php {
    proxy_pass http://front_cluster;
}

回答1:

Nginx does not process the content stored in Memcached, it just gets it and returns to the browser as is.

The real cause is the Memcached client library your application uses. Most of the libraries compress large values (usually when value size exceeds some threshold), so you must configure it not to do so, or set memcached_gzip_flag (first appeared in Nginx "unstable" 1.3.6) with gunzip module enabled.

回答2:

The response you've posted seems like gzipped one. My first guess is that Apache is returning response with transfer-encoding=gzip which is stored into memcached, but then when popped and returned from nginx through memcached, the transfer-encoding header is omitted, thus the browser receives wrong response. You can easily test if this is the case with disabling gzip compression in Apache.

If this is the case, you should look for a solution to preserve the transfer-encoding header... maybe to define different rules - for non-gzipped and gzipped content, and to return always the header in the latter case. But look at this : http://wiki.nginx.org/HttpSRCacheModule. It seems like it handles such cases.

回答3:

so, problem was in Memcached CompressTreshold. when data exceeds 20k symbols, memcached turns on compression, even if conression = false.

回答4:

problem was in specific memcached behavior. even if you turn off data compression, memcached do it if your data exceeds limit in 20k symbols. the cure is - (in my case) on caching backend do smth like $this->_memcache->setCompressThreshold(20000, 1);

p.s. i am using Zend_Cache_Backend_Memcached as a parent class of my backend. so the string above must be at __contstruct()

回答5:

If you are using the PHP Memcached library, bear in mind that it can only store its compressed data with zlib encoding. Nginx cannot expand zlib, even with the memcached_gzip_flag as Alexander suggests above. So, in this situation you probably should not compress data in Memcached unless you feel comfortable compressing everything and passing it directly to the browser with add_header Content-Encoding deflate.

回答6:

I encountered the same issue but I finally discovered that the memcached client was using the PHP Memcached extension to save the data while we was using the PHP Memcache extension to read it from memcached server.

After changing to use Memcache on both, issue was solved!

You may be able to fix compression issue by using Memcache PHP extension to save data.

来源：https://stackoverflow.com/questions/11503111/nginx-return-corrupted-data-from-memcached

标签

nginx

memcached