SRCache with Redis and AUTH

bendulu · 2014-09-24T21:18:39+00:00

Hi,Yep, looks to be working in the latest release :)HamishOn Thursday, 24 December 2015 19:36:09 UTC, agentzh wrote:Hello! On Mon, Oct 13, 2014 at 2:49 PM,...

SRCache with Redis and AUTH

bendulu

Hey,

i am trying to build reverse proxy cache for our frontend farm that will share it's cache via redis.

the problem i am facing is that the redis instance is password protected and i cant get the reading from redis to work.

after some searching i found out that i can't do the writing with the redis module but should use lua-redis to do it. and here is where i am stuck. i cant figure out how to return the read value form redis to srcache - it always gives me the error "srcache_fetch: cache sent truncated response body".

i think i understand why this is happening : ngx.echo sends some headers along with the actual result form redis and so srcache can't read it properly. but what can i do to circumvent this? i am very new to lua and nginx so frogiveme if thats a simple thing to do...

here is what i have (auth is disabled for now...) :

    upstream redisbackend 
    {
          server 192.168.50.1:6379;
          keepalive 1024;
    }

    server 
    {
        listen 9000 ;
        server_name test;   

        #
        # Cache settings
        #
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_max_temp_file_size 0;
        proxy_connect_timeout 60;
        proxy_send_timeout 120;
        proxy_read_timeout 180;
        proxy_buffer_size 4k;
        proxy_buffers 4 32k;
        proxy_busy_buffers_size 64k;
        proxy_temp_file_write_size 64k;
        proxy_temp_path /tmp;
        proxy_cache_use_stale updating;
        proxy_set_header Accept-Encoding "";
        proxy_ignore_headers Cache-Control;
        more_clear_headers Server X-Powered-By;

        #
        # Logging
        #
        access_log /mnt/logs/openresty/test-access.log access;
        error_log /mnt/logs/openresty/test-error.log error;


        #
        # Redis Read
        #
        location = /__redis_read
        {
            internal;

            content_by_lua '
                local key = ngx.md5(ngx.var.arg_key)
                local redis = require "resty.redis"
                local instance = redis:new()
                instance:set_timeout(1000)
                local ok, err = instance:connect("192.168.50.1", 6379)
                if not ok then
                    ngx.say("failed to connect: ", err)
                    return
                end
                local res, err = instance:get(key)
                if not res then
                    ngx.say("failed to get cache: ", err)
                    return
                end

                if res == ngx.null then
                    ngx.say("failed to get cache: ")
                    return
                end                

                ngx.print(res)
               ';
        }

        #
        # Redis Write
        #
        location = /__redis_write
        {
            internal;
     
            set_unescape_uri $ttl $arg_ttl;
            set_unescape_uri $key $arg_key;
            set_md5 $key;
     
            #redis2_query auth "password"
            #redis2_query select 1;
            redis2_query set $key $echo_request_body;
            redis2_query expire $key $ttl;
            redis2_pass redisbackend;
        }


        #
        # Catch all
        #
        location / 
        {
            # Source
            proxy_pass http://test:8080/;

            # Redis
            set $key $uri?$args;
            set_escape_uri $escaped_key $key;
            srcache_fetch GET /__redis_read key=$escaped_key;
            srcache_store PUT /__redis_write key=$escaped_key&ttl=86400;

            # No Cookies, Sir
            proxy_ignore_headers Set-Cookie;
            proxy_hide_header Set-Cookie;
 
            # Adjust headers
            add_header X-Cache-Status $srcache_fetch_status;  
            add_header X-Cache-Server $server_addr;

            # Default Document
            index index.hml;

            # Rewrite to root
            rewrite ^/$ /index.html permanent;
            rewrite ^$ /index.html permanent;
        }
    }

cheers

Chris

agentzh

Hello!

On Wed, Sep 24, 2014 at 6:18 AM, Christian A. wrote:
> after some searching i found out that i can't do the writing with the redis
> module but should use lua-redis to do it. and here is where i am stuck. i
> cant figure out how to return the read value form redis to srcache - it
> always gives me the error "srcache_fetch: cache sent truncated response
> body".

This error happens when the response body from the srcache_fetch
subrequest is not a well-formed HTTP response. You can query your
srcache_fetch subrequest directly from outside and check if the
response body is well-formed. If you're not sure, just paste the raw
output here.

> i think i understand why this is happening : ngx.echo sends some headers
> along with the actual result form redis and so srcache can't read it
> properly. but what can i do to circumvent this?

The subrequest's response headers are ignored by ngx_srcache because
ngx_srcache just reads the cached response headers from the subrequest
response body data.

>             content_by_lua '
>                 local key = ngx.md5(ngx.var.arg_key)
>                 local redis = require "resty.redis"
>                 local instance = redis:new()
>                 instance:set_timeout(1000)
>                 local ok, err = instance:connect("192.168.50.1", 6379)
>                 if not ok then
>                     ngx.say("failed to connect: ", err)

Do not emit your own error text to the response body with 200 status
code, because srcache_fetch will treat it as the cached response data
(i.e., a cache hit). The correct way is to log the error and exit with
an error status code, as in

    ngx.log(ngx.ERR, "failed to connect: ", err)
    ngx.exit(500)

>                     return
>                 end
>                 local res, err = instance:get(key)
>                 if not res then
>                     ngx.say("failed to get cache: ", err)

Ditto.

>                     return
>                 end
>
>                 if res == ngx.null then
>                     ngx.say("failed to get cache: ")

Here, it is a cache miss. You should return 404 here to notify
srcache_fetch. That is,

    ngx.exit(404)

Otherwise, the 200 status code will confuse srcache_fetch and make it
treat it as a cache hit and your "failed to get cache: " text as the
cached response data.

>                     return

BTW, before quitting, you should put the current redis connection back
to the connection pool via the set_keepalive() method call.

>                 end
>
>                 ngx.print(res)

Ditto.

Regards,
-agentzh