HttpSRCacheModule need help

mfeldheim · 2013-06-21T00:38:38+00:00

Thanks so much! That looks neat and elegant!fc2...@gmail.com於 2013年6月11日星期二UTC+8下午4時56&...

HttpSRCacheModule need help

mfeldheim

Hey guys,

i am trying to setup caching to a CouchDB server.

This didn't work so far, so I decided to simplify the configuration to the following minimum:

my nginx.conf

http://pastebin.com/9NmNmhvA

The subrequests to both, cache_read and cache_write are fired, but for some reason the response can't be read, I am getting a cache miss

# curl -i http://localhost/

HTTP/1.1 200 OK

Server: ngx_openresty/1.2.7.8

Date: Thu, 20 Jun 2013 16:34:57 GMT

Content-Type: text/html

Content-Length: 28

Connection: keep-alive

Last-Modified: Thu, 20 Jun 2013 16:03:08 GMT

Accept-Ranges: bytes

<h1>backend index.html</h1>

when calling the fake cache_read host directly I get 200 OK and content

# curl -i http://localhost:8081/cache-read.html

HTTP/1.1 200 OK

Server: ngx_openresty/1.2.7.8

Date: Thu, 20 Jun 2013 16:36:02 GMT

Content-Type: text/html

Content-Length: 20

Last-Modified: Thu, 20 Jun 2013 15:53:56 GMT

Connection: keep-alive

Accept-Ranges: bytes

<h1>cache read</h1>

error.log:

2013/06/20 16:34:57 [error] 1156#0: *90 srcache_fetch: cache sent invalid status line while sending to client, client: 127.0.0.1, server: _, request: "GET / HTTP/1.1", subrequest: "/cache-read.html", upstream: "http://127.0.0.1:8081/cache-read.html", host: "localhost"

What am I missing?

# nginx -V

nginx version: ngx_openresty/1.2.7.8

built by gcc 4.7.2 20121109 (Red Hat 4.7.2-8) (GCC)

TLS SNI support enabled

configure arguments: --prefix=/usr/nginx --add-module=../ngx_devel_kit-0.2.18 --add-module=../echo-nginx-module-0.45 --add-module=../xss-nginx-module-0.03rc9 --add-module=../ngx_coolkit-0.2rc1 --add-module=../set-misc-nginx-module-0.22rc8 --add-module=../form-input-nginx-module-0.07 --add-module=../encrypted-session-nginx-module-0.03 --add-module=../srcache-nginx-module-0.19 --add-module=../ngx_lua-0.7.21 --add-module=../headers-more-nginx-module-0.19 --add-module=../array-var-nginx-module-0.03rc1 --add-module=../memc-nginx-module-0.13rc3 --add-module=../redis2-nginx-module-0.10 --add-module=../redis-nginx-module-0.3.6 --add-module=../auth-request-nginx-module-0.2 --add-module=../rds-json-nginx-module-0.12rc10 --add-module=../rds-csv-nginx-module-0.05rc2 --with-ld-opt=-Wl,-rpath,/usr/luajit/lib --sbin-path=/usr/local/sbin --http-log-path=/var/log/nginx/access.log --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --pid-path=/var/run/nginx/nginx.pid --lock-path=/var/run/nginx/nginx.lock --user=nginx --group=nginx --with-http_ssl_module

agentzh

Hello!

On Thu, Jun 20, 2013 at 9:38 AM, mfeldheim wrote:
> i am trying to setup caching to a CouchDB server.
> This didn't work so far, so I decided to simplify the configuration to the
> following minimum:
>
> my nginx.conf
> http://pastebin.com/9NmNmhvA
>

You're using ngx_srcache in a wrong way.

1. Your srcache_store subrequest does *not* write the request body as
the cached data and
2. your srcache_fetch subrequest does not return a well formed HTTP response.

Please note that, ngx_srcache caches the *whole* HTTP response (with
the response header!) instead of just the response body.

> The subrequests to both, cache_read and cache_write are fired, but for some
> reason the response can't be read, I am getting a cache miss
>

That is because srcache_fetch sees ill-formed HTTP response data.

> error.log:
>
> 2013/06/20 16:34:57 [error] 1156#0: *90 srcache_fetch: cache sent invalid
> status line while sending to client, client: 127.0.0.1, server: _, request:
> "GET / HTTP/1.1", subrequest: "/cache-read.html", upstream:
> "http://127.0.0.1:8081/cache-read.html";, host: "localhost"
>

The error message explains that srcache_fetch sees bad cached data. It
expects the cached data begin with a proper HTTP response status line
like

    HTTP/1.1 200 OK

Best regards,
-agentzh

mfeldheim

Hi agentzh,

thank's for your tip, added headers to my fake cache properly and now I get a cache hit.

Really curious how well nginx + ngx_srcache + couchdb performs compared to proxy_cache file caching. Can't use redis unfortunately, we got ~200GB cache size / 8-10m documents, Memory is too expensive, but that's also not doing well on ext4 / Amazon EBS (cache miss is faster, which is "okay" because a miss would not be faster if the backends would have to serve all requests live.. still faster is better right? ;))

so the next step will be to get the whole document into couch..

  srcache_store PUT /couchdoc

does srcache_store PUT the result document including headers already?

e.g. response returned by backend upstream

HTTP/1.1 200 OK

Content-Type: text/html

Connection: keep-alive

Server: ngx_openresty/1.2.7.8

Date: Thu, 20 Jun 2013 21:49:26 GMT

Content-Length: 20

Last-Modified: Thu, 20 Jun 2013 15:53:56 GMT

Accept-Ranges: bytes

<h1>backend read</h1>

results in _store subrequest

PUT /couchdoc HTTP/1.1

Content-Length: 20+upstream_response_header_size

HTTP/1.1 200 OK

Content-Type: text/html

Connection: keep-alive

Server: ngx_openresty/1.2.7.8

Date: Thu, 20 Jun 2013 21:49:26 GMT

Content-Length: 20

Last-Modified: Thu, 20 Jun 2013 15:53:56 GMT

Accept-Ranges: bytes

<h1>backend read</h1>

Is that correct?

Regards

Michel

agentzh

Hello!

Sorry for the delay! I missed your last email somehow.

On Thu, Jun 20, 2013 at 3:21 PM, mfeldheim wrote:
> so the next step will be to get the whole document into couch..
>
>   srcache_store PUT /couchdoc
>
> does srcache_store PUT the result document including headers already?
>

Yes.

> e.g. response returned by backend upstream
>
> HTTP/1.1 200 OK
> Content-Type: text/html
> Connection: keep-alive
> Server: ngx_openresty/1.2.7.8
> Date: Thu, 20 Jun 2013 21:49:26 GMT
> Content-Length: 20
> Last-Modified: Thu, 20 Jun 2013 15:53:56 GMT
> Accept-Ranges: bytes
>
> <h1>backend read</h1>
>
> results in _store subrequest
>
> PUT /couchdoc HTTP/1.1
> Content-Length: 20+upstream_response_header_size
>
> HTTP/1.1 200 OK
> Content-Type: text/html
> Connection: keep-alive
> Server: ngx_openresty/1.2.7.8
> Date: Thu, 20 Jun 2013 21:49:26 GMT
> Content-Length: 20
> Last-Modified: Thu, 20 Jun 2013 15:53:56 GMT
> Accept-Ranges: bytes
>
> <h1>backend read</h1>
>
> Is that correct?
>

Yes, right.

Best regards,
-agentzh