Suggestion about nginx caching system

leandro.ribeiro.moreira · 2017-04-07T22:33:49+00:00

Why not post an appropriate snippet of the Lua code in question? Certainly that will be easier to examine. On Apr 7, 2017, at 11:41, Balinder Walia <...

Suggestion about nginx caching system

leandro.ribeiro.moreira

Before I jump in to write an "openrest" Lua library (and even if it's possible) or an entire custom Nginx module, I would like to know if there is any "module" or Lua library that offer: A way to use a Memcached cluster as caching storage for Nginx instead of pure local (SSD, tmpfs or HD) caching.

The idea is to concentrate the cache objects in a set of machines instead of spreading them all over all the Nginx's.

Thanks

jim.robinson

I also needed something where nginx caching was implemented via memcached pools, I couldn't find anything I could use off the shelf. The solution I ended up implementing used resty.memcached and Lua-MCCH to handle the memcache objects.

On Friday, April 7, 2017 at 7:33:50 AM UTC-7, Leandro Moreira wrote:

Before I jump in to write an "openrest" Lua library (and even if it's possible) or an entire custom Nginx module, I would like to know if there is any "module" or Lua library that offer: A way to use a Memcached cluster as caching storage for Nginx instead of pure local (SSD, tmpfs or HD) caching.

The idea is to concentrate the cache objects in a set of machines instead of spreading them all over all the Nginx's.

Thanks

leandro.ribeiro.moreira

I saw that there is a module that does the read from caching https://github.com/bpaquet/ngx_http_enhanced_memcached_module but it doesn't write.

I was that somehow people were able to do something similar using HAProxy https://medium.com/vimeo-engineering-blog/improving-load-balancing-with-a-new-consistent-hashing-algorithm-9f1bd75709ed

Since it seems that we don't have anything like that, I was wondering if it's possible to do it in Lua.

The idea is to proxy all the income requests to see if the request is on cache otherwise intercept the response from upstream and populate the cache (redis or memcached), is this possible with lua or should I move to pure C Nginx modules?

If it's possible to do it in lua, can you point docs or the hooks for this integration?

On Friday, April 7, 2017 at 8:53:43 PM UTC-3, Jim Robinson wrote:

I also needed something where nginx caching was implemented via memcached pools, I couldn't find anything I could use off the shelf. The solution I ended up implementing used resty.memcached and Lua-MCCH to handle the memcache objects.

On Friday, April 7, 2017 at 7:33:50 AM UTC-7, Leandro Moreira wrote:
Before I jump in to write an "openrest" Lua library (and even if it's possible) or an entire custom Nginx module, I would like to know if there is any "module" or Lua library that offer: A way to use a Memcached cluster as caching storage for Nginx instead of pure local (SSD, tmpfs or HD) caching.

The idea is to concentrate the cache objects in a set of machines instead of spreading them all over all the Nginx's.

Thanks

jim.robinson

The one I used was part of the openresty complex of packages:

https://github.com/openresty/lua-resty-memcached

The caching logic I put together was much as you described. As to whether or not it's a better idea to do it in C vs Lua, I will say I would expect a much shorter development turn around in Lua. While I have no doubt the solution would be faster in pure C, the Lua response times were fast enough for my needs. As I recall, using a Lua based memcache lookup reduced the average time to serve a request from 600ms to 6ms.

On Monday, April 10, 2017 at 6:09:37 AM UTC-7, Leandro Moreira wrote:

I saw that there is a module that does the read from caching https://github.com/bpaquet/ngx_http_enhanced_memcached_module but it doesn't write.

I was that somehow people were able to do something similar using HAProxy https://medium.com/vimeo-engineering-blog/improving-load-balancing-with-a-new-consistent-hashing-algorithm-9f1bd75709ed

Since it seems that we don't have anything like that, I was wondering if it's possible to do it in Lua.

The idea is to proxy all the income requests to see if the request is on cache otherwise intercept the response from upstream and populate the cache (redis or memcached), is this possible with lua or should I move to pure C Nginx modules?

If it's possible to do it in lua, can you point docs or the hooks for this integration?

jonathan

On Monday, April 10, 2017 at 9:30:13 AM UTC-4, Jim Robinson wrote:

The caching logic I put together was much as you described. As to whether or not it's a better idea to do it in C vs Lua, I will say I would expect a much shorter development turn around in Lua. While I have no doubt the solution would be faster in pure C, the Lua response times were fast enough for my needs. As I recall, using a Lua based memcache lookup reduced the average time to serve a request from 600ms to 6ms.

in my experience, the bottleneck on local area caching is always on the network traffic layer, so implementing in C isn't likely to have a noticeable effect. the bigger savings come from avoiding network stores whenever possible.

In an afternoon, we built/tested/deployed a caching strategy that roughly does this:

* lru cache per-worker (only on most popular routes) - 60s

* fallback to nginx shared dict - 600s
* fallback to redis - 3600s

* grab upstream

the tiered fallbacks are very lightweight. it seems a bit silly as the same data is cached in so many places -- but the effects are noticeable.

building that as C modules would have taken my same team a few days.

jim.robinson

In my particular case I'm (a) parsing details out of the request/response headers to try and conform to the logic outlined in the RFCs that discuss caching HTTP requests, and (b) packing/unpacking data structures in memcached to deal with the cached responses (e.g., handling the Vary header and its impact on caching). I imagine those operations could be done more quickly in C, but that the additional development costs just wouldn't be worth the effort for our particular use case.

On Monday, April 10, 2017 at 9:15:04 AM UTC-7, jona...@findmeon.com wrote:

On Monday, April 10, 2017 at 9:30:13 AM UTC-4, Jim Robinson wrote:

The caching logic I put together was much as you described. As to whether or not it's a better idea to do it in C vs Lua, I will say I would expect a much shorter development turn around in Lua. While I have no doubt the solution would be faster in pure C, the Lua response times were fast enough for my needs. As I recall, using a Lua based memcache lookup reduced the average time to serve a request from 600ms to 6ms.

in my experience, the bottleneck on local area caching is always on the network traffic layer, so implementing in C isn't likely to have a noticeable effect. the bigger savings come from avoiding network stores whenever possible.

jim.robinson

That "Consistent Hashing with Bounded Loads" paper discussed in the haproxy post looks very interesting, thank you for sharing it!

On Monday, April 10, 2017 at 6:09:37 AM UTC-7, Leandro Moreira wrote:

I saw that there is a module that does the read from caching https://github.com/bpaquet/ngx_http_enhanced_memcached_module but it doesn't write.

I was that somehow people were able to do something similar using HAProxy https://medium.com/vimeo-engineering-blog/improving-load-balancing-with-a-new-consistent-hashing-algorithm-9f1bd75709ed

Since it seems that we don't have anything like that, I was wondering if it's possible to do it in Lua.

leandro.ribeiro.moreira

Cool, what are the hooks for pre caching (request) and post caching (response) in Lua?

On Monday, April 10, 2017 at 10:30:13 AM UTC-3, Jim Robinson wrote:

The one I used was part of the openresty complex of packages:

https://github.com/openresty/lua-resty-memcached

The caching logic I put together was much as you described. As to whether or not it's a better idea to do it in C vs Lua, I will say I would expect a much shorter development turn around in Lua. While I have no doubt the solution would be faster in pure C, the Lua response times were fast enough for my needs. As I recall, using a Lua based memcache lookup reduced the average time to serve a request from 600ms to 6ms.

On Monday, April 10, 2017 at 6:09:37 AM UTC-7, Leandro Moreira wrote:
I saw that there is a module that does the read from caching https://github.com/bpaquet/ngx_http_enhanced_memcached_module but it doesn't write.

I was that somehow people were able to do something similar using HAProxy https://medium.com/vimeo-engineering-blog/improving-load-balancing-with-a-new-consistent-hashing-algorithm-9f1bd75709ed

Since it seems that we don't have anything like that, I was wondering if it's possible to do it in Lua.

The idea is to proxy all the income requests to see if the request is on cache otherwise intercept the response from upstream and populate the cache (redis or memcached), is this possible with lua or should I move to pure C Nginx modules?

If it's possible to do it in lua, can you point docs or the hooks for this integration?

leandro.ribeiro.moreira

Thanks, did you implement all these strategies in lua? is it open sourced? or can you link me to docs so I can try to do the same? :D

Thanks a lot.

On Monday, April 10, 2017 at 1:15:04 PM UTC-3, jona...@findmeon.com wrote:

On Monday, April 10, 2017 at 9:30:13 AM UTC-4, Jim Robinson wrote:

The caching logic I put together was much as you described. As to whether or not it's a better idea to do it in C vs Lua, I will say I would expect a much shorter development turn around in Lua. While I have no doubt the solution would be faster in pure C, the Lua response times were fast enough for my needs. As I recall, using a Lua based memcache lookup reduced the average time to serve a request from 600ms to 6ms.

in my experience, the bottleneck on local area caching is always on the network traffic layer, so implementing in C isn't likely to have a noticeable effect. the bigger savings come from avoiding network stores whenever possible.

In an afternoon, we built/tested/deployed a caching strategy that roughly does this:

* lru cache per-worker (only on most popular routes) - 60s
* fallback to nginx shared dict - 600s
* fallback to redis - 3600s
* grab upstream

the tiered fallbacks are very lightweight. it seems a bit silly as the same data is cached in so many places -- but the effects are noticeable.

building that as C modules would have taken my same team a few days.

leandro.ribeiro.moreira

Wow, I'm looking at this doc now https://github.com/openresty/lua-nginx-module#ngxshareddict :)

On Monday, April 10, 2017 at 7:01:46 PM UTC-3, Leandro Moreira wrote:

Thanks, did you implement all these strategies in lua? is it open sourced? or can you link me to docs so I can try to do the same? :D

Thanks a lot.

On Monday, April 10, 2017 at 1:15:04 PM UTC-3, jona...@findmeon.com wrote:

On Monday, April 10, 2017 at 9:30:13 AM UTC-4, Jim Robinson wrote:

The caching logic I put together was much as you described. As to whether or not it's a better idea to do it in C vs Lua, I will say I would expect a much shorter development turn around in Lua. While I have no doubt the solution would be faster in pure C, the Lua response times were fast enough for my needs. As I recall, using a Lua based memcache lookup reduced the average time to serve a request from 600ms to 6ms.

in my experience, the bottleneck on local area caching is always on the network traffic layer, so implementing in C isn't likely to have a noticeable effect. the bigger savings come from avoiding network stores whenever possible.

In an afternoon, we built/tested/deployed a caching strategy that roughly does this:

* lru cache per-worker (only on most popular routes) - 60s
* fallback to nginx shared dict - 600s
* fallback to redis - 3600s
* grab upstream

the tiered fallbacks are very lightweight. it seems a bit silly as the same data is cached in so many places -- but the effects are noticeable.

building that as C modules would have taken my same team a few days.

jim.robinson

There's an old thread I kicked off asking about streaming the response back to the user while also saving it in cache:

"[openresty-en] clone response into a coroutine or post-process the entire response after ngx.say?"

I haven't implemented the recommended solution, though I do want to if I can ever find the time.

Right now I'm just hooking into

content_by_lua_block {

...

}

and then inside that block I'm loading and executing lua libraries to process the request.

Within the lua libraries, if I decide it is a cache miss, I execute:

local response = ngx.location.capture("/proxy" .. self.request.uri, {

method = self:ngx_method(request.method),

body = self.request.body,

ctx = {},

})

which executes this subrequest from the nginx configuration:

# Call the origin server, if ngx.ctx has specified

# if_none_match or if_modified_since then set the

# corresponding If-None-Match or If-Modified-Since

# headers on the request (overriding any existing

# ones).

location /proxy {

internal;

# strip off the leading /proxy added to

# trigger the subrequest

rewrite ^/proxy/+(.*) /$1 break;

# clear the range header and add in validators

# if specified by the proxy

rewrite_by_lua_block {

ngx.req.clear_header("range")

if ngx.ctx.if_none_match then

ngx.req.set_header("If-None-Match", ngx.ctx.if_none_match)

end

if ngx.ctx.if_modified_since then

ngx.req.set_header("If-Modified-Since", ngx.ctx.if_modified_since)

end

}

# always pass back the Date from the response

proxy_pass_header "date";

# call the backend

proxy_pass http://$proxy_host;

}

The proxy_host is just a dictionary mapping of incoming host to backend hosts, e.g.,

# $proxy_server is the origin server to pass requests to

# when we have determined it cannot fulfil the request

# completely out of cache.

map $http_host $proxy_host {

default "localhost";

test-cache.example.org "test.example.org";

}

On Monday, April 10, 2017 at 3:00:10 PM UTC-7, Leandro Moreira wrote:

Cool, what are the hooks for pre caching (request) and post caching (response) in Lua?

hamish.forbes

You might want to take a look at https://github.com/pintsized/ledge and avoid re-implementing all the HTTP RFC caching logic.

It's Redis backed not memcache and full Vary support isn't implemented yet but its scriptable and flexible enough it might meet your needs.

FWIW in our experience the time spent doing network i/o outweighs any extra time you spend in Lua vs C for most cases.

On Monday, 10 April 2017 17:50:13 UTC+1, Jim Robinson wrote:

In my particular case I'm (a) parsing details out of the request/response headers to try and conform to the logic outlined in the RFCs that discuss caching HTTP requests, and (b) packing/unpacking data structures in memcached to deal with the cached responses (e.g., handling the Vary header and its impact on caching). I imagine those operations could be done more quickly in C, but that the additional development costs just wouldn't be worth the effort for our particular use case.

On Monday, April 10, 2017 at 9:15:04 AM UTC-7, jona...@findmeon.com wrote:

On Monday, April 10, 2017 at 9:30:13 AM UTC-4, Jim Robinson wrote:

The caching logic I put together was much as you described. As to whether or not it's a better idea to do it in C vs Lua, I will say I would expect a much shorter development turn around in Lua. While I have no doubt the solution would be faster in pure C, the Lua response times were fast enough for my needs. As I recall, using a Lua based memcache lookup reduced the average time to serve a request from 600ms to 6ms.

in my experience, the bottleneck on local area caching is always on the network traffic layer, so implementing in C isn't likely to have a noticeable effect. the bigger savings come from avoiding network stores whenever possible.

leandro.ribeiro.moreira

Good Thank you so much!!!

On Tuesday, April 11, 2017 at 6:06:36 AM UTC-3, Hamish Forbes wrote:

You might want to take a look at https://github.com/pintsized/ledge and avoid re-implementing all the HTTP RFC caching logic.
It's Redis backed not memcache and full Vary support isn't implemented yet but its scriptable and flexible enough it might meet your needs.

FWIW in our experience the time spent doing network i/o outweighs any extra time you spend in Lua vs C for most cases.

On Monday, 10 April 2017 17:50:13 UTC+1, Jim Robinson wrote:
In my particular case I'm (a) parsing details out of the request/response headers to try and conform to the logic outlined in the RFCs that discuss caching HTTP requests, and (b) packing/unpacking data structures in memcached to deal with the cached responses (e.g., handling the Vary header and its impact on caching). I imagine those operations could be done more quickly in C, but that the additional development costs just wouldn't be worth the effort for our particular use case.

On Monday, April 10, 2017 at 9:15:04 AM UTC-7, jona...@findmeon.com wrote:

On Monday, April 10, 2017 at 9:30:13 AM UTC-4, Jim Robinson wrote:

The caching logic I put together was much as you described. As to whether or not it's a better idea to do it in C vs Lua, I will say I would expect a much shorter development turn around in Lua. While I have no doubt the solution would be faster in pure C, the Lua response times were fast enough for my needs. As I recall, using a Lua based memcache lookup reduced the average time to serve a request from 600ms to 6ms.

in my experience, the bottleneck on local area caching is always on the network traffic layer, so implementing in C isn't likely to have a noticeable effect. the bigger savings come from avoiding network stores whenever possible.

jim.robinson

Thank you, that does look very useful.

On Tuesday, April 11, 2017 at 2:06:36 AM UTC-7, Hamish Forbes wrote:

You might want to take a look at https://github.com/pintsized/ledge and avoid re-implementing all the HTTP RFC caching logic.
It's Redis backed not memcache and full Vary support isn't implemented yet but its scriptable and flexible enough it might meet your needs.

FWIW in our experience the time spent doing network i/o outweighs any extra time you spend in Lua vs C for most cases.

jonathan

On Monday, April 10, 2017 at 6:01:46 PM UTC-4, Leandro Moreira wrote:

Thanks, did you implement all these strategies in lua? is it open sourced? or can you link me to docs so I can try to do the same? :D

All of these are using the core lua libraries; an upstream lookup would need something like pintsized/lua-resty-http

We open sourced a SSL certificate manager that uses all the same lookups - https://github.com/aptise/peter_sslers-lua-resty

In terms of implementing them, you'd need to use a `content_by_lua_block` instead

The pintsized/ledge option looks like it'll work.

On Monday, April 10, 2017 at 12:50:13 PM UTC-4, Jim Robinson wrote:

I imagine those operations could be done more quickly in C, but that the additional development costs just wouldn't be worth the effort for our particular use case.

Those could be done more quickly in C, but not noticeably quicker. Assuming your 6ms timing, you probably have 5.95ms dealing with memcached and .05ms dealing with those operations. The biggest impacts to speed would be minimizing the network traffic. You can achieve that by having a local cache within openresty, at the worker and shared levels (all in lua).