Check whether a request is in Nginx cache or not?

p.ohanlon · 2018-11-21T22:56:43+00:00

Hi, Gary,the 'import' command can be used on ssdb-cli, however it can not be used here. the ssdb-cli is coded as follows:1. check the cmdif...

Check whether a request is in Nginx cache or not?

p.ohanlon

Hi,

I was wondering if there was a cleaner Lua way to check immediately whether a particular URI was present the Nginx cache (specifically when using proxy_pass and proxy_cache) without actually making a request that would subsequently causing Nginx to populate it (e.g when trying to use the $upstream_cache_status). So far I can think of a few of approaches (but most suffer from lack of access to cache 'shared memory zone' info):

1) Use the cache key and generate the corresponding md5 hash in conjunction with the levels setting and directly check for the presence of the actual cache file in the cache directory - but this can't check the cache 'shared memory zone' info...so it may be out of sync with the actual Nginx cache state.

2) Use some kind of FFI call to directly query the Nginx cache API? (Has anyone done this?)

3) Creating a separate lua_shared_dict cache key in Lua for the URI - again it may not be in sync with cache's 'shared memory zone' info

4) Using the [defunct?] ngx-lua-upstream-cache-module module - this isn't so useful as it requires additional compilation/linking to Openresty

Ideally it would be great if there was a way to serve an incoming request from the Nginx cache if present or else do something in Lua.

Any suggestions/ideas welcome.

Piers

tahmi.hamza

the only simple way I can think of now is to split the lua and the nginx proxy ! which means you put a Nginx proxy cache in front of openresty ! so whenver you get a request on the lua code you are sure 100% that the cache is expired or non-existent

all the other ways I can think of now involve hacking the module C code somehow to add some triggers or exposed some APIs to LUA

On Wednesday, November 21, 2018 at 3:56:44 PM UTC+1, Piers O'Hanlon wrote:

Hi,

I was wondering if there was a cleaner Lua way to check immediately whether a particular URI was present the Nginx cache (specifically when using proxy_pass and proxy_cache) without actually making a request that would subsequently causing Nginx to populate it (e.g when trying to use the $upstream_cache_status). So far I can think of a few of approaches (but most suffer from lack of access to cache 'shared memory zone' info):
1) Use the cache key and generate the corresponding md5 hash in conjunction with the levels setting and directly check for the presence of the actual cache file in the cache directory - but this can't check the cache 'shared memory zone' info...so it may be out of sync with the actual Nginx cache state.
2) Use some kind of FFI call to directly query the Nginx cache API? (Has anyone done this?)
3) Creating a separate lua_shared_dict cache key in Lua for the URI - again it may not be in sync with cache's 'shared memory zone' info
4) Using the [defunct?] ngx-lua-upstream-cache-module module - this isn't so useful as it requires additional compilation/linking to Openresty

Ideally it would be great if there was a way to serve an incoming request from the Nginx cache if present or else do something in Lua.

Any suggestions/ideas welcome.

Piers

p.ohanlon

Hi Hamza,

Thanks for your reply. If understand you correctly the problem with putting the normal Nginx cache in front of Openresty is that if it encounters a cache MISS then it will attempt to get the object and put it in the Nginx cache before returning control to Openresty. But I am interested to only know if an object is in the Nginx cache and not for Nginx to fetch the object at that moment if there's a MISS.

Yeah I'm sure one could do it by modifying the C code or possibly by using an FFI call to the C code - though I'm not sure how to obtain the ngx_http_request_t C structure using FFI - so one can call nginx-1.XX.Y/src/http/ngx_http_file_cache.c:ngx_http_file_cache_valid()?

On Thursday, 22 November 2018 12:29:26 UTC, hamza t wrote:

the only simple way I can think of now is to split the lua and the nginx proxy ! which means you put a Nginx proxy cache in front of openresty ! so whenver you get a request on the lua code you are sure 100% that the cache is expired or non-existent

all the other ways I can think of now involve hacking the module C code somehow to add some triggers or exposed some APIs to LUA

On Wednesday, November 21, 2018 at 3:56:44 PM UTC+1, Piers O'Hanlon wrote:
Hi,

I was wondering if there was a cleaner Lua way to check immediately whether a particular URI was present the Nginx cache (specifically when using proxy_pass and proxy_cache) without actually making a request that would subsequently causing Nginx to populate it (e.g when trying to use the $upstream_cache_status). So far I can think of a few of approaches (but most suffer from lack of access to cache 'shared memory zone' info):
1) Use the cache key and generate the corresponding md5 hash in conjunction with the levels setting and directly check for the presence of the actual cache file in the cache directory - but this can't check the cache 'shared memory zone' info...so it may be out of sync with the actual Nginx cache state.
2) Use some kind of FFI call to directly query the Nginx cache API? (Has anyone done this?)
3) Creating a separate lua_shared_dict cache key in Lua for the URI - again it may not be in sync with cache's 'shared memory zone' info
4) Using the [defunct?] ngx-lua-upstream-cache-module module - this isn't so useful as it requires additional compilation/linking to Openresty

Ideally it would be great if there was a way to serve an incoming request from the Nginx cache if present or else do something in Lua.

Any suggestions/ideas welcome.

Piers