Clearning data from resty.lrucache cache from all worker processes

rohit.c.joshi · 2015-11-18T03:08:05+00:00

On Wednesday, November 18, 2015 at 1:11:43 AM UTC+7, agentzh wrote:Hello! Note the duplicate "resty/" part in the line of the error message above....

Clearning data from resty.lrucache cache from all worker processes

rohit.c.joshi

Hello,

I am managing user backlist in resty.lrucache. When new users are added to the blacklist, I want to clear the cache and retrieve data from backend.

Clearing cache request is invoked using a HTTP request and able to clear the cache.

My understanding is that resty.lrucache is per worker so how do I make sure I clear cache from all worker process?

Thanks,

lordnynex

Hello,

For clearing LRU cache you will need to use ngx.shared.dict to implement some sort of message signaling between the workers.

IMHO, though, a dynamic blacklist is better implemented in ngx.shared.dict directly. All workers would read/write from 1 entry in shared memory as opposed to duplicating memory storage per worker.

On Tue, Nov 17, 2015 at 11:08 AM, RJoshi <rohit....@gmail.com> wrote:

Hello,
I am managing user backlist in resty.lrucache. When new users are added to the blacklist, I want to clear the cache and retrieve data from backend.
Clearing cache request is invoked using a HTTP request and able to clear the cache.

My understanding is that resty.lrucache is per worker so how do I make sure I clear cache from all worker process?

Thanks,

.

rohit.c.joshi

Problem with this approach is when we have multiple instances of nginx/openresty.

My ideas of implementing black list is to maintain a blacklisted userid in a table in MySQL which will be retrieved by openresty app when not found in the cache.

When new user is added/removed into/from the backend, http requests will be sent to each nginx instance to flush the cache so next lookup will not find in the cache and retrieve from backend/ update cache again.

Does ngx.shared.dict allow lookup by partial key? Blackedlisted.user.xxx where xxx is userid. Now if I can lookup by Blacklisted.user.*, your suggested approach will work.

On Nov 17, 2015, at 2:37 PM, Lord Nynex <lord...@gmail.com> wrote:

Hello,

For clearing LRU cache you will need to use ngx.shared.dict to implement some sort of message signaling between the workers.

IMHO, though, a dynamic blacklist is better implemented in ngx.shared.dict directly. All workers would read/write from 1 entry in shared memory as opposed to duplicating memory storage per worker.

On Tue, Nov 17, 2015 at 11:08 AM, RJoshi <rohit....@gmail.com> wrote:
Hello,
I am managing user backlist in resty.lrucache. When new users are added to the blacklist, I want to clear the cache and retrieve data from backend.
Clearing cache request is invoked using a HTTP request and able to clear the cache.

My understanding is that resty.lrucache is per worker so how do I make sure I clear cache from all worker process?

Thanks,

.

agentzh

Hello!

On Wed, Nov 18, 2015 at 3:08 AM, RJoshi wrote:
>   I am managing user backlist in resty.lrucache.  When new users are added
> to the blacklist,  I want to clear the cache and retrieve data from backend.
>   Clearing cache request is invoked using a HTTP request and able to clear
> the cache.
>
> My understanding is that resty.lrucache is per worker so how do I make sure
> I clear cache from all worker process?

Why not just use a shorter exptime on the keys? Then the keys always
expire periodically and automatically.

Regards,
-agentzh

rohit.c.joshi

That's how I have implemented but business/security team wants to have immediate effect. Maybe, I will have to set expires to 30sec and take the overhead of sql query, storing into lualru and shared cache.

On Sunday, 22 November 2015 22:58:57 UTC-5, agentzh wrote:

Hello!

On Wed, Nov 18, 2015 at 3:08 AM, RJoshi wrote:
> I am managing user backlist in resty.lrucache. When new users are added
> to the blacklist, I want to clear the cache and retrieve data from backend.
> Clearing cache request is invoked using a HTTP request and able to clear
> the cache.
>
> My understanding is that resty.lrucache is per worker so how do I make sure
> I clear cache from all worker process?

Why not just use a shorter exptime on the keys? Then the keys always
expire periodically and automatically.

Regards,
-agentzh