Clarification on shared dict eviction or expiration

ales.bregar · 2017-01-20T00:25:18+00:00

Apparently since nginx returns:Content-Type: application/json; qs=2I have to use the directive:replace_filter_types qs=2 application/json *;before it will wo...

Clarification on shared dict eviction or expiration

ales.bregar

Hi,

was looking through available documentation on github and openresty web but couldn't clarify fully how shared dict is cleaned.
According to
"When it fails to allocate memory for the current key-value item, then set will try removing existing items in the storage according to the Least-Recently Used (LRU) algorithm. Note that, LRU takes priority over expiration time here. If up to tens of existing items have been removed and the storage left is still insufficient (either due to the total capacity limit specified by lua_shared_dict or memory segmentation), then the err return value will be no memory and success will be false."

Does LRU eviction distinguish between values where exptime was set and non-expirable items? Or just dumps oldest items regardless?

Other question is - is there any trigger which periodically (or when over some threshold) automatically flush_expired items?

Thanks in advance for the clarifications

rpaprocki

Hi,

On Thu, Jan 19, 2017 at 11:25 AM, Aleš Bregar <ales....@gmail.com> wrote:

> Hi,
>
> was looking through available documentation on github and openresty web
> but couldn't clarify fully how shared dict is cleaned.
> According to
> "When it fails to allocate memory for the current key-value item, then set
> will try removing existing items in the storage according to the
> Least-Recently Used (LRU) algorithm. Note that, LRU takes priority over
> expiration time here. If up to tens of existing items have been removed and
> the storage left is still insufficient (either due to the total capacity
> limit specified by lua_shared_dict
> <https://github.com/openresty/lua-nginx-module#lua_shared_dict> or memory
> segmentation), then the err return value will be no memory and success
> will be false."
>

> Does LRU eviction distinguish between values where exptime was set and
> non-expirable items? Or just dumps oldest items regardless?
>

ales.bregar

Hi,
thank you for the prompt and excellent answer, I think it is clear now.

May I ask further - wondering about fragmentation (mentioned in https://github.com/openresty/lua-nginx-module/issues/175). My intentions with stored entities size are not going above 4k.
Issue is closed, there is some ngx patch created but I found no clear severity and recommendation for necessity in using it?

Thanks again and best regards

rpaprocki

Hi,

On Fri, Jan 20, 2017 at 3:50 AM, Aleš Bregar <ales....@gmail.com> wrote:

Hi,
thank you for the prompt and excellent answer, I think it is clear now.

May I ask further - wondering about fragmentation (mentioned in https://github.com/openresty/lua-nginx-module/issues/175). My intentions with stored entities size are not going above 4k.
Issue is closed, there is some ngx patch created but I found no clear severity and recommendation for necessity in using it?

This patch was merged into core several years ago:

https://github.com/nginx/nginx/commit/afb4aafc6e42781068ce3f23d22fd6e2012787c7

So you don't have anything to worry about :) Cheers!

ales.bregar

That's great. Thank you for the valuable answer!