ngx.shared if no memory the expired key donot delete

378255237 · 2018-09-03T21:03:23+00:00

Hi, On 09/03/2018 05:47 AM, Bjørnar Ness wrote: > Is it possible to get the socket filedescriptor from ngx.req.socket? > > Would like to use...

ngx.shared if no memory the expired key donot delete

378255237

first:

local key = "testkey"

-- only the testkey is expire key

local ok, err = shmdb:safe_set(key, 1,20)

ngx.say(" ok:".. tostring(ok).. " err:" .. tostring(err))

local i = 1

while true do

local ok, err = shmdb:safe_set("key-" .. i, i)

if not ok then

ngx.say("aborted: ", err)

break

end

i = i + 1

end

ngx.say("inserted ", i, " keys.")

run:

ok:true err:nil
aborted: no memory
inserted 4072095 keys.

then:

local key = "testkey"

-- for deletes one or two expired entries
local value = shmdb:get(key)
ngx.say(" value:".. tostring(value))

local value, flags, stale = shmdb:get_stale(key)
ngx.say(" value:".. tostring(value).. " flags:" .. tostring(flags).. " stale:".. tostring(stale))

 run(no expired):

 value:1
 value:1 flags:nil stale:false

 run(expired):

 value:nil
 value:1 flags:nil stale:true

 run:
 value:nil
 value:1 flags:nil stale:true

 the expired key donot delete

I see the src/ngx_http_lua_shdict.c 509 
if (!get_stale) {
    ngx_http_lua_shdict_expire(ctx, 1);
}
 /*
  * n == 1 deletes one or two expired entries
  * n == 0 deletes oldest entry by force
  *        and one or two zero rate entries
  */



sudo ~/openresty-systemtap-toolkit/ngx-shm -p 17532 -n darknet_shared_db
Tracing 17532 (/usr/local/nginx-1.12.1/sbin/nginx)...

shm zone "darknet_shared_db"
    owner: ngx_http_lua_shared_memory
    total size: 512000 KB
    free pages: 0 KB (0 pages, 0 blocks)

zchao1995

Hello!

> local key = "testkey"

>
> -- for deletes one or two expired entries
> local value = shmdb:get(key)
> ngx.say(" value:".. tostring(value))
>
> local value, flags, stale = shmdb:get_stale(key)
> ngx.say(" value:".. tostring(value).. " flags:" .. tostring(flags).. " stale:".. tostring(stale))

> run(no expired):

> value:1
> value:1 flags:nil stale:false

> run(expired):

> value:nil
> value:1 flags:nil stale:true
>
> run:
> value:nil
> value:1 flags:nil stale:true
>
> the expired key donot delete

The eviction policy is not mandatory inside ngx.shared.DICT.get(), only the expired key can be removed,
while the eviction is from the least recently used key, i.e the first key inserted in your while loop, which is never expired.

So no key will be deleted.

378255237

Look at the following example

sudo ~/openresty-systemtap-toolkit/ngx-shm -n darknet_shared_db -p 32413

Tracing 32413 (/usr/local/nginx-1.12.1/sbin/nginx)...

shm zone "darknet_shared_db"

owner: ngx_http_lua_shared_memory

total size: 512000 KB

free pages: 509008 KB (127252 pages, 1 blocks)

Run code :

local i = 1

while true do

local ok, err = shmdb:safe_set("key1-" .. i, i, 20)

if not ok then

ngx.say("aborted: ", err)

break

end

i = i + 1

end

ngx.say("inserted ", i, " keys.")

--------

aborted: no memory
inserted 4072095 keys.

Run

code2 :

local keys = shmdb:get_keys()

local data = "">

for _,v in pairs(keys) do

local val = shmdb:get(v)

data[v] = val

end

ngx.say(tools.makeJson(data,0,"success"))

-------------

{"errno":0,"data":{"key1-1023":1023,"key1-802":802,"key1-922":922,"key1-15":15,"key1-376":376,"key1-696":696,"key1-1018":1018,"key1-344":344,"key1-1019":1019,"key1-767":767,"key1-1016":1016,"key1-1015":1015,"key1-430":430,"key1-968":968,"key1-74":74,....

shm zone "darknet_shared_db"
    owner: ngx_http_lua_shared_memory
    total size: 512000 KB
    free pages: 0 KB (0 pages, 0 blocks)

after 20 second

Run code2 :

------------
{"errno":0,"data":{},"errmsg":"success"}

Tracing 32413 (/usr/local/nginx-1.12.1/sbin/nginx)...

shm zone "darknet_shared_db"
    owner: ngx_http_lua_shared_memory
    total size: 512000 KB
    free pages: 0 KB (0 pages, 0 blocks)

 keys has deleted, free pages: 0 KB ? When the memory will be free?

在 2018年9月6日星期四 UTC+8上午11:04:59，tokers写道：

Hello!

> local key = "testkey"

>
> -- for deletes one or two expired entries
> local value = shmdb:get(key)
> ngx.say(" value:".. tostring(value))
>
> local value, flags, stale = shmdb:get_stale(key)
> ngx.say(" value:".. tostring(value).. " flags:" .. tostring(flags).. " stale:".. tostring(stale))

> run(no expired):

> value:1
> value:1 flags:nil stale:false

> run(expired):

> value:nil
> value:1 flags:nil stale:true
>
> run:
> value:nil
> value:1 flags:nil stale:true
>
> the expired key donot delete

The eviction policy is not mandatory inside ngx.shared.DICT.get(), only the expired key can be removed,
while the eviction is from the least recently used key, i.e the first key inserted in your while loop, which is never expired.

So no key will be deleted.