Sharing data across workers dynamically with ngx.shared.DICT API

odel.nikolaj · 2019-08-17T12:20:02+00:00

connot connect Azure Database for MySQL, the error is:old pre-4.1 authentication protocol not supported

Sharing data across workers dynamically with ngx.shared.DICT API

odel.nikolaj

Hi,

I am working with NGINX server and I need to make sure all workers are dynamically sharing in-memory data. When data is updated, all workers should be up-to-date.

Reading the docs ngx.shared.DICT can be used for sharing data among workers. So i put that to the test, and it doesn't work.

It seems ngx.shared created copy data for each worker, instead of having one data for all workers.

Here is my simple test code.

nginx.conf

conf

worker_processes  4;

events {
    worker_connections  5;
}

http{

    lua_shared_dict metadata 10m;

    init_by_lua_file lua/init.lua;

    server{
        listen 80 reuseport;

        location / {
            return 200 "Hello";
        }

        location =/worker {
            content_by_lua '
                ngx.say("worker id: " .. ngx.var.pid)
                ngx.say("worker count: " .. ngx.worker.count())

                local metadata = ngx.shared.metadata;
                ngx.say("shared data: " .. metadata["last"])
            ';
        }


        location =/change{
            content_by_lua '
            local metadata = ngx.shared.metadata;
            local t = metadata["last"];
            metadata["last"] = t + 1;
            ';
        }

    }
}

init.lua

ngx.shared.metadata["last"] = 1;

On initialization a shared value last is declared with value 1.

After I make /change request and increment the value +1, only 1/4 workers will have shared value updated, while the others are going to keep using the old value. Probably because that is the worker that caught the request.

I am curling the server in a loop to see the results updated. And in the printout i can see that exactly which worker has newer version

worker id: 10
worker count: 4
shared data: 1
----
worker id: 9
worker count: 4
shared data: 2

I am using dockerized NGINX with open resty Lua module. And also activated the reuseport directive which forces nginx to loadbalance to all workers (just for developing).

How do I make all nginx workers share the date from shared dict dynamically? Is there a system in nginx that can?

Can someone please help,

Thank you

nishant.kumar_3c1d

Hi Nikolaj,

You should use

local metadata = ngx.shared.metadata
local succ, err, forcible = metadata:incr("last", 1)

ngx.shared.DICT expose a lot of API's to play with it. Please use it.

Please refer:

https://github.com/openresty/lua-nginx-module#ngxshareddictincr

On Saturday, August 17, 2019 at 9:50:05 AM UTC+5:30, Nikolaj Odel wrote:

Hi,
I am working with NGINX server and I need to make sure all workers are dynamically sharing in-memory data. When data is updated, all workers should be up-to-date.

Reading the docs ngx.shared.DICT can be used for sharing data among workers. So i put that to the test, and it doesn't work.

It seems ngx.shared created copy data for each worker, instead of having one data for all workers.

Here is my simple test code.

nginx.conf
conf

worker_processes  4;

events {
    worker_connections  5;
}

http{

    lua_shared_dict metadata 10m;

    init_by_lua_file lua/init.lua;

    server{
        listen 80 reuseport;

        location / {
            return 200 "Hello";
        }

        location =/worker {
            content_by_lua '
                ngx.say("worker id: " .. ngx.var.pid)
                ngx.say("worker count: " .. ngx.worker.count())

                local metadata = ngx.shared.metadata;
                ngx.say("shared data: " .. metadata["last"])
            ';
        }


        location =/change{
            content_by_lua '
            local metadata = ngx.shared.metadata;
            local t = metadata["last"];
            metadata["last"] = t + 1;
            ';
        }

    }
}
init.lua
ngx.shared.metadata["last"] = 1;
On initialization a shared value last is declared with value 1.

After I make /change request and increment the value +1, only 1/4 workers will have shared value updated, while the others are going to keep using the old value. Probably because that is the worker that caught the request.

I am curling the server in a loop to see the results updated. And in the printout i can see that exactly which worker has newer version
worker id: 10
worker count: 4
shared data: 1
----
worker id: 9
worker count: 4
shared data: 2
I am using dockerized NGINX with open resty Lua module. And also activated the reuseport directive which forces nginx to loadbalance to all workers (just for developing).

How do I make all nginx workers share the date from shared dict dynamically? Is there a system in nginx that can?
Can someone please help,

Thank you

odel.nikolaj

Thanks for answering Nishant,

I was looking at the documentation,

the issue is when i increment a value in shared dict, that value is not dynamically shared by all workers.

Only the worker that catches the request will have updated value

for example

on init:
ngx.shared.metadata["value"] = 1

worker id1: value = 1:

worker id2: value = 1:

worker id3: value = 1:

i increment the shared value to 2, with get request "/change". Only worker that catches the request will update the value, the rest will keep using old value.

worker id1: value = 1:

worker id2: value = 2:

worker id3: value = 1:

It says in documentation that shared.dict is shared memory for all workers.

But it looks like it is shared by creating a local copy of data on initialization instead of actually being shared.

Is there a way in NGINX + Lua to have a global variable that can be updated which all workers can use?

On Mon, Aug 19, 2019 at 7:02 AM Nishant Kumar <nishan...@kayzen.io> wrote:

Hi Nikolaj,
You should use
local metadata = ngx.shared.metadata
local succ, err, forcible = metadata:incr("last", 1)
ngx.shared.DICT expose a lot of API's to play with it. Please use it.

Please refer:
https://github.com/openresty/lua-nginx-module#ngxshareddictincr

On Saturday, August 17, 2019 at 9:50:05 AM UTC+5:30, Nikolaj Odel wrote:
Hi,
I am working with NGINX server and I need to make sure all workers are dynamically sharing in-memory data. When data is updated, all workers should be up-to-date.

Reading the docs ngx.shared.DICT can be used for sharing data among workers. So i put that to the test, and it doesn't work.

It seems ngx.shared created copy data for each worker, instead of having one data for all workers.

Here is my simple test code.

nginx.conf
conf

worker_processes  4;

events {
    worker_connections  5;
}

http{

    lua_shared_dict metadata 10m;

    init_by_lua_file lua/init.lua;

    server{
        listen 80 reuseport;

        location / {
            return 200 "Hello";
        }

        location =/worker {
            content_by_lua '
                ngx.say("worker id: " .. ngx.var.pid)
                ngx.say("worker count: " .. ngx.worker.count())

                local metadata = ngx.shared.metadata;
                ngx.say("shared data: " .. metadata["last"])
            ';
        }


        location =/change{
            content_by_lua '
            local metadata = ngx.shared.metadata;
            local t = metadata["last"];
            metadata["last"] = t + 1;
            ';
        }

    }
}
init.lua
ngx.shared.metadata["last"] = 1;
On initialization a shared value last is declared with value 1.

After I make /change request and increment the value +1, only 1/4 workers will have shared value updated, while the others are going to keep using the old value. Probably because that is the worker that caught the request.

I am curling the server in a loop to see the results updated. And in the printout i can see that exactly which worker has newer version
worker id: 10
worker count: 4
shared data: 1
----
worker id: 9
worker count: 4
shared data: 2
I am using dockerized NGINX with open resty Lua module. And also activated the reuseport directive which forces nginx to loadbalance to all workers (just for developing).

How do I make all nginx workers share the date from shared dict dynamically? Is there a system in nginx that can?
Can someone please help,

Thank you

jarmo

Is that a valid way to use the DICT API?

As Nishant mentioned, try using the incr and get APIs instead of the hash notation.

On Mon, Aug 19, 2019 at 12:14 PM Nikolaj Odel <odel....@gmail.com> wrote:

Thanks for answering Nishant,
I was looking at the documentation,

the issue is when i increment a value in shared dict, that value is not dynamically shared by all workers.
Only the worker that catches the request will have updated value
for example
on init:
ngx.shared.metadata["value"] = 1
worker id1: value = 1:
worker id2: value = 1:
worker id3: value = 1:

i increment the shared value to 2, with get request "/change". Only worker that catches the request will update the value, the rest will keep using old value.
worker id1: value = 1:
worker id2: value = 2:
worker id3: value = 1:

It says in documentation that shared.dict is shared memory for all workers.
But it looks like it is shared by creating a local copy of data on initialization instead of actually being shared.

Is there a way in NGINX + Lua to have a global variable that can be updated which all workers can use?
On Mon, Aug 19, 2019 at 7:02 AM Nishant Kumar <nishan...@kayzen.io> wrote:
Hi Nikolaj,
You should use
local metadata = ngx.shared.metadata
local succ, err, forcible = metadata:incr("last", 1)
ngx.shared.DICT expose a lot of API's to play with it. Please use it.

Please refer:
https://github.com/openresty/lua-nginx-module#ngxshareddictincr

On Saturday, August 17, 2019 at 9:50:05 AM UTC+5:30, Nikolaj Odel wrote:
Hi,
I am working with NGINX server and I need to make sure all workers are dynamically sharing in-memory data. When data is updated, all workers should be up-to-date.

Reading the docs ngx.shared.DICT can be used for sharing data among workers. So i put that to the test, and it doesn't work.

It seems ngx.shared created copy data for each worker, instead of having one data for all workers.

Here is my simple test code.

nginx.conf
conf

worker_processes  4;

events {
    worker_connections  5;
}

http{

    lua_shared_dict metadata 10m;

    init_by_lua_file lua/init.lua;

    server{
        listen 80 reuseport;

        location / {
            return 200 "Hello";
        }

        location =/worker {
            content_by_lua '
                ngx.say("worker id: " .. ngx.var.pid)
                ngx.say("worker count: " .. ngx.worker.count())

                local metadata = ngx.shared.metadata;
                ngx.say("shared data: " .. metadata["last"])
            ';
        }


        location =/change{
            content_by_lua '
            local metadata = ngx.shared.metadata;
            local t = metadata["last"];
            metadata["last"] = t + 1;
            ';
        }

    }
}
init.lua
ngx.shared.metadata["last"] = 1;
On initialization a shared value last is declared with value 1.

After I make /change request and increment the value +1, only 1/4 workers will have shared value updated, while the others are going to keep using the old value. Probably because that is the worker that caught the request.

I am curling the server in a loop to see the results updated. And in the printout i can see that exactly which worker has newer version
worker id: 10
worker count: 4
shared data: 1
----
worker id: 9
worker count: 4
shared data: 2
I am using dockerized NGINX with open resty Lua module. And also activated the reuseport directive which forces nginx to loadbalance to all workers (just for developing).

How do I make all nginx workers share the date from shared dict dynamically? Is there a system in nginx that can?
Can someone please help,

Thank you

moonbingbing

the shared dict only supports API instead of hash notation, so you can use the `set` and `incr` to change the value of shared dict:

local metadata = ngx.shared.metadata
metadata:set("value", 1)

Jarmo Isotalo <ja...@isotalo.fi> 于2019年8月19日周一下午7:00写道：

Is that a valid way to use the DICT API?
As Nishant mentioned, try using the incr and get APIs instead of the hash notation.
On Mon, Aug 19, 2019 at 12:14 PM Nikolaj Odel <odel....@gmail.com> wrote:
Thanks for answering Nishant,
I was looking at the documentation,

the issue is when i increment a value in shared dict, that value is not dynamically shared by all workers.
Only the worker that catches the request will have updated value
for example
on init:
ngx.shared.metadata["value"] = 1
worker id1: value = 1:
worker id2: value = 1:
worker id3: value = 1:

i increment the shared value to 2, with get request "/change". Only worker that catches the request will update the value, the rest will keep using old value.
worker id1: value = 1:
worker id2: value = 2:
worker id3: value = 1:

It says in documentation that shared.dict is shared memory for all workers.
But it looks like it is shared by creating a local copy of data on initialization instead of actually being shared.

Is there a way in NGINX + Lua to have a global variable that can be updated which all workers can use?
On Mon, Aug 19, 2019 at 7:02 AM Nishant Kumar <nishan...@kayzen.io> wrote:
Hi Nikolaj,
You should use
local metadata = ngx.shared.metadata
local succ, err, forcible = metadata:incr("last", 1)
ngx.shared.DICT expose a lot of API's to play with it. Please use it.

Please refer:
https://github.com/openresty/lua-nginx-module#ngxshareddictincr

On Saturday, August 17, 2019 at 9:50:05 AM UTC+5:30, Nikolaj Odel wrote:
Hi,
I am working with NGINX server and I need to make sure all workers are dynamically sharing in-memory data. When data is updated, all workers should be up-to-date.

Reading the docs ngx.shared.DICT can be used for sharing data among workers. So i put that to the test, and it doesn't work.

It seems ngx.shared created copy data for each worker, instead of having one data for all workers.

Here is my simple test code.

nginx.conf
conf

worker_processes  4;

events {
    worker_connections  5;
}

http{

    lua_shared_dict metadata 10m;

    init_by_lua_file lua/init.lua;

    server{
        listen 80 reuseport;

        location / {
            return 200 "Hello";
        }

        location =/worker {
            content_by_lua '
                ngx.say("worker id: " .. ngx.var.pid)
                ngx.say("worker count: " .. ngx.worker.count())

                local metadata = ngx.shared.metadata;
                ngx.say("shared data: " .. metadata["last"])
            ';
        }


        location =/change{
            content_by_lua '
            local metadata = ngx.shared.metadata;
            local t = metadata["last"];
            metadata["last"] = t + 1;
            ';
        }

    }
}
init.lua
ngx.shared.metadata["last"] = 1;
On initialization a shared value last is declared with value 1.

After I make /change request and increment the value +1, only 1/4 workers will have shared value updated, while the others are going to keep using the old value. Probably because that is the worker that caught the request.

I am curling the server in a loop to see the results updated. And in the printout i can see that exactly which worker has newer version
worker id: 10
worker count: 4
shared data: 1
----
worker id: 9
worker count: 4
shared data: 2
I am using dockerized NGINX with open resty Lua module. And also activated the reuseport directive which forces nginx to loadbalance to all workers (just for developing).

How do I make all nginx workers share the date from shared dict dynamically? Is there a system in nginx that can?
Can someone please help,

Thank you

odel.nikolaj

Yes, you were all correct.

I was using the API in a bad way. I was using syntax i would normally use in lua, as if it was an array.

so instead of metadata[index] switch to metadata:get(index)

Thank you all.

On Tue, Aug 20, 2019 at 7:56 AM Ming <moonb...@gmail.com> wrote:

the shared dict only supports API instead of hash notation, so you can use the `set` and `incr` to change the value of shared dict:
local metadata = ngx.shared.metadata
metadata:set("value", 1)
Jarmo Isotalo <ja...@isotalo.fi> 于2019年8月19日周一下午7:00写道：
Is that a valid way to use the DICT API?
As Nishant mentioned, try using the incr and get APIs instead of the hash notation.
On Mon, Aug 19, 2019 at 12:14 PM Nikolaj Odel <odel....@gmail.com> wrote:
Thanks for answering Nishant,
I was looking at the documentation,

the issue is when i increment a value in shared dict, that value is not dynamically shared by all workers.
Only the worker that catches the request will have updated value
for example
on init:
ngx.shared.metadata["value"] = 1
worker id1: value = 1:
worker id2: value = 1:
worker id3: value = 1:

i increment the shared value to 2, with get request "/change". Only worker that catches the request will update the value, the rest will keep using old value.
worker id1: value = 1:
worker id2: value = 2:
worker id3: value = 1:

It says in documentation that shared.dict is shared memory for all workers.
But it looks like it is shared by creating a local copy of data on initialization instead of actually being shared.

Is there a way in NGINX + Lua to have a global variable that can be updated which all workers can use?
On Mon, Aug 19, 2019 at 7:02 AM Nishant Kumar <nishan...@kayzen.io> wrote:
Hi Nikolaj,
You should use
local metadata = ngx.shared.metadata
local succ, err, forcible = metadata:incr("last", 1)
ngx.shared.DICT expose a lot of API's to play with it. Please use it.

Please refer:
https://github.com/openresty/lua-nginx-module#ngxshareddictincr

On Saturday, August 17, 2019 at 9:50:05 AM UTC+5:30, Nikolaj Odel wrote:
Hi,
I am working with NGINX server and I need to make sure all workers are dynamically sharing in-memory data. When data is updated, all workers should be up-to-date.

Reading the docs ngx.shared.DICT can be used for sharing data among workers. So i put that to the test, and it doesn't work.

It seems ngx.shared created copy data for each worker, instead of having one data for all workers.

Here is my simple test code.

nginx.conf
conf

worker_processes  4;

events {
    worker_connections  5;
}

http{

    lua_shared_dict metadata 10m;

    init_by_lua_file lua/init.lua;

    server{
        listen 80 reuseport;

        location / {
            return 200 "Hello";
        }

        location =/worker {
            content_by_lua '
                ngx.say("worker id: " .. ngx.var.pid)
                ngx.say("worker count: " .. ngx.worker.count())

                local metadata = ngx.shared.metadata;
                ngx.say("shared data: " .. metadata["last"])
            ';
        }


        location =/change{
            content_by_lua '
            local metadata = ngx.shared.metadata;
            local t = metadata["last"];
            metadata["last"] = t + 1;
            ';
        }

    }
}
init.lua
ngx.shared.metadata["last"] = 1;
On initialization a shared value last is declared with value 1.

After I make /change request and increment the value +1, only 1/4 workers will have shared value updated, while the others are going to keep using the old value. Probably because that is the worker that caught the request.

I am curling the server in a loop to see the results updated. And in the printout i can see that exactly which worker has newer version
worker id: 10
worker count: 4
shared data: 1
----
worker id: 9
worker count: 4
shared data: 2
I am using dockerized NGINX with open resty Lua module. And also activated the reuseport directive which forces nginx to loadbalance to all workers (just for developing).

How do I make all nginx workers share the date from shared dict dynamically? Is there a system in nginx that can?
Can someone please help,

Thank you