Passing data to ngx.timer.at callbacks

manchev · 2014-05-27T07:38:40+00:00

Hello! On Fri, May 30, 2014 at 5:10 AM, Pravin tatti wrote: > I am using Modsecurity + Nginx if you see the modsec plugin for nginx. All > the rules a...

Passing data to ngx.timer.at callbacks

manchev

Hi all,

I've written a simple websocket server to test some concepts:

local server = require("resty.websocket.server")
local ws, err = server:new({
  timeout = 1000,
  max_payload_len = 65535
})

local stats
stats, err = redis:hgetall("stats")
ok, err = ws:send_text(json.encode(stats))

handler = function(premature, ws)
  if premature then
    return 
  end
  redis = require("resty.redis").new()
  redis:connect("unix:/tmp/redis.sock")
  stats, err = redis:hgetall("stats")
  ok, err = ws:send_text(json.encode(stats))
  ok, err = ngx.timer.at(60, handler)
end

ok, err = ngx.timer.at(0, handler, ws)

Here's a simple nginx conf to test this:

worker_processes 4;
error_log stderr notice;

events {
  worker_connections 1024;
}

http {
  include mime.types;

  server {
    listen 80 so_keepalive=30s:30s:8;

    location /ws {
      lua_socket_log_errors off;
      content_by_lua_file "ws.lua";
    }
  }
}

Basically a client connects and gets some data, then a recurring timer is set to query Redis and push data through the websocket connection.

This does not work though even if I pass the websocket object as a parameter to the ngx.timer.at callback. Same goes for the Redis connection (it would be nice to reuse the current connection instead of initiating a new one), but I don't care much about that as I'm using a pool.

So, is there any way around this? I don't think that creating a new websocket connection to the client in the context of the callback makes much sense.

A possible work around would be to use Redis pub/sub and publish messages in the ngx.timer.at callback and subscribe in the outer context in order to send data over the already established websocket connection.

Any suggestions are welcome, really interested to see different approaches.

Best,
Vladislav

manchev

Just realized I forgot to mention that the code fails with "bad request", while nginx starts eating 100% of the CPU time in about a minute.

Also,

local json = require("cjson")
local redis = require("resty.redis").new()

is needed on top of the lua code as it's used in this example, but any string will do and Redis code can be dropped too.

On Tuesday, May 27, 2014 2:38:39 AM UTC+3, Vladislav Manchev wrote:

Hi all,

I've written a simple websocket server to test some concepts:
local server = require("resty.websocket.server") local ws, err = server:new({ timeout = 1000, max_payload_len = 65535 }) local stats stats, err = redis:hgetall("stats") ok, err = ws:send_text(json.encode(stats)) handler = function(premature, ws) if premature then return end redis = require("resty.redis").new() redis:connect("unix:/tmp/redis.sock") stats, err = redis:hgetall("stats") ok, err = ws:send_text(json.encode(stats)) ok, err = ngx.timer.at(60, handler) end ok, err = ngx.timer.at(0, handler, ws)

Here's a simple nginx conf to test this:
worker_processes 4; error_log stderr notice; events { worker_connections 1024; } http { include mime.types; server { listen 80 so_keepalive=30s:30s:8; location /ws { lua_socket_log_errors off; content_by_lua_file "ws.lua"; } } }

Basically a client connects and gets some data, then a recurring timer is set to query Redis and push data through the websocket connection.

This does not work though even if I pass the websocket object as a parameter to the ngx.timer.at callback. Same goes for the Redis connection (it would be nice to reuse the current connection instead of initiating a new one), but I don't care much about that as I'm using a pool.

So, is there any way around this? I don't think that creating a new websocket connection to the client in the context of the callback makes much sense.

A possible work around would be to use Redis pub/sub and publish messages in the ngx.timer.at callback and subscribe in the outer context in order to send data over the already established websocket connection.

Any suggestions are welcome, really interested to see different approaches.

Best,
Vladislav

hughserious

hello,

how does it work with local variables like "ws". do they go out of scope at the end of Lua for each request? . I've been trying to use the sharing data trick here to keep things around, but that's not what you want either.

my guess is you need what i've been trying to do, which is to have some sort of per-user-session "bag" so you can put your connections and other state inside it.

i'm expecting someone to point out something really obvious since this sort of problem seems to be the very first thing anyone using this sort of setup will encounter.

good luck with your stuff.

manchev

On 05/27/2014 04:03 AM, hugh wrote:

hello,

how does it work with local variables like "ws". do they go out of scope at the end of Lua for each request? . I've been trying to use the sharing data trick here to keep things around, but that's not what you want either.

Well, seems that ngx.timer.at executes code in a new Lua VM and passing objects between your original instance to the new one that exists inside ngx.timer.at context is not possible. Maybe I'm wrong and there's something else so I'll be greatful if Yichun comments on this.

Don't think the approach you pointed out will work for me as I want to use more than one worker.
Also, sharing with the lua_shared_dict directive does not work with user data as far as I know.

my guess is you need what i've been trying to do, which is to have some sort of per-user-session "bag" so you can put your connections and other state inside it.

Yes, trying to do something very similar.

i'm expecting someone to point out something really obvious since this sort of problem seems to be the very first thing anyone using this sort of setup will encounter.

Still, Redis pub/sub is a viable work around for me if nothing obvious pops up.

good luck with your stuff.

Thanks!

.

hamish.forbes

Hi, I think you're misunderstanding how the ngx.timer.at feature is supposed to work.

Its intended for running background jobs etc, not for delaying execution of code within a request.

https://github.com/openresty/lua-nginx-module#ngxtimerat

"When a timer expires, the user Lua code in the timer callback is running in a "light thread" detached completely from the original request creating the timer. So objects with the same lifetime as the request creating them, like cosockets, cannot be shared between the original request and the timer user callback function."

"0 delay can also be specified, in which case the timer will immediately expire when the current handler yields execution."

Essentially you're trying to run code that is running in a seperate 'fake' request totally independent of the initial request and will only be executed once the content phase has finished.

Try using ngx.sleep in a loop instead.

Hamish

On Tuesday, 27 May 2014 00:38:39 UTC+1, Vladislav Manchev wrote:

Hi all,

I've written a simple websocket server to test some concepts:
Basically a client connects and gets some data, then a recurring timer is set to query Redis and push data through the websocket connection.

This does not work though even if I pass the websocket object as a parameter to the ngx.timer.at callback. Same goes for the Redis connection (it would be nice to reuse the current connection instead of initiating a new one), but I don't care much about that as I'm using a pool.

So, is there any way around this? I don't think that creating a new websocket connection to the client in the context of the callback makes much sense.

A possible work around would be to use Redis pub/sub and publish messages in the ngx.timer.at callback and subscribe in the outer context in order to send data over the already established websocket connection.

Any suggestions are welcome, really interested to see different approaches.

Best,
Vladislav

manchev

Thanks for the input, Hamish.

I actually understand that code runs in its own light thread, so it makes sense that none of the data from the outer context is available in the ngx.timer.at callback, I was just looking for a way to pass some of this data.

On the other hand, I already have some code in an infinite while loop in the outer context so adding another infinite loop with ngx.sleep would be useless here I think.

Best,
Vladislav

On 05/27/2014 11:18 AM, Hamish Forbes wrote:

Hi, I think you're misunderstanding how the ngx.timer.at feature is supposed to work.
Its intended for running background jobs etc, not for delaying execution of code within a request.

https://github.com/openresty/lua-nginx-module#ngxtimerat

"When a timer expires, the user Lua code in the timer callback is running in a "light thread" detached completely from the original request creating the timer. So objects with the same lifetime as the request creating them, like cosockets, cannot be shared between the original request and the timer user callback function."

"0 delay can also be specified, in which case the timer will immediately expire when the current handler yields execution."

Essentially you're trying to run code that is running in a seperate 'fake' request totally independent of the initial request and will only be executed once the content phase has finished.

Try using ngx.sleep in a loop instead.

Hamish

On Tuesday, 27 May 2014 00:38:39 UTC+1, Vladislav Manchev wrote:

Hi all,

I've written a simple websocket server to test some concepts:

Basically a client connects and gets some data, then a recurring timer is set to query Redis and push data through the websocket connection.

This does not work though even if I pass the websocket object as a parameter to the ngx.timer.at callback. Same goes for the Redis connection (it would be nice to reuse the current connection instead of initiating a new one), but I don't care much about that as I'm using a pool.

So, is there any way around this? I don't think that creating a new websocket connection to the client in the context of the callback makes much sense.

A possible work around would be to use Redis pub/sub and publish messages in the ngx.timer.at callback and subscribe in the outer context in order to send data over the already established websocket connection.

Any suggestions are welcome, really interested to see different approaches.

Best,
Vladislav

.

hamish.forbes

You can pass in data, but you can't pass in the current request and be able to pass data back to the client though.

On Tuesday, 27 May 2014 10:54:28 UTC+1, Vladislav Manchev wrote:

Thanks for the input, Hamish.

I actually understand that code runs in its own light thread, so it makes sense that none of the data from the outer context is available in the ngx.timer.at callback, I was just looking for a way to pass some of this data.

On the other hand, I already have some code in an infinite while loop in the outer context so adding another infinite loop with ngx.sleep would be useless here I think.

Best,
Vladislav

On 05/27/2014 11:18 AM, Hamish Forbes wrote:

Hi, I think you're misunderstanding how the ngx.timer.at feature is supposed to work.
Its intended for running background jobs etc, not for delaying execution of code within a request.

https://github.com/openresty/lua-nginx-module#ngxtimerat

"When a timer expires, the user Lua code in the timer callback is running in a "light thread" detached completely from the original request creating the timer. So objects with the same lifetime as the request creating them, like cosockets, cannot be shared between the original request and the timer user callback function."

"0 delay can also be specified, in which case the timer will immediately expire when the current handler yields execution."

Essentially you're trying to run code that is running in a seperate 'fake' request totally independent of the initial request and will only be executed once the content phase has finished.

Try using ngx.sleep in a loop instead.

Hamish

On Tuesday, 27 May 2014 00:38:39 UTC+1, Vladislav Manchev wrote:

Hi all,

I've written a simple websocket server to test some concepts:

Basically a client connects and gets some data, then a recurring timer is set to query Redis and push data through the websocket connection.

This does not work though even if I pass the websocket object as a parameter to the ngx.timer.at callback. Same goes for the Redis connection (it would be nice to reuse the current connection instead of initiating a new one), but I don't care much about that as I'm using a pool.

So, is there any way around this? I don't think that creating a new websocket connection to the client in the context of the callback makes much sense.

A possible work around would be to use Redis pub/sub and publish messages in the ngx.timer.at callback and subscribe in the outer context in order to send data over the already established websocket connection.

Any suggestions are welcome, really interested to see different approaches.

Best,
Vladislav

.

manchev

Right, I can pass "simple" data, not the current request with all attached state.

In any case, I have some great options here and was only interested in different ways to solve the initial problem, but it's obvious now that the initial approach is wrong in the context of ngx_lua.

Thanks for the sensible discussion!

On 05/27/2014 01:45 PM, Hamish Forbes wrote:

You can pass in data, but you can't pass in the current request and be able to pass data back to the client though.

On Tuesday, 27 May 2014 10:54:28 UTC+1, Vladislav Manchev wrote:

Thanks for the input, Hamish.

I actually understand that code runs in its own light thread, so it makes sense that none of the data from the outer context is available in the ngx.timer.at callback, I was just looking for a way to pass some of this data.

On the other hand, I already have some code in an infinite while loop in the outer context so adding another infinite loop with ngx.sleep would be useless here I think.

Best,
Vladislav

On 05/27/2014 11:18 AM, Hamish Forbes wrote:

Hi, I think you're misunderstanding how the ngx.timer.at feature is supposed to work.
Its intended for running background jobs etc, not for delaying execution of code within a request.

https://github.com/openresty/lua-nginx-module#ngxtimerat

"When a timer expires, the user Lua code in the timer callback is running in a "light thread" detached completely from the original request creating the timer. So objects with the same lifetime as the request creating them, like cosockets, cannot be shared between the original request and the timer user callback function."

"0 delay can also be specified, in which case the timer will immediately expire when the current handler yields execution."

Essentially you're trying to run code that is running in a seperate 'fake' request totally independent of the initial request and will only be executed once the content phase has finished.

Try using ngx.sleep in a loop instead.

Hamish

On Tuesday, 27 May 2014 00:38:39 UTC+1, Vladislav Manchev wrote:

Hi all,

I've written a simple websocket server to test some concepts:

Basically a client connects and gets some data, then a recurring timer is set to query Redis and push data through the websocket connection.

This does not work though even if I pass the websocket object as a parameter to the ngx.timer.at callback. Same goes for the Redis connection (it would be nice to reuse the current connection instead of initiating a new one), but I don't care much about that as I'm using a pool.

So, is there any way around this? I don't think that creating a new websocket connection to the client in the context of the callback makes much sense.

A possible work around would be to use Redis pub/sub and publish messages in the ngx.timer.at callback and subscribe in the outer context in order to send data over the already established websocket connection.

Any suggestions are welcome, really interested to see different approaches.

Best,
Vladislav

.

agentzh

Hello!

On Mon, May 26, 2014 at 4:38 PM, Vladislav Manchev wrote:
> This does not work though even if I pass the websocket object as a parameter
> to the ngx.timer.at callback. Same goes for the Redis connection (it would
> be nice to reuse the current connection instead of initiating a new one),
> but I don't care much about that as I'm using a pool.
>

No, never pass any cosocket objects (including the websocket ones) to
the timer handler! Because the cosocket object is strictly bound to
the request creating it while the timer callback is detached from the
request creating the timer. Using the cosocket object out of the
boundary the creating request will yield the "bad request" error.

Also, never use a recurring timer chain with 0 timeout, because you're
essentially running a hot tight infinite loop here, which does no good
by exhausting the CPU resources

BTW, the timer callback is never never run in a separate Lua VM, even
when lua_code_cache is turned off :)

Regards,
-agentzh