Long-running HTTP/WS client connections to upstream?

igor.clark · 2015-05-27T02:03:10+00:00

Useful info, thanks all. Sounds like, basically, yeah, it's going to block wherever it does the work, so we'll have to deal with that somehow else.Cheers for...

Long-running HTTP/WS client connections to upstream?

igor.clark

Hello,

- I have an upstream server which publishes events via websocket and/or EventSource channels as necessary.

- I want my OpenResty app to know about and act on those events straightaway.

- I don't want the upstream server to have to maintain a list of downstream servers, so I want the downstream app to subscribe to the channel.

It'd be pretty straightforward to create a supervised process, running as a separate script on the OpenResty app servers, which listens to the upstream channel and informs the OpenResty app by hitting appropriate local URLs, according to incoming message data. (It's a pretty simple data model in this part of the application, and this would be consistent enough for our purposes.) But it seems like it'd be so much better/neater to do it inside OpenResty, if it can be done, without having to run supervisors, other script runtimes and so forth.

So is it possible to:

- open a long-running resty.websocket.client, lua-resty-http or other client connection when the OpenResty server starts;

- set a Lua event handler function to deal with incoming messages received on the connection,

- notify local URLs via HTTP requests in the handler function, and

- leave it running as long as nginx stays running?

- Ideally with a handler for socket errors so that it could try to reconnect if the connection failed?

If so, what's the best way to do it? And at what point in the system should this happen? I've been reading a lot about the various phases and when which APIs are available, e.g. not being able to call ngx.location.capture from "light threads" started by ngx.timer.at - but I'm struggling to get it at all clear in my head at the moment.

Would really appreciate some tips as to how to go about this, if it's possible!

Thanks very much,

Igor

manchev

You could use something along the lines of this gist [0]. It's a pretty stripped down version of one of my scripts, you should definitely add more error handling.

I've got some code lying around that could help with keeping a persistent connection (i.e. reconnecting on all fatal errors), but that is pretty specific to my use case and you should be able to easily implement it anyway.

One thing to be careful with is getting into hot tight infinite loops which will exhaust CPU resources.

[0]: https://gist.github.com/blablacio/9d81aad39e587eacc881

Best,

Vladislav

On Tue, May 26, 2015 at 9:03 PM, Igor Clark <igor...@gmail.com> wrote:

Hello,

- I have an upstream server which publishes events via websocket and/or EventSource channels as necessary.
- I want my OpenResty app to know about and act on those events straightaway.
- I don't want the upstream server to have to maintain a list of downstream servers, so I want the downstream app to subscribe to the channel.

It'd be pretty straightforward to create a supervised process, running as a separate script on the OpenResty app servers, which listens to the upstream channel and informs the OpenResty app by hitting appropriate local URLs, according to incoming message data. (It's a pretty simple data model in this part of the application, and this would be consistent enough for our purposes.) But it seems like it'd be so much better/neater to do it inside OpenResty, if it can be done, without having to run supervisors, other script runtimes and so forth.

So is it possible to:

- open a long-running resty.websocket.client, lua-resty-http or other client connection when the OpenResty server starts;
- set a Lua event handler function to deal with incoming messages received on the connection,
- notify local URLs via HTTP requests in the handler function, and
- leave it running as long as nginx stays running?
- Ideally with a handler for socket errors so that it could try to reconnect if the connection failed?

If so, what's the best way to do it? And at what point in the system should this happen? I've been reading a lot about the various phases and when which APIs are available, e.g. not being able to call ngx.location.capture from "light threads" started by ngx.timer.at - but I'm struggling to get it at all clear in my head at the moment.

Would really appreciate some tips as to how to go about this, if it's possible!

Thanks very much,
Igor

.

igor.clark

Thanks Vladislav, this looks like what I'm looking for, and got me started with a websocket client connecting upstream on nginx start. Appreciate it!

It seems to run the code once for each worker, including cache manager and cache loader processes. Which makes sense, as it's "init_worker_by_lua". If I try to run something similar in "init_by_lua" I just get "no request" on the ngx.timer.at call.

Is there an idiomatic ngx_lua way to change that behaviour, or do I need to manage it in Lua code?

Assuming it has to be in Lua, right now I've managed to get it to only start one client by doing this:

- setting a "connected" flag and a "pause" value in an ngx.shared.DICT

- incrementing the pause value in the dict each time through my module's "init" function

- setting the delay on the ngx.timer.at (in the init function) to the value in ngx.shared.DICT:get("pause"), and

- only making the connection if the "connected" flag is not set.

This way each worker starts 1 second after the last one, and it works - but it seems clunky, lock-like and error-prone. Importantly I also don't know if e.g. the cache loader process might start first, hence killing the client.

Would love to know if I'm at least on a vaguely right track, or just being a total newb :-)

Thanks so much for your help!

Best,

Igor

On Wednesday, May 27, 2015 at 11:21:06 AM UTC+1, Vladislav Manchev wrote:

You could use something along the lines of this gist [0]. It's a pretty stripped down version of one of my scripts, you should definitely add more error handling.

I've got some code lying around that could help with keeping a persistent connection (i.e. reconnecting on all fatal errors), but that is pretty specific to my use case and you should be able to easily implement it anyway.

One thing to be careful with is getting into hot tight infinite loops which will exhaust CPU resources.

[0]: https://gist.github.com/blablacio/9d81aad39e587eacc881

Best,
Vladislav

On Tue, May 26, 2015 at 9:03 PM, Igor Clark <igor....@gmail.com> wrote:
Hello,

- I have an upstream server which publishes events via websocket and/or EventSource channels as necessary.
- I want my OpenResty app to know about and act on those events straightaway.
- I don't want the upstream server to have to maintain a list of downstream servers, so I want the downstream app to subscribe to the channel.

It'd be pretty straightforward to create a supervised process, running as a separate script on the OpenResty app servers, which listens to the upstream channel and informs the OpenResty app by hitting appropriate local URLs, according to incoming message data. (It's a pretty simple data model in this part of the application, and this would be consistent enough for our purposes.) But it seems like it'd be so much better/neater to do it inside OpenResty, if it can be done, without having to run supervisors, other script runtimes and so forth.

So is it possible to:

- open a long-running resty.websocket.client, lua-resty-http or other client connection when the OpenResty server starts;
- set a Lua event handler function to deal with incoming messages received on the connection,
- notify local URLs via HTTP requests in the handler function, and
- leave it running as long as nginx stays running?
- Ideally with a handler for socket errors so that it could try to reconnect if the connection failed?

If so, what's the best way to do it? And at what point in the system should this happen? I've been reading a lot about the various phases and when which APIs are available, e.g. not being able to call ngx.location.capture from "light threads" started by ngx.timer.at - but I'm struggling to get it at all clear in my head at the moment.

Would really appreciate some tips as to how to go about this, if it's possible!

Thanks very much,
Igor

.

manchev

Yep, init_by_lua context allows only logging and shared dict APIs, so you must use init_worker_by_lua right now.

If you're only interested in running the code only in one worker you could use shared dict add [0] to ensure that, see an example here [1].

Not entirely sure what you're trying to achieve and why you need the pause value, but usually if you're loading some stuff to cache (shared dict, Redis, etc.) you would want to do this from one worker only. Managing/updating the cache is an entirely different story and you could either do it from a single worker like cache loading or use multiple workers with some kind of locking to achieve this depending on your data and data sources.

[0]: https://github.com/openresty/lua-nginx-module/#ngxshareddictadd

[1]: https://gist.github.com/blablacio/2c940d20bc754716d842

Best,

Vladislav

On Thu, May 28, 2015 at 2:34 AM, Igor Clark <igor...@gmail.com> wrote:

Thanks Vladislav, this looks like what I'm looking for, and got me started with a websocket client connecting upstream on nginx start. Appreciate it!

It seems to run the code once for each worker, including cache manager and cache loader processes. Which makes sense, as it's "init_worker_by_lua". If I try to run something similar in "init_by_lua" I just get "no request" on the ngx.timer.at call.

Is there an idiomatic ngx_lua way to change that behaviour, or do I need to manage it in Lua code?

Assuming it has to be in Lua, right now I've managed to get it to only start one client by doing this:

- setting a "connected" flag and a "pause" value in an ngx.shared.DICT
- incrementing the pause value in the dict each time through my module's "init" function
- setting the delay on the ngx.timer.at (in the init function) to the value in ngx.shared.DICT:get("pause"), and
- only making the connection if the "connected" flag is not set.

This way each worker starts 1 second after the last one, and it works - but it seems clunky, lock-like and error-prone. Importantly I also don't know if e.g. the cache loader process might start first, hence killing the client.

Would love to know if I'm at least on a vaguely right track, or just being a total newb :-)

Thanks so much for your help!

Best,
Igor

On Wednesday, May 27, 2015 at 11:21:06 AM UTC+1, Vladislav Manchev wrote:
You could use something along the lines of this gist [0]. It's a pretty stripped down version of one of my scripts, you should definitely add more error handling.

I've got some code lying around that could help with keeping a persistent connection (i.e. reconnecting on all fatal errors), but that is pretty specific to my use case and you should be able to easily implement it anyway.

One thing to be careful with is getting into hot tight infinite loops which will exhaust CPU resources.

[0]: https://gist.github.com/blablacio/9d81aad39e587eacc881

Best,
Vladislav

On Tue, May 26, 2015 at 9:03 PM, Igor Clark <igor....@gmail.com> wrote:
Hello,

- I have an upstream server which publishes events via websocket and/or EventSource channels as necessary.
- I want my OpenResty app to know about and act on those events straightaway.
- I don't want the upstream server to have to maintain a list of downstream servers, so I want the downstream app to subscribe to the channel.

It'd be pretty straightforward to create a supervised process, running as a separate script on the OpenResty app servers, which listens to the upstream channel and informs the OpenResty app by hitting appropriate local URLs, according to incoming message data. (It's a pretty simple data model in this part of the application, and this would be consistent enough for our purposes.) But it seems like it'd be so much better/neater to do it inside OpenResty, if it can be done, without having to run supervisors, other script runtimes and so forth.

So is it possible to:

- open a long-running resty.websocket.client, lua-resty-http or other client connection when the OpenResty server starts;
- set a Lua event handler function to deal with incoming messages received on the connection,
- notify local URLs via HTTP requests in the handler function, and
- leave it running as long as nginx stays running?
- Ideally with a handler for socket errors so that it could try to reconnect if the connection failed?

If so, what's the best way to do it? And at what point in the system should this happen? I've been reading a lot about the various phases and when which APIs are available, e.g. not being able to call ngx.location.capture from "light threads" started by ngx.timer.at - but I'm struggling to get it at all clear in my head at the moment.

Would really appreciate some tips as to how to go about this, if it's possible!

Thanks very much,
Igor

.

hamish.forbes

I asked about the correct way of doing this a while back and created this gist based on Yichun's explanation

https://gist.github.com/hamishforbes/1bdc679893eabbbab3bd

This handles things like a worker getting stuck and never releasing locks etc.

Hamish

On Thursday, 28 May 2015 00:34:31 UTC+1, Igor Clark wrote:

Thanks Vladislav, this looks like what I'm looking for, and got me started with a websocket client connecting upstream on nginx start. Appreciate it!

It seems to run the code once for each worker, including cache manager and cache loader processes. Which makes sense, as it's "init_worker_by_lua". If I try to run something similar in "init_by_lua" I just get "no request" on the ngx.timer.at call.

Is there an idiomatic ngx_lua way to change that behaviour, or do I need to manage it in Lua code?

Assuming it has to be in Lua, right now I've managed to get it to only start one client by doing this:

- setting a "connected" flag and a "pause" value in an ngx.shared.DICT
- incrementing the pause value in the dict each time through my module's "init" function
- setting the delay on the ngx.timer.at (in the init function) to the value in ngx.shared.DICT:get("pause"), and
- only making the connection if the "connected" flag is not set.

This way each worker starts 1 second after the last one, and it works - but it seems clunky, lock-like and error-prone. Importantly I also don't know if e.g. the cache loader process might start first, hence killing the client.

Would love to know if I'm at least on a vaguely right track, or just being a total newb :-)

Thanks so much for your help!

Best,
Igor

On Wednesday, May 27, 2015 at 11:21:06 AM UTC+1, Vladislav Manchev wrote:
You could use something along the lines of this gist [0]. It's a pretty stripped down version of one of my scripts, you should definitely add more error handling.

I've got some code lying around that could help with keeping a persistent connection (i.e. reconnecting on all fatal errors), but that is pretty specific to my use case and you should be able to easily implement it anyway.

One thing to be careful with is getting into hot tight infinite loops which will exhaust CPU resources.

[0]: https://gist.github.com/blablacio/9d81aad39e587eacc881

Best,
Vladislav

On Tue, May 26, 2015 at 9:03 PM, Igor Clark <igor....@gmail.com> wrote:
Hello,

- I have an upstream server which publishes events via websocket and/or EventSource channels as necessary.
- I want my OpenResty app to know about and act on those events straightaway.
- I don't want the upstream server to have to maintain a list of downstream servers, so I want the downstream app to subscribe to the channel.

It'd be pretty straightforward to create a supervised process, running as a separate script on the OpenResty app servers, which listens to the upstream channel and informs the OpenResty app by hitting appropriate local URLs, according to incoming message data. (It's a pretty simple data model in this part of the application, and this would be consistent enough for our purposes.) But it seems like it'd be so much better/neater to do it inside OpenResty, if it can be done, without having to run supervisors, other script runtimes and so forth.

So is it possible to:

- open a long-running resty.websocket.client, lua-resty-http or other client connection when the OpenResty server starts;
- set a Lua event handler function to deal with incoming messages received on the connection,
- notify local URLs via HTTP requests in the handler function, and
- leave it running as long as nginx stays running?
- Ideally with a handler for socket errors so that it could try to reconnect if the connection failed?

If so, what's the best way to do it? And at what point in the system should this happen? I've been reading a lot about the various phases and when which APIs are available, e.g. not being able to call ngx.location.capture from "light threads" started by ngx.timer.at - but I'm struggling to get it at all clear in my head at the moment.

Would really appreciate some tips as to how to go about this, if it's possible!

Thanks very much,
Igor

.

igor.clark

Thanks Vladislav and Hamish, these are both really helpful.

Vladislav, sorry, I didn't explain very clearly. I just want to only have one worker do this, so your example helps. The pause was due to a silly mistake on my part - I was only setting the flag in the dictionary once the connection had been made, so of course workers 2/3/4 were not seeing it set, as they started before the connection was established. Doing the flag/lock differently fixes this.

Hamish, that's very useful, thanks, looks like it'll help with the locking, and hence some of the problems I'm immediately coming up against. I'll try it out, study what it's doing further and do some more reading of the docs.

Appreciate your help, both :-)

On Thursday, May 28, 2015 at 10:17:55 AM UTC+1, Hamish Forbes wrote:

I asked about the correct way of doing this a while back and created this gist based on Yichun's explanation
https://gist.github.com/hamishforbes/1bdc679893eabbbab3bd

This handles things like a worker getting stuck and never releasing locks etc.

Hamish

On Thursday, 28 May 2015 00:34:31 UTC+1, Igor Clark wrote:
Thanks Vladislav, this looks like what I'm looking for, and got me started with a websocket client connecting upstream on nginx start. Appreciate it!

It seems to run the code once for each worker, including cache manager and cache loader processes. Which makes sense, as it's "init_worker_by_lua". If I try to run something similar in "init_by_lua" I just get "no request" on the ngx.timer.at call.

Is there an idiomatic ngx_lua way to change that behaviour, or do I need to manage it in Lua code?

Assuming it has to be in Lua, right now I've managed to get it to only start one client by doing this:

- setting a "connected" flag and a "pause" value in an ngx.shared.DICT
- incrementing the pause value in the dict each time through my module's "init" function
- setting the delay on the ngx.timer.at (in the init function) to the value in ngx.shared.DICT:get("pause"), and
- only making the connection if the "connected" flag is not set.

This way each worker starts 1 second after the last one, and it works - but it seems clunky, lock-like and error-prone. Importantly I also don't know if e.g. the cache loader process might start first, hence killing the client.

Would love to know if I'm at least on a vaguely right track, or just being a total newb :-)

Thanks so much for your help!

Best,
Igor

On Wednesday, May 27, 2015 at 11:21:06 AM UTC+1, Vladislav Manchev wrote:
You could use something along the lines of this gist [0]. It's a pretty stripped down version of one of my scripts, you should definitely add more error handling.

I've got some code lying around that could help with keeping a persistent connection (i.e. reconnecting on all fatal errors), but that is pretty specific to my use case and you should be able to easily implement it anyway.

One thing to be careful with is getting into hot tight infinite loops which will exhaust CPU resources.

[0]: https://gist.github.com/blablacio/9d81aad39e587eacc881

Best,
Vladislav

On Tue, May 26, 2015 at 9:03 PM, Igor Clark <igor....@gmail.com> wrote:
Hello,

- I have an upstream server which publishes events via websocket and/or EventSource channels as necessary.
- I want my OpenResty app to know about and act on those events straightaway.
- I don't want the upstream server to have to maintain a list of downstream servers, so I want the downstream app to subscribe to the channel.

It'd be pretty straightforward to create a supervised process, running as a separate script on the OpenResty app servers, which listens to the upstream channel and informs the OpenResty app by hitting appropriate local URLs, according to incoming message data. (It's a pretty simple data model in this part of the application, and this would be consistent enough for our purposes.) But it seems like it'd be so much better/neater to do it inside OpenResty, if it can be done, without having to run supervisors, other script runtimes and so forth.

So is it possible to:

- open a long-running resty.websocket.client, lua-resty-http or other client connection when the OpenResty server starts;
- set a Lua event handler function to deal with incoming messages received on the connection,
- notify local URLs via HTTP requests in the handler function, and
- leave it running as long as nginx stays running?
- Ideally with a handler for socket errors so that it could try to reconnect if the connection failed?

If so, what's the best way to do it? And at what point in the system should this happen? I've been reading a lot about the various phases and when which APIs are available, e.g. not being able to call ngx.location.capture from "light threads" started by ngx.timer.at - but I'm struggling to get it at all clear in my head at the moment.

Would really appreciate some tips as to how to go about this, if it's possible!

Thanks very much,
Igor

.

igor.clark

Hey, just a quick note to say thanks, I got this working as I wanted, with your help. Turned out pretty simple to get the lock working as I wanted, with a few very useful pointers in the samples you sent. The Lua code listening and responding to events is so much more comprehensible than the alternative in node.js or whatever, and operationally it's *so* much more simple to do this inside nginx than with a separate process that I'd have to supervise etc.

I still have to do some proving in terms of hitting it with lots of data, but this seems to be a great way to do this. openresty + ngx_lua are really getting me excited!

Thanks all,

Igor

On Thursday, May 28, 2015 at 10:39:37 AM UTC+1, Igor Clark wrote:

Thanks Vladislav and Hamish, these are both really helpful.

Vladislav, sorry, I didn't explain very clearly. I just want to only have one worker do this, so your example helps. The pause was due to a silly mistake on my part - I was only setting the flag in the dictionary once the connection had been made, so of course workers 2/3/4 were not seeing it set, as they started before the connection was established. Doing the flag/lock differently fixes this.

Hamish, that's very useful, thanks, looks like it'll help with the locking, and hence some of the problems I'm immediately coming up against. I'll try it out, study what it's doing further and do some more reading of the docs.

Appreciate your help, both :-)

On Thursday, May 28, 2015 at 10:17:55 AM UTC+1, Hamish Forbes wrote:
I asked about the correct way of doing this a while back and created this gist based on Yichun's explanation
https://gist.github.com/hamishforbes/1bdc679893eabbbab3bd

This handles things like a worker getting stuck and never releasing locks etc.

Hamish

On Thursday, 28 May 2015 00:34:31 UTC+1, Igor Clark wrote:
Thanks Vladislav, this looks like what I'm looking for, and got me started with a websocket client connecting upstream on nginx start. Appreciate it!

It seems to run the code once for each worker, including cache manager and cache loader processes. Which makes sense, as it's "init_worker_by_lua". If I try to run something similar in "init_by_lua" I just get "no request" on the ngx.timer.at call.

Is there an idiomatic ngx_lua way to change that behaviour, or do I need to manage it in Lua code?

Assuming it has to be in Lua, right now I've managed to get it to only start one client by doing this:

- setting a "connected" flag and a "pause" value in an ngx.shared.DICT
- incrementing the pause value in the dict each time through my module's "init" function
- setting the delay on the ngx.timer.at (in the init function) to the value in ngx.shared.DICT:get("pause"), and
- only making the connection if the "connected" flag is not set.

This way each worker starts 1 second after the last one, and it works - but it seems clunky, lock-like and error-prone. Importantly I also don't know if e.g. the cache loader process might start first, hence killing the client.

Would love to know if I'm at least on a vaguely right track, or just being a total newb :-)

Thanks so much for your help!

Best,
Igor

On Wednesday, May 27, 2015 at 11:21:06 AM UTC+1, Vladislav Manchev wrote:
You could use something along the lines of this gist [0]. It's a pretty stripped down version of one of my scripts, you should definitely add more error handling.

I've got some code lying around that could help with keeping a persistent connection (i.e. reconnecting on all fatal errors), but that is pretty specific to my use case and you should be able to easily implement it anyway.

One thing to be careful with is getting into hot tight infinite loops which will exhaust CPU resources.

[0]: https://gist.github.com/blablacio/9d81aad39e587eacc881

Best,
Vladislav

On Tue, May 26, 2015 at 9:03 PM, Igor Clark <igor....@gmail.com> wrote:
Hello,

- I have an upstream server which publishes events via websocket and/or EventSource channels as necessary.
- I want my OpenResty app to know about and act on those events straightaway.
- I don't want the upstream server to have to maintain a list of downstream servers, so I want the downstream app to subscribe to the channel.

It'd be pretty straightforward to create a supervised process, running as a separate script on the OpenResty app servers, which listens to the upstream channel and informs the OpenResty app by hitting appropriate local URLs, according to incoming message data. (It's a pretty simple data model in this part of the application, and this would be consistent enough for our purposes.) But it seems like it'd be so much better/neater to do it inside OpenResty, if it can be done, without having to run supervisors, other script runtimes and so forth.

So is it possible to:

- open a long-running resty.websocket.client, lua-resty-http or other client connection when the OpenResty server starts;
- set a Lua event handler function to deal with incoming messages received on the connection,
- notify local URLs via HTTP requests in the handler function, and
- leave it running as long as nginx stays running?
- Ideally with a handler for socket errors so that it could try to reconnect if the connection failed?

If so, what's the best way to do it? And at what point in the system should this happen? I've been reading a lot about the various phases and when which APIs are available, e.g. not being able to call ngx.location.capture from "light threads" started by ngx.timer.at - but I'm struggling to get it at all clear in my head at the moment.

Would really appreciate some tips as to how to go about this, if it's possible!

Thanks very much,
Igor

.