Async operation in openresty

bogdan · 2015-02-26T20:08:18+00:00

Hello!On Wednesday, February 25, 2015 at 11:26:54 PM UTC+1, agentzh wrote:Personally I like Brian Akins's lua-resty-http-simple library the most: &nb...

Async operation in openresty

bogdan

Hello again

I know I might ask a question which has been asked many times before, but I can't find a definitive answer now.

I am in the situation where I need to implement a task that might take some time, and I don't want to block the user waiting for an answer.
I tried to implement this with light threads (ngx.thread.spawn), but I noticed that when I try to finish the main thread with ngx.exit(ngx.OK) the light thread also gets killed. If I don't use ngx.exit, the main thread keeps running until the light thread finishes, and this result in a timeout on the client's side.

Is it possible to launch an asynchronous task and then to immediately return a response to the HTTP request that I received?

Thank you

Bogdan

james_9172

Hi,

Take a look at ngx.eof() [1]. The client timeout you're seeing might depend on the protocol version of the request or the client.

I ended up with quite an over-engineered solution for background work, because I have tasks which *must* be completed, and cannot be dropped etc. For this, I wrote a binding [2] to qless-core [3], which is a Redis based job/queue system implemented as Lua scripts within Redis. I run workers [4] for this in init_worker_by_lua, which pick off the jobs and process them reliably. So this is quite complicated, but reliable.

[1] https://github.com/openresty/lua-nginx-module#ngxeof

[2] https://github.com/pintsized/lua-resty-qless

[3] https://github.com/seomoz/qless-core

[4] https://github.com/pintsized/lua-resty-qless#running-workers

James.

On 26 February 2015 at 12:08, Bogdan Irimia <bo...@digitair.ro> wrote:

Hello again

I know I might ask a question which has been asked many times before, but I can't find a definitive answer now.

I am in the situation where I need to implement a task that might take some time, and I don't want to block the user waiting for an answer.
I tried to implement this with light threads (ngx.thread.spawn), but I noticed that when I try to finish the main thread with ngx.exit(ngx.OK) the light thread also gets killed. If I don't use ngx.exit, the main thread keeps running until the light thread finishes, and this result in a timeout on the client's side.

Is it possible to launch an asynchronous task and then to immediately return a response to the HTTP request that I received?

Thank you

Bogdan

.

James Hurst

ja...@pintsized.co.uk

bogdan

Wow, excellent! Just the answer I've been looking for - very professional.

I think the ngx.eof way is just right for us right now, but the qless alternative is an excellent thing to keep in mind.

Thank you

Bogdan

James Hurst

Thursday, February 26, 2015 14:24

Hi,

Take a look at ngx.eof() [1]. The client timeout you're seeing might depend on the protocol version of the request or the client.

I ended up with quite an over-engineered solution for background work, because I have tasks which *must* be completed, and cannot be dropped etc. For this, I wrote a binding [2] to qless-core [3], which is a Redis based job/queue system implemented as Lua scripts within Redis. I run workers [4] for this in init_worker_by_lua, which pick off the jobs and process them reliably. So this is quite complicated, but reliable.

[1] https://github.com/openresty/lua-nginx-module#ngxeof
[2] https://github.com/pintsized/lua-resty-qless
[3] https://github.com/seomoz/qless-core
[4] https://github.com/pintsized/lua-resty-qless#running-workers

James.

--
James Hurst
ja...@pintsized.co.uk

Is it possible to launch an asynchronous task and then to immediately return a response to the HTTP request that I received?

Thank you

Bogdan