Hi there! Author of lua-resty-exec here, long-running processes are tricky, but do-able.
lua-resty-exec
keeps a connection open while a program is running, and you can't pass
connections to different contexts (you'll get a "attempt yield across c
boundary" error).
So, in my init_worker_by_lua_block, I fire up a
"process manager" in an nginx timer - this basically spawns a
background thread, like
init_worker_by_lua_block {
processmgr = require('process_manager').new()
ngx.timer.at(0,function()
processmgr:run()
end)
}
That
"process manager"'s run function connects to some notification system -
in my case, I use redis. Any other pub/sub type of system could work,
like
https://github.com/slact/ngx_lua_ipc, or maybe the process manager
just continuously polls for new jobs from some queue system, whatever.
Then
when I want to fire up some long-running, background program, I fire
off a notification with details on what program to run.
The background "process manager" picks up the notification, and it calls the "prog" function in an nginx thread, like
local handle = ngx.thread.spawn(function()
local res = prog('long_running_program')
call_some_useful_function_when_done
end)
and
then stores that handle in a table, and said table is keyed to
something based on the program arguments or some other property (ie,
maybe you're keeping track of users so you key off of that).
There's
a few ways to check the status - I *think* you can call
"coroutine.status" on the thread handle, or that might not work, I'm not
really sure. Everything I do is event-based, so when my program exits, I
broadcast a notification that it's done and things react appropriately.
Some other ideas:
Have your ngx.thread.spawn function set/unset something in shared memory, like:
local handle = ngx.thread.spawn(function()
ngx.shared.progs.set('some-key',true)
local res = prog('long_running_program')
ngx.shared.progs.set('some-key',nil)
end)
Then you can call "ngx.shared.progs.get('some-
key')" to check if the process is still running or not.
Hope that helps!
On Thursday, January 12, 2017 at 1:10:28 PM UTC-6, Nelson, Erik - 2 wrote:
I'm trying to asynchronously execute programs in openresty, and lua-resty-exec and lua-resty shell have nice nonblocking interfaces along the lines of
local exec
=
require'resty.exec'
local prog
= exec.new('/tmp/exec.sock')
local res, err
=
prog('uname')
I know that prog() in this case doesn’t block the web werver, but it does block progression the request.
I’m hoping to do something along the lines of this pseudocode.
location /run_program {
content_by_lua_block {
local prog = require'resty.exec'.new('/tmp/exec.sock')
local handle = async_prog(‘long_running_program') -- I don’t want to wait until the subprocess exits here
ngx.say(‘started a long running program at handle ’ .. handle)
}
}
location /check_program_status {
content_by_lua_block {
local handle = ngx.req.get_uri_args().handle
local res, err = async_prog_wait(handle) -- also don’t block here, just status like waitpid() with WNOHANG
ngx.say(‘long running program status is ’ .. res)
}
}
Anyone have any ideas? Or a suggestion for a better way to do it?
Thanks
Erik
This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at
http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.
