#0 0x000000000050f888 in ngx_http_lua_socket_tcp_handler (ev=0xe8da18) at ../ngx_lua-0.9.0/src/ngx_http_lua_socket_tcp.c:2028
#1 0x000000000044b02b in ngx_epoll_process_events (cycle=0xdf6f40, timer=1000, flags=1) at src/event/modules/ngx_epoll_module.c:714
#2 0x000000000043b3d4 in ngx_process_events_and_timers (cycle=0xdf6f40) at src/event/ngx_event.c:249
#3 0x0000000000448e1f in ngx_worker_process_cycle (cycle=0xdf6f40, data="" at src/os/unix/ngx_process_cycle.c:813
#4 0x0000000000445600 in ngx_spawn_process (cycle=0xdf6f40, proc=0x448c59 <ngx_worker_process_cycle>, data="" name=0x541fcb "worker process", respawn=-3)
#5 0x0000000000447bcc in ngx_start_worker_processes (cycle=0xdf6f40, n=1, type=-3) at src/os/unix/ngx_process_cycle.c:364
#6 0x0000000000447207 in ngx_master_process_cycle (cycle=0xdf6f40) at src/os/unix/ngx_process_cycle.c:136
#7 0x0000000000416732 in main (argc=5, argv=0x7fff05faa718) at src/core/nginx.c:412
(gdb) backtrace full
#0 0x000000000050f888 in ngx_http_lua_socket_tcp_handler (ev=0xe8da18) at ../ngx_lua-0.9.0/src/ngx_http_lua_socket_tcp.c:2028
c = 0xe43a20
r = 0xe01ca0
ctx = 0x2917882ad920910b
u = 0x41733de0
#1 0x000000000044b02b in ngx_epoll_process_events (cycle=0xdf6f40, timer=1000, flags=1) at src/event/modules/ngx_epoll_module.c:714
events = 1
revents = 5
instance = 0
i = 0
level = 7920512
err = 0
rev = 0xe73a08
wev = 0xe8da18
queue = 0xe73960
c = 0xe43c60
#2 0x000000000043b3d4 in ngx_process_events_and_timers (cycle=0xdf6f40) at src/event/ngx_event.c:249
flags = 1
timer = 1000
delta = 1382134470469
#3 0x0000000000448e1f in ngx_worker_process_cycle (cycle=0xdf6f40, data="" at src/os/unix/ngx_process_cycle.c:813
worker = 0
i = 4284512
c = 0x0
#4 0x0000000000445600 in ngx_spawn_process (cycle=0xdf6f40, proc=0x448c59 <ngx_worker_process_cycle>, data="" name=0x541fcb "worker process", respawn=-3)
at src/os/unix/ngx_process.c:198
on = 1
pid = 0
s = 0
#5 0x0000000000447bcc in ngx_start_worker_processes (cycle=0xdf6f40, n=1, type=-3) at src/os/unix/ngx_process_cycle.c:364
i = 0
ch = {command = 1, pid = 0, slot = 0, fd = 0}
#6 0x0000000000447207 in ngx_master_process_cycle (cycle=0xdf6f40) at src/os/unix/ngx_process_cycle.c:136
title = 0xe3e79c "master process /usr/local/openresty/nginx/sbin/nginx -p /home/cc/clickcertain-nginx/ -c nginx.conf.direct"
p = 0xe3e805 ""
size = 106
i = 5
n = 140416990712373
sigio = 4
set = {__val = {0 <repeats 16 times>}}
itv = {it_interval = {tv_sec = 140416964924664, tv_usec = 0}, it_value = {tv_sec = 0, tv_usec = 0}}
live = 140416988470640
delay = 6
ls = 0x0
ccf = 0xdf7f10
#7 0x0000000000416732 in main (argc=5, argv=0x7fff05faa718) at src/core/nginx.c:412
i = 66
log = 0x78b980
cycle = 0xdf6f40
init_cycle = {conf_ctx = 0x0, pool = 0xdf6330, log = 0x78b980, new_log = {log_level = 0, file = 0x0, connection = 0, handler = 0, data = "" action = "" files = 0x0,
free_connections = 0x0, free_connection_n = 0, reusable_connections_queue = {prev = 0x0, next = 0x0}, listening = {elts = 0x0, nelts = 0, size = 0, nalloc = 0, pool = 0x0},
paths = {elts = 0x0, nelts = 0, size = 0, nalloc = 0, pool = 0x0}, open_files = {last = 0x0, part = {elts = 0x0, nelts = 0, next = 0x0}, size = 0, nalloc = 0, pool = 0x0},
shared_memory = {last = 0x0, part = {elts = 0x0, nelts = 0, next = 0x0}, size = 0, nalloc = 0, pool = 0x0}, connection_n = 0, files_n = 0, connections = 0x0,
read_events = 0x0, write_events = 0x0, old_cycle = 0x0, conf_file = {len = 45, data = "" "\002"}, conf_param = {len = 0, data = "" conf_prefix = {len = 28,
data = "" "\002"}, prefix = {len = 28, data = "" ""}, lock_file = {len = 0, data = "" hostname = {len = 0, data = "">
ccf = 0xdf7f10
Hope this helps... :)
Erik
On Friday, October 18, 2013 2:29:13 PM UTC-7, Erik Wickstrom wrote:
Hi agentzh,
Thanks for your quick response!
On Friday, October 18, 2013 12:06:04 PM UTC-7, agentzh wrote:
Hello!
On Thu, Oct 17, 2013 at 5:58 PM, <erikwi...@gmail.com> wrote:
>
> I've narrowed the error down to a call to ngx.location.capture which
> initiates a proxy_pass to some backend application servers. I then modify
> the response with Lua before returning it to the client.
>
[...]
>
> If I remove the ngx.location.capture call, the error goes away.
>
It looks like a bug in ngx_lua's subrequest scheduler. Thank you for the report!
Here're some suggestions on tracing this further:
1. If possible, please prepare a self-contained minimal example that
can reproduce the issue you're seeing relatively reliably on my side?
Preferably without MoonScript. If I can reproduce the issue on my
side, it'll be trivial for me to fix it :)
Here is a self contained version of the Lua and nginx.conf. Note the cjson import at the top of the Lua file -- I'm not using cjson in any of that code, but if I remove it I get the error in the comments at the top of the file. Might be another bug...
2. Please make your nginx produce a core dump file and then use gdb to
get a full backtrace (your backtrace lacks a lot of info). See here
for more details: http://wiki.nginx.org/Debugging#Core_dump
I recompiled openresty --with-debug and added the config parameters to nginx.conf -- but I can't get nginx to drop the core dump into my working directory. Not sure why that isn't working. Will my test code be enough to create the problem on your machine?
3. If possible, please use valgrind memcheck to run your nginx to see
if it can catch any memory issues. You need to turn off
"master_process" and "daemon" in your nginx.conf when using valgrind
memcheck.
This is getting a bit out of my area of expertise...
Just as a side note: in the long run, it'll be recommended to use Lua
libraries based completely on ngx_lua's cosocket API for such kind of
things (like lua-resty-http, for example). Fiddling with nginx's
subrequests is no fun at all :)
Would you advise using lua-resty-http now? Or is this "in the long run" only? :) I googled and there appear to be 3 different projects with that name. Any thoughts on which are stable currently?
Thanks!
-agentzh
Thanks!
Erik (erikcw)