Hello
I want to add a new data structure (bloom filter - shared_bloom) in addition to shared_dict to the lua_nginx_module. The new data structure works but causes the shared_dict to not work anymore. More specifically, It seems that lua_shared_dict is interpreted as lua_shared_bloom when both are being used. I would greatly appreciate some help. Please find below some details. Thank you in advance for your time.
1. Added the following files:
~/ngx_openresty-1.7.7.1/bundle/ngx_lua-0.9.13/src/ngx_http_lua_shbloom.c
~/ngx_openresty-1.7.7.1/bundle/ngx_lua-0.9.13/src/ngx_http_lua_shbloom.h
2. Added the following declaration:
ngx_int_t ngx_http_lua_shared_bloom_get(ngx_shm_zone_t *shm_zone,
u_char *key_data, size_t key_len, ngx_http_lua_value_t *value);
to the following file:
~/ngx_openresty-1.7.7.1/bundle/ngx_lua-0.9.13/src/api/ngx_http_lua_api.h
3. Added the files to the build system
$ngx_addon_dir/src/ngx_http_lua_shdict.h \
$ngx_addon_dir/src/ngx_http_lua_shbloom.h \and
$ngx_addon_dir/src/ngx_http_lua_shdict.c \
$ngx_addon_dir/src/ngx_http_lua_shbloom.c \
to
~/ngx_openresty-1.7.7.1/bundle/ngx_lua-0.9.13/config
4. Added
#include "ngx_http_lua_shdict.h"
#include "ngx_http_lua_shbloom.h"
static void
ngx_http_lua_inject_ngx_api(lua_State *L, ngx_http_lua_main_conf_t *lmcf, ngx_log_t *log)
[...]
ngx_http_lua_inject_shdict_api(lmcf, L);
ngx_http_lua_inject_shbloom_api(lmcf, L);
to
~/ngx_openresty-1.7.7.1/bundle/ngx_lua-0.9.13/src/ngx_http_lua_uti.c
5. Added
static ngx_command_t ngx_http_lua_cmds[] = {
[...]
{ ngx_string("lua_shared_dict"),
NGX_HTTP_MAIN_CONF|NGX_CONF_TAKE2,
ngx_http_lua_shared_dict,
0,
0,
NULL },
{ ngx_string("lua_shared_bloom"),
NGX_HTTP_MAIN_CONF|NGX_CONF_TAKE2,
ngx_http_lua_shared_bloom,
0,
0,
NULL },
to
~/ngx_openresty-1.7.7.1/bundle/ngx_lua-0.9.13/src/ngx_http_lua_module.c
6. Added declaration and implementation of
char *
ngx_http_lua_shared_bloom(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
to
~/ngx_openresty-1.7.7.1/bundle/ngx_lua-0.9.13/src/ngx_http_lua_directive.h
~/ngx_openresty-1.7.7.1/bundle/ngx_lua-0.9.13/src/ngx_http_lua_directive.c
7. In the nginx.conf, I have the following
[...]
server {
listen XXXX;
location /printcounters {
content_by_lua '
local counters = ngx.shared.counters
local totalreq = zcounters:get("totalreq")
local acceptedreq = zcounters:get("acceptedreq")
ngx.say("counters totalreq=", totalreq, " acceptedreq=", acceptedreq)
';
}
location = /probe1 {
content_by_lua '
local blooms = ngx.shared.blooms
local key = "1b4e28ba-2fa1-11d2-883f-0016d3cca427"
local key1 = "1b4e27ba-3fa1-12d2-883f-0016d3cca427"
local len = #key
ngx.log(ngx.NOTICE, "key=", key, " len=", len)
local ok, reason = blooms:bloom_put(key)
ngx.log(ngx.NOTICE, "bloom_put=", ok, " err=", reason)
local ok, reason = blooms:bloom_put(key)
ngx.log(ngx.NOTICE, "bloom_put=", ok, " err=", reason)
local ok, reason = blooms:bloom_check(key)
ngx.log(ngx.NOTICE, "bloom_check=", ok)
local ok, err, reason = blooms:bloom_check(key1, val)
ngx.log(ngx.NOTICE, "bloom_check=", ok)
ngx.say("bloom filter test done")
';
}
lua_shared_dict counters 10m;
lua_shared_bloom blooms 3000m;
[...]
If I call http://localhost:XXXX/printcounters then with gdb's help, I see that the code crashes in the ngx_http_lua_shbloom.c but really I would expect the code in ngx_http_lua_shdict.c to be called. Here is a stack trace.
(gdb) bt
#0 ngx_http_lua_shbloom_get_helper (L=0x415d1250, get_stale=<optimized out>) at ../ngx_lua-0.9.13/src/ngx_http_lua_shbloom.c:559
#1 0x00007ffae9530e15 in lj_BC_FUNCC () from /usr/local/openresty/luajit/lib/libluajit-5.1.so.2 #2 0x00000000004bbdfa in ngx_http_lua_run_thread (L=L@entry=0x415c2378, r=r@entry=0xea1f70, ctx=ctx@entry=0xe9b1f0, nrets=nrets@entry=0) at ../ngx_lua-0.9.13/src/ngx_http_lua_util.c:999
#3 0x00000000004c061e in ngx_http_lua_content_by_chunk (L=0x415c2378, r=0xea1f70) at ../ngx_lua-0.9.13/src/ngx_http_lua_contentby.c:103
#4 0x00000000004bfe7f in ngx_http_lua_content_handler (r=0xea1f70) at ../ngx_lua-0.9.13/src/ngx_http_lua_contentby.c:205
#5 0x000000000044cb4f in ngx_http_core_content_phase (r=0xea1f70, ph=<optimized out>) at src/http/ngx_http_core_module.c:1418
#6 0x0000000000447375 in ngx_http_core_run_phases (r=r@entry=0xea1f70) at src/http/ngx_http_core_module.c:899
#7 0x000000000044745d in ngx_http_handler (r=r@entry=0xea1f70) at src/http/ngx_http_core_module.c:882
#8 0x0000000000452f03 in ngx_http_process_request (r=0xea1f70) at src/http/ngx_http_request.c:1903
#9 0x0000000000453508 in ngx_http_process_request_headers (rev=rev@entry=0x7ffae9d20148) at src/http/ngx_http_request.c:1334
#10 0x00000000004538c6 in ngx_http_process_request_line (rev=0x7ffae9d20148) at src/http/ngx_http_request.c:1013
#11 0x000000000043cbf9 in ngx_epoll_process_events (cycle=0xe97210, timer=<optimized out>, flags=<optimized out>) at src/event/modules/ngx_epoll_module.c:685
#12 0x0000000000433737 in ngx_process_events_and_timers (cycle=cycle@entry=0xe97210) at src/event/ngx_event.c:248
#13 0x000000000043a99a in ngx_worker_process_cycle (cycle=cycle@entry=0xe97210, data="" at src/os/unix/ngx_process_cycle.c:822
#14 0x0000000000439390 in ngx_spawn_process (cycle=cycle@entry=0xe97210, proc=proc@entry=0x43a8e0 <ngx_worker_process_cycle>, data="" name=name@entry=0x4f346c "worker process", respawn=respawn@entry=-3)
at src/os/unix/ngx_process.c:198
#15 0x000000000043abe4 in ngx_start_worker_processes (cycle=cycle@entry=0xe97210, n=1, type=type@entry=-3) at src/os/unix/ngx_process_cycle.c:368
#16 0x000000000043b91f in ngx_master_process_cycle (cycle=cycle@entry=0xe97210) at src/os/unix/ngx_process_cycle.c:140
#17 0x000000000041968f in main (argc=<optimized out>, argv=<optimized out>) at src/core/nginx.c:407
However, If I do not inject ngx_http_lua_inject_shbloom_api in ~/ngx_openresty-1.7.7.1/bundle/ngx_lua-0.9.13/src/ngx_http_lua_uti.c then shared_dict works and obviously shared_bloom does not.
[...]
2015/02/08 16:42:04 [error] 9627#0: *2 lua entry thread aborted: runtime error: content_by_lua(nginx.conf:155):10: attempt to call method 'bloom_put' (a nil value)
[...]
Again, thanks a lot for any help
Cheers
