Attempt to share shdict between stream and http subsystem

fffonion · 2018-01-25T12:38:06+00:00

local env = {ngx = ngx}local hacked_set_header_function = function() endenv.ngx.req.set_header = hacked_set_header_function-- f is functionsetfenv(f, env)pca...

Attempt to share shdict between stream and http subsystem

fffonion

Hi,

First thanks for the excellent work for bringing meta-lua-nginx-module to life!

Since stream-lua-nginx-module is now generated by meta-lua-nginx-module and the shdict part is identical to that in lua-nginx-module. Are we safe to let them share shdicts? My attempt is to pass a same parameter to either ngx_[subsys]_lua_shared_memory_add:

diff --git a/ngx_lua-0.10.11/src/ngx_http_lua_directive.c b/ngx_lua-0.10.11/src/ngx_http_lua_directive.c
index 014a472..60493b2 100644
--- a/ngx_lua-0.10.11/src/ngx_http_lua_directive.c
+++ b/ngx_lua-0.10.11/src/ngx_http_lua_directive.c
@@ -125,7 +125,7 @@ ngx_http_lua_shared_dict(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
     ctx->log = &cf->cycle->new_log;

     zone = ngx_http_lua_shared_memory_add(cf, &name, (size_t) size,
-                                          &ngx_http_lua_module);
+                                          &ngx_shared_memory_add);
     if (zone == NULL) {
         return NGX_CONF_ERROR;
     }
@@ -133,10 +133,16 @@ ngx_http_lua_shared_dict(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
     if (zone->data) {
         ctx = zone->data;

-        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
-                           "lua_shared_dict \"%V\" is already defined as "
-                           "\"%V\"", &name, &ctx->name);
-        return NGX_CONF_ERROR;
+        if((&name)->len == (&ctx->name)->len &&
+                ! ngx_strcmp((&name)->data, (&ctx->name)->data) &&
+                zone->tag == &ngx_shared_memory_add) {
+            dd("same zone in other subsystem");
+        } else {
+            ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
+                               "http lua_shared_dict \"%V\" is already defined as "
+                               "\"%V\"", &name, &ctx->name);
+            return NGX_CONF_ERROR;
+        }
     }

     zone->init = ngx_http_lua_shdict_init_zone;
diff --git a/ngx_stream_lua-0.0.3/src/ngx_stream_lua_directive.c b/ngx_stream_lua-0.0.3/src/ngx_stream_lua_directive.c
index dbf491c..e59f019 100644
--- a/ngx_stream_lua-0.0.3/src/ngx_stream_lua_directive.c
+++ b/ngx_stream_lua-0.0.3/src/ngx_stream_lua_directive.c
@@ -115,7 +115,7 @@ ngx_stream_lua_shared_dict(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
     ctx->log = &cf->cycle->new_log;

     zone = ngx_stream_lua_shared_memory_add(cf, &name, (size_t) size,
-                                          &ngx_stream_lua_module);
+                                          &ngx_shared_memory_add);
     if (zone == NULL) {
         return NGX_CONF_ERROR;
     }
@@ -123,10 +123,16 @@ ngx_stream_lua_shared_dict(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
     if (zone->data) {
         ctx = zone->data;

-        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
-                           "lua_shared_dict \"%V\" is already defined as "
-                           "\"%V\"", &name, &ctx->name);
-        return NGX_CONF_ERROR;
+        if((&name)->len == (&ctx->name)->len &&
+                ! ngx_strcmp((&name)->data, (&ctx->name)->data) &&
+                zone->tag == &ngx_shared_memory_add) {
+            dd("same zone in other subsystem");
+        } else {
+            ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
+                               "stream lua_shared_dict \"%V\" is already defined as "
+                               "\"%V\"", &name, &ctx->name);
+            return NGX_CONF_ERROR;
+        }
     }

     zone->init = ngx_stream_lua_shdict_init_zone;

It's better to use something other than the address of as a tag but currently I can't find some value that is shared only between stream-lua and http-lua module.

Then we can simply add two lua_shared_dict directive declaring same name and same size.

http{
    lua_shared_dict dog 10m;
}
stream
    lua_shared_dict dog 10m;
}

Also I've checked the ngx_shmtx_lock and the lock address is same for a same shdict so atomic is also ensured between two subsystems.

Using meta-lua-nginx-module to merged shdict into one module is clearly a better choice as is stated in https://github.com/openresty/meta-lua-nginx-module/issues/24

My initial purpose is to build an easy interface that can be used to test shdict. Currently with this patch and this library we would be able to debug shdict using redis client. : )