Access a shared C++ resource from openresty

kaspersky_us · 2015-05-27T09:40:49+00:00

Hey, just a quick note to say thanks, I got this working as I wanted, with your help. Turned out pretty simple to get the lock working as I wanted, with a fe...

Access a shared C++ resource from openresty

kaspersky_us

I have a C++ application that uses a rather large and complex data structure (i.e cannot be ported easily). I currently leverage it in a java based REST application using JNI and Sprint Boot, and would like to experiment porting it to Openresty, with Lua as the glue.

I am unsure how to integrate my C++ code in a way that would let me initialize it (which loads the data in memory) and then serve all requests across workers from nginx.

I saw information about developing C modules (but nothing about C++) or using FFI, but it's unclear to me how to use a shared C++ application (with data loaded once in memory)

Thanks in advance for your help

Note: The underlying data is read-only, so there is no concurrency concerns

agentzh

Hello!

On Wed, May 27, 2015 at 9:40 AM, kaspersky_us via openresty-en wrote:
> I am unsure how to integrate my C++ code in a way that would let me
> initialize it (which loads the data in memory) and then serve all requests
> across workers from nginx.
>

Will you update this data structure in your requests? If it's
read-only, then you can just load it in the context of "init_by_lua"
and let the operating system's "Copy-on-Write" (COW) feature save the
physical memory pages automatically for you across all the (forked)
nginx worker processes.

Otherwise you'll have to use nginx's shm facility to host your data
structure to ensure proper locking and data consistency. You can take
a look at ngx_lua's shared dict API for such an example.

> I saw information about developing C modules (but nothing about C++) or
> using FFI, but it's unclear to me how to use a shared C++ application (with
> data loaded once in memory)
>

Generally speaking, the best practice is to wrap your C++ operations
with a clean C interface and let FFI do the rest for you. C++ has
excellent C cooperation support anyway :) But it'll surely be more
complicated if you have to use shm to host your C++ data strucutres in
case that you need to modify them at request time.

Hope it helps :)

Best regards,
-agentzh

kaspersky_us

The data structure is read-only (updated by another process that requires a reload). I will follow your recommendation and maybe post back the outcome if I run into things that might interest others.

Thank you very much for your detailed answer and for all the work so far!

On Wednesday, May 27, 2015 at 7:38:19 AM UTC-7, agentzh wrote:

Hello!

On Wed, May 27, 2015 at 9:40 AM, kaspersky_us via openresty-en wrote:
> I am unsure how to integrate my C++ code in a way that would let me
> initialize it (which loads the data in memory) and then serve all requests
> across workers from nginx.
>

Will you update this data structure in your requests? If it's
read-only, then you can just load it in the context of "init_by_lua"
and let the operating system's "Copy-on-Write" (COW) feature save the
physical memory pages automatically for you across all the (forked)
nginx worker processes.

Otherwise you'll have to use nginx's shm facility to host your data
structure to ensure proper locking and data consistency. You can take
a look at ngx_lua's shared dict API for such an example.

> I saw information about developing C modules (but nothing about C++) or
> using FFI, but it's unclear to me how to use a shared C++ application (with
> data loaded once in memory)
>

Generally speaking, the best practice is to wrap your C++ operations
with a clean C interface and let FFI do the rest for you. C++ has
excellent C cooperation support anyway :) But it'll surely be more
complicated if you have to use shm to host your C++ data strucutres in
case that you need to modify them at request time.

Hope it helps :)

Best regards,
-agentzh

aapo.talvensaari

On Wednesday, 27 May 2015 17:38:19 UTC+3, agentzh  wrote:
> Hello!
> 
> Will you update this data structure in your requests? If it's
> read-only, then you can just load it in the context of "init_by_lua"
> and let the operating system's "Copy-on-Write" (COW) feature save the
> physical memory pages automatically for you across all the (forked)
> nginx worker processes.

So basically in init_by_lua you cache struture on a Lua module level table (or maybe cache cdata).

I have been thinking adding preload function in lua-resty-template to make preloading templates easier on init_by_lua.

I have also thinked about how to notify workers to reload vor flush cache from say web page on a click of the UI element without using shared dict as actual cache. My plan goes something like this:

1. load and cache to Lua module level variable on init_by_Lua.
2. Set loaded time with a timestamp to module level variable
3. now on each request we check if timestamp is less than say 1 minute ago (can be anything else)
4. if it is we check timestamp value from shared duct and if it is more recent than module level timestamp we do something (e.g. reload cache or flush cache) and set new module level timestamp the same as shared duct timestamp (or more recent)
5. now user clicks flush or something that modies the data. The worker that gets that request will update shared dict timestamp and refresh its own cache
6. other workers will do the same when a new request comes that needs that data refreshed

Is this sound design or is there even easier ways to do that?

This at least minifies shared dict usage.

Other way is to have timer on worker that wilk run periodically (maybe that timer can even be one off that subscribes say redia channel to get notifications about the need to refresh the cache).

Regards
Aapo

info

On Thursday, May 28, 2015 at 4:57:02 PM UTC+2, Aapo Talvensaari wrote:

I have also thinked about how to notify workers to reload vor flush cache from say web page on a click of the UI element without using shared dict as actual cache. My plan goes something like this:

Already done and working, see IWCP https://groups.google.com/forum/#!topic/openresty-en/wt_9m7GvROg

agentzh

Hello!

On Thu, May 28, 2015 at 10:57 PM, Aapo Talvensaari wrote:
> I have also thinked about how to notify workers to reload vor flush cache from say web page on a click of the UI element without using shared dict as actual cache. My plan goes something like this:
>
> 1. load and cache to Lua module level variable on init_by_Lua.
> 2. Set loaded time with a timestamp to module level variable
> 3. now on each request we check if timestamp is less than say 1 minute ago (can be anything else)
> 4. if it is we check timestamp value from shared duct and if it is more recent than module level timestamp we do something (e.g. reload cache or flush cache) and set new module level timestamp the same as shared duct timestamp (or more recent)
> 5. now user clicks flush or something that modies the data. The worker that gets that request will update shared dict timestamp and refresh its own cache
> 6. other workers will do the same when a new request comes that needs that data refreshed
>

You could certainly do this though personally I'm not sure it's the
preferred solution.

1. I'd recommend using 2-level caching, that is, lua-resty-lrucache or
something similar + ngx.shared.DICT. This way, you can avoid every
nginx worker recompile things from scratch by allowing only one worker
recompile things and other workers reuse the compiled results from
shm. At CloudFlare we do this 2-level caching extensively.

2. Periodically flushing the cache and recompile everything seem a bit
overkill. You could just stat() on the template files on the file
system and only recompile things when the files actually change (and
avoid recopmiling template files that do not change). This looks more
clever to me :)

Just my 2 cents.

Best regards,
-agentzh

manchev

I'm also doing two level caching with lua-resty-lrucache + ngx.shared.DICT and can vouch for this combination, it works miracles for me.

Best,

Vladislav

On Fri, May 29, 2015 at 10:23 AM, Yichun Zhang (agentzh) <age...@gmail.com> wrote:

Hello!

On Thu, May 28, 2015 at 10:57 PM, Aapo Talvensaari wrote:
> I have also thinked about how to notify workers to reload vor flush cache from say web page on a click of the UI element without using shared dict as actual cache. My plan goes something like this:
>
> 1. load and cache to Lua module level variable on init_by_Lua.
> 2. Set loaded time with a timestamp to module level variable
> 3. now on each request we check if timestamp is less than say 1 minute ago (can be anything else)
> 4. if it is we check timestamp value from shared duct and if it is more recent than module level timestamp we do something (e.g. reload cache or flush cache) and set new module level timestamp the same as shared duct timestamp (or more recent)
> 5. now user clicks flush or something that modies the data. The worker that gets that request will update shared dict timestamp and refresh its own cache
> 6. other workers will do the same when a new request comes that needs that data refreshed
>

You could certainly do this though personally I'm not sure it's the
preferred solution.

1. I'd recommend using 2-level caching, that is, lua-resty-lrucache or
something similar + ngx.shared.DICT. This way, you can avoid every
nginx worker recompile things from scratch by allowing only one worker
recompile things and other workers reuse the compiled results from
shm. At CloudFlare we do this 2-level caching extensively.

2. Periodically flushing the cache and recompile everything seem a bit
overkill. You could just stat() on the template files on the file
system and only recompile things when the files actually change (and
avoid recopmiling template files that do not change). This looks more
clever to me :)

Just my 2 cents.

Best regards,
-agentzh

.

aapo.talvensaari

On Friday, 29 May 2015 10:23:37 UTC+3, agentzh wrote:

You could certainly do this though personally I'm not sure it's the
preferred solution.

1. I'd recommend using 2-level caching, that is, lua-resty-lrucache or
something similar + ngx.shared.DICT. This way, you can avoid every
nginx worker recompile things from scratch by allowing only one worker
recompile things and other workers reuse the compiled results from
shm. At CloudFlare we do this 2-level caching extensively.

I was not referring template on that example. For lua-resty-template I cannot use shared-dict as a cache as I need to cache Lua functions (not strings for example), and that is not supported with ngx.shared.DICT. The example I provided was from a totally different use case. Yes, my example can be modified so that in addition to timestamp we json_encode or something the data to shared dict as well. But I don't think in my use case that adds anything as it is best to cache things on worker level. I was just thinking what could be the best way to notify workers to refresh their caches. But good to know that the design is sound.

Regards

Aapo

agentzh

Hello!

On Fri, May 29, 2015 at 6:41 PM, Aapo Talvensaari wrote:
> I was not referring template on that example. For lua-resty-template I
> cannot use shared-dict as a cache as I need to cache Lua functions (not
> strings for example), and that is not supported with ngx.shared.DICT. The
> example I provided was from a totally different use case.

To me this is still a perfect use case of 2-level caching:

1. you can cache loaded Lua functions in the worker-level cache (like
in package.loaded or lua-resty-lrucache);
2. you can cache the raw Lua source code for for those Lua functions in shm.

We do this kind of "Lua code caching" in CloudFlare's WAF system.

> Yes, my example
> can be modified so that in addition to timestamp we json_encode or something
> the data to shared dict as well. But I don't think in my use case that adds
> anything as it is best to cache things on worker level. I was just thinking
> what could be the best way to notify workers to refresh their caches. But
> good to know that the design is sound.
>

The worker-level cache is best for caching complicated values like
native Lua functions and Lua tables which require (expensive)
serialization and deserialization operations for the shdict cache.

Regards,
-agentzh