synchronous/blocking network call

rohit.c.joshi · 2017-02-01T19:47:44+00:00

res[1] is itOn Thursday, January 26, 2017 at 2:14:47 PM UTC+1, Michael Gohl wrote:Hey guys I am using the mysql-lua extension:res, err, errcode, sqlstat...

synchronous/blocking network call

rohit.c.joshi

What is the best way to use 3rd party library which is making blocking network call? Example I need to use Pkcs11 library which makes blocking call to HSM for hardware based encryption.

aapo.talvensaari

On Wednesday, 1 February 2017 13:47:44 UTC+2, RJoshi wrote:

What is the best way to use 3rd party library which is making blocking network call? Example I need to use Pkcs11 library which makes blocking call to HSM for hardware based encryption.

If you don't want to block your web server, then you don't make blocking calls or long running Lua code that doesn't yield (it is co-operative, and not pre-emptive, scheduling is done automatically, and NIO yields automatically when using OpenResty's network libs, or manually by using coroutine.yield).

You could (just some ideas):

1. write a OpenResty driver where you use OpenResty's network libraries (aka write a non-blocking driver)

2. have another network endpoint where you connect with OpenResty's network libraries. In this another network endpoint, you can use that 3rd party library in a blocking fashion, but be aware that it could just create another bottleneck or move the bottleneck elsewhere (compared to just making blocking calls from OpenResty). It depends on your usage scenario if this is worth it (if the server is only serving requests that access HSM, then it surely is not worth it, in that case it is probably better just to launch as many worker processes as the HSM can handle before starting to queuing up too much).

3. If you can wait for the answer, aka you could off load the work to message queue or something and process those as fast as HSM allows, but notifies subscribers asynchronously, that is possibly another approach.

Scaling HSM could be done by adding a farm of HSM servers(?).

If accessing HSM is rare compared to other accesses, then well, it may not matter if you block one or two workers for small amounts of time to connect the HSM. But I don't know the specifics. You could traffic limit the HSM accessing endpoints so that they don't block too much, aka make an easy to target denial of service endpoint.

erik.l.nelson

Hi Aapo,

What’s the difference between ideas 1 and 2? It seems (on the OpenResty side) you’re using the OpenResty network libraries either way. Am I missing something?

Thanks

Erik

From: openresty-en@googlegroups.com [mailto:openresty-en@googlegroups.com] On Behalf Of Aapo Talvensaari
Sent: Wednesday, February 01, 2017 7:36 AM
To: openresty-en
Subject: [openresty-en] Re: synchronous/blocking network call

On Wednesday, 1 February 2017 13:47:44 UTC+2, RJoshi wrote:

What is the best way to use 3rd party library which is making blocking network call? Example I need to use Pkcs11 library which makes blocking call to HSM for hardware based encryption.

You could (just some ideas):

1. write a OpenResty driver where you use OpenResty's network libraries (aka write a non-blocking driver)

Scaling HSM could be done by adding a farm of HSM servers(?).

This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.

aapo.talvensaari

On Wednesday, 1 February 2017 17:05:58 UTC+2, Nelson, Erik - 2 wrote:

What’s the difference between ideas 1 and 2? It seems (on the OpenResty side) you’re using the OpenResty network libraries either way. Am I missing something?

The main difference that I was thinking about is that you can have:

1) a front-end web server where you don't do blocking calls

2) off-load blocking calls to another server (possibly) also OpenResty where you do the blocking NIO.

This way you can ensure that the frontend server(s) is/are never actually using blocked io. Here you of course need to build your own endpoints / API to communicate with these servers (or just proxy to it). And if something starts to act as a bottleneck, you can move it to a different machine or add more machines.

The difference is that on this another server you are using that 3rd party library as is, aka doing blocked calls and that server may not handle as much concurrent traffic easily. But at least it doesn't halt the front-end server. Of course this is not as good as solution 1), because it doesn't really solve the issue of blocking calls. It is a compromise, and I agree, not particularly good one. If HSM module is only used for something like logging in, it is still possible to (D)DOS login, but all the rest still works quite well (as the front-end is serving them and nothing blocks the front-end). If you put this HSM access that blocks in front end, you can (D)DOS the whole front-end (but as said, you can possibly also traffic limit HSM accessing endpoints).

It is hard to know how much of work is it to code that blocking client with OpenResty's non-blocking NIO. If it is not straightforward and easy, then it is about balancing that is it worth to do, how much does it take a time to build, how much it costs, and other possible business or other reasons.

This may or may not be related to this:

https://github.com/openresty/lua-nginx-module/pull/450

I wish that many C libraries would take this in account, and decouple io access as much as possible from the other stuff the library may be doing (encodings, serialization, cryptography, etc.).

Regards

Aapo

erik.l.nelson

Got it, thanks!

From: openresty-en@googlegroups.com [mailto:openresty-en@googlegroups.com] On Behalf Of Aapo Talvensaari
Sent: Wednesday, February 01, 2017 1:24 PM
To: openresty-en
Cc: Nelson, Erik - 2
Subject: Re: [openresty-en] Re: synchronous/blocking network call

On Wednesday, 1 February 2017 17:05:58 UTC+2, Nelson, Erik - 2 wrote:

What’s the difference between ideas 1 and 2? It seems (on the OpenResty side) you’re using the OpenResty network libraries either way. Am I missing something?

The main difference that I was thinking about is that you can have:

1) a front-end web server where you don't do blocking calls

2) off-load blocking calls to another server (possibly) also OpenResty where you do the blocking NIO.

This may or may not be related to this:

https://github.com/openresty/lua-nginx-module/pull/450

I wish that many C libraries would take this in account, and decouple io access as much as possible from the other stuff the library may be doing (encodings, serialization, cryptography, etc.).

Regards

Aapo