worker process request routing

dlinenbe · 2015-10-13T02:14:55+00:00

Hi,Thanks for the insight. I fixed up the code and now it is working. Like you said, I increase the size of variable buf by 16 and then set the...

worker process request routing

dlinenbe

Hello

Anther newbie question:

- If multiple calls are made to NGINX, I would like N calls to an nginx master/ N worker set-up guarantee that each of the N calls get routed to a different worker.

- Can I have a single client call get routed to *all* worker processes?

Is any of this possible? If so, any pointers on doing so would be appreciated.

Thanks!

David Linenberg

lordnynex

Not only is this not possible this doesn't make any sense.

- Master processes do not do any actual work.

- Worker processes are separate processes that all listen on a shared socket, thusly can not operate on a request in parallel.

Can you explain why you think this is a valid use case for any software?

Please note, what you're asking is not specific to openresty. You should spend a bit of time studying the nginx documentation.

On Mon, Oct 12, 2015 at 11:14 AM, David Linenberg <dli...@gmail.com> wrote:

Hello
Anther newbie question:

- If multiple calls are made to NGINX, I would like N calls to an nginx master/ N worker set-up guarantee that each of the N calls get routed to a different worker.

- Can I have a single client call get routed to *all* worker processes?

Is any of this possible? If so, any pointers on doing so would be appreciated.

Thanks!

David Linenberg

.

info

Of course this is possible, see https://groups.google.com/forum/#!topic/openresty-en/wt_9m7GvROg

This is a basic framework which can be used for alot of other things, data processing is one thing, for example with some slight changes we use it to process SAP data by a worker who is the least busy without the need to run a separate process. Once its done it offloads the results in memcache for other workers to work with. Netweaver is used to send a trigger message to nginx and IWCP(Lua) does the rest.
You can't use it to run parallel incoming requests since nginx architecture is not suited for that but for worker management and a wide range of task distribution its perfect.

On Monday, October 12, 2015 at 8:14:55 PM UTC+2, David Linenberg wrote:

Hello
Anther newbie question:

- If multiple calls are made to NGINX, I would like N calls to an nginx master/ N worker set-up guarantee that each of the N calls get routed to a different worker.

- Can I have a single client call get routed to *all* worker processes?

Is any of this possible? If so, any pointers on doing so would be appreciated.

Thanks!

David Linenberg

erik.l.nelson


>Lord Nynex wrote on Monday, October 12, 2015 2:32 PM
>Subject: Re: [openresty-en] worker process request routing

>Not only is this not possible this doesn't make any sense. 

I feel like it could make sense

>- Master processes do not do any actual work. 
>- Worker processes are separate processes that all listen on a shared socket, thusly can not operate on a request in parallel. 

>Can you explain why you think this is a valid use case for any software? 

Suppose each of N workers had their own state, and a single call to /update_state?new_value=x could be made to update the state in each of the workers simultaneously- the single call to /update_state could be expanded into N calls to /update_state, one that was routed to each worker.  This would solve some of the same problems that lua-resty-lock and the ngx.dict solve.

>Please note, what you're asking is not specific to openresty. You should spend a bit of time studying the nginx documentation. 

True, not specific to openresty, and not currently implemented in nginx.  But this isn't the first time that the concept of mapping a certain location to a particular worker has been raised, and it seems (to me, at least) like it could be quite useful in some circumstances.

>>Another newbie question:

>>- If multiple calls are made to NGINX, I would like N calls to an nginx master/ N worker set-up guarantee that each of the N calls get routed to >>a different worker.

>>- Can I have a single client call get routed to *all* worker processes?

>>Is any of this possible?  If so, any pointers on doing so would be appreciated.


----------------------------------------------------------------------
This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer.   If you are not the intended recipient, please delete this message.

dlinenbe

Erik -

This is exactly the case I need to address. We are considering replacing our MPI worker nodes with nginx nodes. In MPI (Message Passing Interface), we can specify exactly what information gets sent to which node (or rank).

We send commands (requests) to the workers which change their internal state. For example, for particularly complex financial instruments, each worker node must receive interest rate curve and volatility data before

any other calculations can be done. We can't just use shared memory. We must guarantee that all nodes receive the new state data and run a function which updates the state accordingly.

We need to know all nodes have the new state, after which we run any number of calculations which can be run on any node. We then may request a change in state again..etc.

This is not just for state...

We do stochastic Monte Carlo method calculations - where we do simulations. This is *very* common in the financial industry, running such calculations on compute grids. We send a request to all nodes in our grid (think 1000 cores or worker nodes). Each worker node comes back with an answer which is a bit different from another worker node - because there is an element of randomness (e.g. random number generator) embedded in each worker node's calculation. We expect to get N responses from the N nodes - where we do some calculations on the result set. Consider even simple calculations on the basket of results, such as mean or standard deviation...

A hacked work around - would be to start N nginx master/worker procesess on a machine, each on a different port, where N is the number of cores. (e.g. 32 workers & 32 masters on a 32 core machine ) - But I'd rather just start one nginx master with N workers, and have the master distribute the request to the workers, as nginx was intended to be used.

I understand that I probably sent this to the wrong group - and that the nginx newsgroup would be more appropriate - so sorry about that.. However, given that I am using nginx-lua for this, I thought that I might consider using Lua code - to replace the custom C++ routing code we now employ. Also, I can see a *huge* reason to have web servers be able to supporting more compute models than the current stateless request/reply model. Considering that on AWS, we can spawn up arbitrarily any number of EC2 instances, and that REST APIs using JSON over HTTP are quite commonly hosted on webservers, it actually surprises me that web servers would *not* support this capability - though it might seem unusual...

Dave Linenberg

On Monday, October 12, 2015 at 7:35:34 PM UTC-4, Nelson, Erik - 2 wrote:

>Lord Nynex wrote on Monday, October 12, 2015 2:32 PM
>Subject: Re: [openresty-en] worker process request routing

>Not only is this not possible this doesn't make any sense.

I feel like it could make sense

>- Master processes do not do any actual work.
>- Worker processes are separate processes that all listen on a shared socket, thusly can not operate on a request in parallel.

>Can you explain why you think this is a valid use case for any software?

Suppose each of N workers had their own state, and a single call to /update_state?new_value=x could be made to update the state in each of the workers simultaneously- the single call to /update_state could be expanded into N calls to /update_state, one that was routed to each worker. This would solve some of the same problems that lua-resty-lock and the ngx.dict solve.

>Please note, what you're asking is not specific to openresty. You should spend a bit of time studying the nginx documentation.

True, not specific to openresty, and not currently implemented in nginx. But this isn't the first time that the concept of mapping a certain location to a particular worker has been raised, and it seems (to me, at least) like it could be quite useful in some circumstances.

>>Another newbie question:

>>- If multiple calls are made to NGINX, I would like N calls to an nginx master/ N worker set-up guarantee that each of the N calls get routed to >>a different worker.

>>- Can I have a single client call get routed to *all* worker processes?

>>Is any of this possible? If so, any pointers on doing so would be appreciated.

----------------------------------------------------------------------
This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.

aapo.talvensaari

On Monday, 12 October 2015 21:14:55 UTC+3, David Linenberg wrote:

- Can I have a single client call get routed to *all* worker processes?

ngx.timer.at(0, ...)?

But this to work in your scenario, you need the results reported somewhere, right? And for that you can use cosockets inside timer, you can report the worker pid through timer as well. This is what I mean:

1. make request to Nginx

2. one of the available workers gets the request

3. In that worker you run ngx.timer.at(0, ...) and return from the request

Now, nginx will start the function on every worker, so (in MAP-REDUCE, this is MAP):

1. timer function gets executed N times (depending on how many workers you have)

2. after the function has done its calculations it will report the results (optionally with worker pid) to a collector

3. collector can be http endpoint that you can call with cosocket (this endpoint is the REDUCE, or just a collector) OR it could be a database (memcached, redis etc).

Not sure if this helped at all. But I think it answers the question above. But remember that ngx.timer.at is detached from the original request. You could have your original request waiting until all timers have been executed, but for this you would need to use something that can tell that (e.g. shared memory or external db). But remember that timers can exhaust the resources as well, but if you always send request in same order etc. I think it is manageable.

Regards

Aapo

aapo.talvensaari

On Tuesday, 13 October 2015 16:27:02 UTC+3, Aapo Talvensaari wrote:

On Monday, 12 October 2015 21:14:55 UTC+3, David Linenberg wrote:
- Can I have a single client call get routed to *all* worker processes?

ngx.timer.at(0, ...)?

Forget what I just said. ngx.timer(0, ...) only executes on one worker if it is done somewhere else than init_worker_by_lua.

aapo.talvensaari

On Tuesday, 13 October 2015 16:43:20 UTC+3, Aapo Talvensaari wrote:

On Tuesday, 13 October 2015 16:27:02 UTC+3, Aapo Talvensaari wrote:
On Monday, 12 October 2015 21:14:55 UTC+3, David Linenberg wrote:
- Can I have a single client call get routed to *all* worker processes?

ngx.timer.at(0, ...)?

Forget what I just said. ngx.timer(0, ...) only executes on one worker if it is done somewhere else than init_worker_by_lua.

You could still start recurring timer on each worker with init_worker_by_lua, but to pass some state or calculation work to it you will need pub/sub or something. And of course timers need to push their data to somewhere.

Something like this:

1. Each worker has a recurring timer

2. Now you request "/start_calculation_with_state_provided", what happens:

1. You send a message in redis pub/sub and then start waiting for a results by subscribing a results channel

2. Each timer gets notified and will start calculating with a new state

3. When the calculations are finished each timer will report back to results channel

4. When N numbers of results have been received from results channel (or timeout occurred) the original request will be finally served

I'm sure this is what you are NOT looking, and I don't really know if it works, but just something to think about.

erik.l.nelson

David Linenberg wrote on Monday, October 12, 2015 10:35 PM

>On Monday, October 12, 2015 at 7:35:34 PM UTC-4, Nelson, Erik - 2 wrote:
>>Suppose each of N workers had their own state, and a single call to
>>/update_state?new_value=x could be made to update the state in each
>>of the workers simultaneously- the single call to /update_state could
>>be expanded into N calls to /update_state, one that was routed to 
>>worker.  This would solve some of the same problems that
>>lua-resty-lock and the ngx.dict solve.

>This is exactly the case I need to address. We are considering replacing
>our MPI worker nodes with nginx  nodes.  In MPI (Message Passing Interface),
>we can specify exactly what information gets sent to which node (or rank).

>We send commands (requests) to the workers which change their internal state.
>For example, for particularly complex financial instruments, each worker
>node must receive interest rate curve and volatility data before
>any other calculations can be done.  We can't just use shared memory. 
>We must guarantee that all nodes receive the new state data and run a
>function which updates the state accordingly.
>We need to know all nodes have the new state, after which we run any number
>of calculations which can be run on any node.  We then may request a
>change in state again..etc.

It sounds like you're currently proscriptively managing the state on each worker and thinking of them as a monolithic MPI machine.  That may be over-constraining the problem.  Another approach could be to embed a state identifier in the unit of work url, like /eval_instrument?cusip=1234&scenario=shock1.  Then it matters little which worker processes the command- the worker can retrieve the required state from a state server and do the calculation.  This could enable you to mix-and match scenarios, and you could get a little more error tolerance.  Any single command that failed could be re-run without having to initialize the whole logical machine.

>We do stochastic Monte Carlo method calculations - where we do simulations. 
>This is *very* common in the financial industry, running such calculations
>on compute grids.  We send a request to all nodes in our grid (think 1000
>cores or worker nodes).  Each worker node comes back with an answer which
>is a bit different from another worker node - because there is an element
>of randomness (e.g. random number generator) embedded in each worker node's
>calculation.  We expect to get N responses from the N nodes - where we do
>some calculations on the result set.  Consider even simple calculations
>on the basket of results, such as mean or standard deviation...

Again referring to my above comments, the N:N result:node constraint might be a result of the MPI programming construct- if you think of all requests over all jobs as a single stream, you may be able to keep your compute servers more fully loaded by just sending the requests to whatever nodes are available.  Or, as Aapo suggested, have the workers self-feed off a request server.

----------------------------------------------------------------------
This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer.   If you are not the intended recipient, please delete this message.

lordnynex

Hello,

As I've read the original question, this is not the same thing. Duplicating the request context and transmitting it to all running workers seems like a very unnecessary design choice when you have so many other alternatives. This makes sense for changing an upstream, but IMO doesn't make sense for an incoming request.

Off the top of my head, ngx.shared.dict with timers are a much more efficient (and supported) alternative.

Additionally, an external messaging queue (redis pub/sub) would get the job done quite well.

I may have misunderstood the original post though.

-Brandon

On Mon, Oct 12, 2015 at 2:42 PM, <i...@ecsystems.nl> wrote:

Of course this is possible, see https://groups.google.com/forum/#!topic/openresty-en/wt_9m7GvROg

This is a basic framework which can be used for alot of other things, data processing is one thing, for example with some slight changes we use it to process SAP data by a worker who is the least busy without the need to run a separate process. Once its done it offloads the results in memcache for other workers to work with. Netweaver is used to send a trigger message to nginx and IWCP(Lua) does the rest.
You can't use it to run parallel incoming requests since nginx architecture is not suited for that but for worker management and a wide range of task distribution its perfect.

On Monday, October 12, 2015 at 8:14:55 PM UTC+2, David Linenberg wrote:

Hello
Anther newbie question:

- If multiple calls are made to NGINX, I would like N calls to an nginx master/ N worker set-up guarantee that each of the N calls get routed to a different worker.

- Can I have a single client call get routed to *all* worker processes?

Is any of this possible? If so, any pointers on doing so would be appreciated.

Thanks!

David Linenberg

.

erik.l.nelson

I’m fairly new to openresty as well- could you describe how this could be implemented with ngx.shared.dict and timers?

From: openresty-en@googlegroups.com [mailto:openresty-en@googlegroups.com] On Behalf Of Lord Nynex
Sent: Tuesday, October 13, 2015 2:04 PM
To: openresty-en@googlegroups.com
Subject: Re: [openresty-en] Re: worker process request routing

Hello,

Off the top of my head, ngx.shared.dict with timers are a much more efficient (and supported) alternative.

Additionally, an external messaging queue (redis pub/sub) would get the job done quite well.

I may have misunderstood the original post though.

-Brandon

On Mon, Oct 12, 2015 at 2:42 PM, <i...@ecsystems.nl> wrote:

On Monday, October 12, 2015 at 8:14:55 PM UTC+2, David Linenberg wrote:

Hello

Anther newbie question:

- If multiple calls are made to NGINX, I would like N calls to an nginx master/ N worker set-up guarantee that each of the N calls get routed to a different worker.

- Can I have a single client call get routed to *all* worker processes?

Is any of this possible? If so, any pointers on doing so would be appreciated.

Thanks!

David Linenberg

This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.

lordnynex

I'm not sure I fully understand. In practice the worker process is long lived. From a design perspective the worker is generic and can die for a lot of reasons. It seems the only safe design approach is to view the worker as simply a pool of generic workers. This makes ngx.shared.dict idempotent, which is preferable to me. I personally have never had a use case for assigning individual states to workers so I may be missing something obvious.

Again, whats being proposed appears to be duplicating/cloning a request and transmitting it to all or some workers. The much more efficient solution is to use the existing shared memory implementations. Technically, you could use the init functions to 'register' workers at startup and assign unique identifiers to them. In the past, I've done something similar and used resty-qless to implement a messaging interface.

On Mon, Oct 12, 2015 at 4:35 PM, Nelson, Erik - 2 <erik.l...@bankofamerica.com> wrote:

>Lord Nynex wrote on Monday, October 12, 2015 2:32 PM
>Subject: Re: [openresty-en] worker process request routing

>Not only is this not possible this doesn't make any sense.

I feel like it could make sense

>- Master processes do not do any actual work.
>- Worker processes are separate processes that all listen on a shared socket, thusly can not operate on a request in parallel.

>Can you explain why you think this is a valid use case for any software?

Suppose each of N workers had their own state, and a single call to /update_state?new_value=x could be made to update the state in each of the workers simultaneously- the single call to /update_state could be expanded into N calls to /update_state, one that was routed to each worker. This would solve some of the same problems that lua-resty-lock and the ngx.dict solve.

>Please note, what you're asking is not specific to openresty. You should spend a bit of time studying the nginx documentation.

True, not specific to openresty, and not currently implemented in nginx. But this isn't the first time that the concept of mapping a certain location to a particular worker has been raised, and it seems (to me, at least) like it could be quite useful in some circumstances.

>>Another newbie question:

>>- If multiple calls are made to NGINX, I would like N calls to an nginx master/ N worker set-up guarantee that each of the N calls get routed to >>a different worker.

>>- Can I have a single client call get routed to *all* worker processes?

>>Is any of this possible? If so, any pointers on doing so would be appreciated.

----------------------------------------------------------------------
This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.

.

erik.l.nelson

Lord Nynex wrote on Tuesday, October 13, 2015 2:16 PM

>I'm not sure I fully understand. In practice the worker process is long lived.
>From a design perspective the worker is generic and can die for a lot of reasons.
>It seems the only safe design approach is to view the worker as simply a pool
>of generic workers.

Agreed. At least a part of the issue may be thinking of the problem from a MPI programming paradigm.

>On Mon, Oct 12, 2015 at 4:35 PM, Nelson, Erik - 2 <erik.l...@bankofamerica.com> wrote:

>>Lord Nynex wrote on Monday, October 12, 2015 2:32 PM
>>Subject: Re: [openresty-en] worker process request routing

>>Not only is this not possible this doesn't make any sense. 

>I feel like it could make sense

>>- Master processes do not do any actual work. 
>>- Worker processes are separate processes that all listen on a shared socket, thusly can not operate on a request in parallel. 

>>Can you explain why you think this is a valid use case for any software? 

Suppose each of N workers had their own state, and a single call to /update_state?new_value=x could be made to update the state in each of the workers simultaneously- the single call to /update_state could be expanded into N calls to /update_state, one that was routed to each worker.  This would solve some of the same problems that lua-resty-lock and the ngx.dict solve.

>Please note, what you're asking is not specific to openresty. You should spend a bit of time studying the nginx documentation. 

True, not specific to openresty, and not currently implemented in nginx.  But this isn't the first time that the concept of mapping a certain location to a particular worker has been raised, and it seems (to me, at least) like it could be quite useful in some circumstances.

>>Another newbie question:

>>- If multiple calls are made to NGINX, I would like N calls to an nginx master/ N worker set-up guarantee that each of the N calls get routed to >>a different worker.

>>- Can I have a single client call get routed to *all* worker processes?

----------------------------------------------------------------------
This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer.   If you are not the intended recipient, please delete this message.

dlinenbe

Right. In MPI, if a worker dies, your entire "grid" of nodes crashes. That is a fundamental difference. But it doesn't mean I can't adapt the web service approach. Thanks for all of your responses. Much to chew on..

Dave Linenberg

On Tue, Oct 13, 2015 at 2:24 PM, Nelson, Erik - 2 <erik.l...@bankofamerica.com> wrote:

Lord Nynex wrote on Tuesday, October 13, 2015 2:16 PM

>I'm not sure I fully understand. In practice the worker process is long lived.
>From a design perspective the worker is generic and can die for a lot of reasons.
>It seems the only safe design approach is to view the worker as simply a pool
>of generic workers.

Agreed. At least a part of the issue may be thinking of the problem from a MPI programming paradigm.

>On Mon, Oct 12, 2015 at 4:35 PM, Nelson, Erik - 2 <erik.l...@bankofamerica.com> wrote:

>>Lord Nynex wrote on Monday, October 12, 2015 2:32 PM
>>Subject: Re: [openresty-en] worker process request routing

>>Not only is this not possible this doesn't make any sense.

>I feel like it could make sense

>>- Master processes do not do any actual work.
>>- Worker processes are separate processes that all listen on a shared socket, thusly can not operate on a request in parallel.

>>Can you explain why you think this is a valid use case for any software?

Suppose each of N workers had their own state, and a single call to /update_state?new_value=x could be made to update the state in each of the workers simultaneously- the single call to /update_state could be expanded into N calls to /update_state, one that was routed to each worker. This would solve some of the same problems that lua-resty-lock and the ngx.dict solve.

>Please note, what you're asking is not specific to openresty. You should spend a bit of time studying the nginx documentation.

True, not specific to openresty, and not currently implemented in nginx. But this isn't the first time that the concept of mapping a certain location to a particular worker has been raised, and it seems (to me, at least) like it could be quite useful in some circumstances.

>>Another newbie question:

>>- If multiple calls are made to NGINX, I would like N calls to an nginx master/ N worker set-up guarantee that each of the N calls get routed to >>a different worker.

>>- Can I have a single client call get routed to *all* worker processes?

----------------------------------------------------------------------
This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.

--

.