ngx.balancer feature request?

wuhao.wise · 2016-03-14T13:51:56+00:00

Hello,The root cause of the issue has been identified. It has nothing to do with Openresty or LuaJIT/FFI, but with certain design choices of malloc / glibc o...

ngx.balancer feature request?

wuhao.wise

I needed to use "session" to dispatch the requests and those sessions are dynamically created and the backend servers are dynamically maintained, ngx.balancer is exactly what I was looking for, thanks!

I do have a few questions (more like feature requests?)

when the backend server sends back the error messages (404 or 500), I will need to inspect it before it moves to the next server. right now, I can set retry so the upstream will do it for me (proxy_next_upstream directive) and the script will get run when it already moves to the next server. basically, I need to do a bit more than header check when the backend server comes back with error messages.

I could use ngx.location.capture, but I feel proxying would be faster w/o much memory copy, and when forwarding the data from the backend if using location.capture, it seems to be more corner cases that need to be handled (temp files/memory/etc)

agentzh

Hello!

On Sun, Mar 13, 2016 at 10:51 PM, Hao Wu wrote:
> when the backend server sends back the error messages (404 or 500), I will
> need to inspect it before it moves to the next server. right now, I can set
> retry so the upstream will do it for me (proxy_next_upstream directive) and
> the script will get run when it already moves to the next server. basically,
> I need to do a bit more than header check when the backend server comes back
> with error messages.
>

Sorry, I don't think I quite understand your requirement here. Will
you elaborate the "a bit more" part? Better with psuedo code snippets
if they can help.

Regards,
-agentzh

wuhao.wise

On Wednesday, March 16, 2016 at 1:08:17 PM UTC-7, agentzh wrote:

Hello!

On Sun, Mar 13, 2016 at 10:51 PM, Hao Wu wrote:
> when the backend server sends back the error messages (404 or 500), I will
> need to inspect it before it moves to the next server. right now, I can set
> retry so the upstream will do it for me (proxy_next_upstream directive) and
> the script will get run when it already moves to the next server. basically,
> I need to do a bit more than header check when the backend server comes back
> with error messages.
>

Sorry, I don't think I quite understand your requirement here. Will
you elaborate the "a bit more" part? Better with psuedo code snippets
if they can help.

Can we have some hooks like what proxy_next_upstream does? i.e.

next_stream_by_lua_block {

if (upstream.connection is broken)

return true

end

if (upstream.status == 404 and upstream.body contains "XXX") then

return true; -- this will try the next upstream

else

return false; -- this will forward back to the client

end

}

and the body will be only read to the memory if it is being accessed and/or to avoid to read the full body response, some API can be like, upstream.getBody(range),

is this possible?

Regards,
-agentzh

agentzh

Hello!

On Wed, Mar 16, 2016 at 9:47 PM, Hao Wu wrote:
>
> Can we have some hooks like what proxy_next_upstream does? i.e.
>
> next_stream_by_lua_block {

balancer_by_lua* are already called upon retries. And you can fetch
the last failed attempt's status via the `balancer.get_last_failure`
method call.

>     if (upstream.connection is broken)
>        return true
>     end
>     if (upstream.status == 404 and upstream.body contains "XXX") then

It's not possible to inspect response body for failed attempts because
the body has not yet been read at that point (and once the nginx
starts reading and sending out the response body data, retries won't
happen because part of the response body has already been sent to the
client).

Regards,
-agentzh

wuhao.wise

On Thursday, March 24, 2016 at 12:52:30 PM UTC-7, agentzh wrote:

Hello!

On Wed, Mar 16, 2016 at 9:47 PM, Hao Wu wrote:
>
> Can we have some hooks like what proxy_next_upstream does? i.e.
>
> next_stream_by_lua_block {

balancer_by_lua* are already called upon retries. And you can fetch
the last failed attempt's status via the `balancer.get_last_failure`
method call.

Yea, I am using it, but I sorta need more than just status code but also the response body.

> if (upstream.connection is broken)
> return true
> end
> if (upstream.status == 404 and upstream.body contains "XXX") then

It's not possible to inspect response body for failed attempts because
the body has not yet been read at that point (and once the nginx
starts reading and sending out the response body data, retries won't
happen because part of the response body has already been sent to the
client).

I see, so is there a way to inspect the response before sending out to the client (other than ngx.location.capture)?

Regards,
-agentzh

right now I can use proxy_next_stream to capture special status code like 502 or 500, but I will need to modify the upstream to return such status code as than baking them in the body like they used to do (which is not ideal but). I am trying to figure out a way to dynamically inspect the upstream response and forward to the client. Can you advise?

agentzh

Hello!

On Thu, Mar 24, 2016 at 2:23 PM, Hao Wu wrote:
>
> I see, so is there a way to inspect the response before sending out to the
> client (other than ngx.location.capture)?
>

The ngx_proxy module uses streaming processing so that it does not
have to buffer all the response body data in memory before sending a
byte to the client. This is crucial to performance otherwise it would
be crazily expensive when serving large files (like gigs of bytes).

> right now I can use proxy_next_stream to capture special status code like
> 502 or 500, but I will need to modify the upstream to return such status
> code as than baking them in the body like they used to do (which is not
> ideal but). I am trying to figure out a way to dynamically inspect the
> upstream response and forward to the client. Can you advise?
>

proxy_next_upstream and response body inspection are exclusive. See above.

Regards,
-agentzh

wuhao.wise

On Saturday, March 26, 2016 at 2:11:19 PM UTC-7, agentzh wrote:

Hello!

On Thu, Mar 24, 2016 at 2:23 PM, Hao Wu wrote:
>
> I see, so is there a way to inspect the response before sending out to the
> client (other than ngx.location.capture)?
>

The ngx_proxy module uses streaming processing so that it does not
have to buffer all the response body data in memory before sending a
byte to the client. This is crucial to performance otherwise it would
be crazily expensive when serving large files (like gigs of bytes).

> right now I can use proxy_next_stream to capture special status code like
> 502 or 500, but I will need to modify the upstream to return such status
> code as than baking them in the body like they used to do (which is not
> ideal but). I am trying to figure out a way to dynamically inspect the
> upstream response and forward to the client. Can you advise?
>

proxy_next_upstream and response body inspection are exclusive. See above.

I see, I think the only option left to me is to modify the upstream and return the proper status code so the proxy can catch it and react.

Thanks a lot!

Regards,
-agentzh