methods available in balancer_by_lua

david.birdsong · 2015-12-23T03:18:23+00:00

Hello! On Mon, Dec 21, 2015 at 9:38 PM, Ted Timmons wrote: > Here's an example: > > access_by_lua ' > if not ngx.var.http_x_forwarded_proto ==...

methods available in balancer_by_lua

david.birdsong

I'm going through trial and error now, but a faster way of reasoning about what's available in balancer_by_lua would be really helpful.

It looks like ngx.req.set_uri runs but has no effect which I'm not too surprised by given those actions probably have to happen much earlier.

What about timers?

To come at it a different way,

if in a

balancer_by_lua_block {
m.set_more_tries(0)
}

...is there a way to pass control to another phase that could choose a different upstream without losing ngx.ctx?

I'm pretty sure that using error_page and proxy_intercept_errors would work, but AFAIK that mechanism is using rewrite under the hood which clears ngx.ctx.

agentzh

Hi David

On Tue, Dec 22, 2015 at 11:18 AM, David Birdsong wrote:
> I'm going through trial and error now, but a faster way of reasoning about
> what's available in balancer_by_lua would be really helpful.
>

The balancer_by_lua thing is still a new territory and we are also
exploring possibilities there ourselves. Support for more and more Lua
APIs are added upon user requests. So if you want to use some API
functions there but they break, please just file a ticket on GitHub
(or better, submitting a GitHub pull request) under the
lua-nginx-module repos :)

> It looks like ngx.req.set_uri runs but has no effect which I'm not too
> surprised by given those actions probably have to happen much earlier.
>

My hunch is that the upstream request is already constructed from the
downstream request URI and your changes in balancer_by_lua still
change the saved copy of the downstream request record, but it won't
trigger re-generation of the upstream request buffer, for obvious
reasons.

As a rule of thumb, one should do preparation (like cosocket accesses
and etc) in an earlier phase, like access_by_lua*, and store the data
in ngx.ctx for later references in balancer_by_lua*, for example.

> What about timers?
>

Timers should work inside balancer_by_lua*, if not, please file a
ticket on GitHub (or a pull request even) :)

> To come at it a different way,
> if in a
> balancer_by_lua_block {
>   m.set_more_tries(0)
> }
> ...is there a way to pass control to another phase that could choose a
> different upstream without losing ngx.ctx?

Sorry, I don't follow your intention here.

>
> I'm pretty sure that using error_page and proxy_intercept_errors would work,
> but AFAIK that mechanism is using rewrite under the hood which clears
> ngx.ctx.
>

Correct. Internal redirects overwrite the magical ngx.ctx table.

Regards,
-agentzh

david.birdsong

On Tue, Dec 22, 2015 at 12:26 PM, Yichun Zhang (agentzh) <age...@gmail.com> wrote:

Hi David

On Tue, Dec 22, 2015 at 11:18 AM, David Birdsong wrote:
> I'm going through trial and error now, but a faster way of reasoning about
> what's available in balancer_by_lua would be really helpful.
>

The balancer_by_lua thing is still a new territory and we are also
exploring possibilities there ourselves. Support for more and more Lua
APIs are added upon user requests. So if you want to use some API
functions there but they break, please just file a ticket on GitHub
(or better, submitting a GitHub pull request) under the
lua-nginx-module repos :)

> It looks like ngx.req.set_uri runs but has no effect which I'm not too
> surprised by given those actions probably have to happen much earlier.
>

My hunch is that the upstream request is already constructed from the
downstream request URI and your changes in balancer_by_lua still
change the saved copy of the downstream request record, but it won't
trigger re-generation of the upstream request buffer, for obvious
reasons.

As a rule of thumb, one should do preparation (like cosocket accesses
and etc) in an earlier phase, like access_by_lua*, and store the data
in ngx.ctx for later references in balancer_by_lua*, for example.

> What about timers?
>

Timers should work inside balancer_by_lua*, if not, please file a
ticket on GitHub (or a pull request even) :)

> To come at it a different way,
> if in a
> balancer_by_lua_block {
> m.set_more_tries(0)
> }
> ...is there a way to pass control to another phase that could choose a
> different upstream without losing ngx.ctx?

Sorry, I don't follow your intention here.

I'm trying to balance minimal example with actual use-case--I'll try again.

I'm using ngx.ctx to attach counters to for metrics which will get sent somewhere useful in log_by_lua. If there's a place to store counter values discovered during a request, I'd love to hear it. For ngx.ctx to be useful, I have to avoid any internal rewrites.

During a request, I have info on how to satisfy the request also attached to ngx.ctx. I want to try one backend and use balancer_by_lua to retry against a set of peers until I exhaust some condition at which point I want try another backend which takes a different URL pattern. balancer_by_lua would have me use a dummy upstream and set the peer accordingly to my backend, which would be fine, but the URL needs to be changed.

Is there a way to:

- preserve ngx.ctx

- try an upstream, find a failure condition, exit the upstream

- re-enter any of (rewrite,access,set)_by_lua to change the url

>
> I'm pretty sure that using error_page and proxy_intercept_errors would work,
> but AFAIK that mechanism is using rewrite under the hood which clears
> ngx.ctx.
>

Correct. Internal redirects overwrite the magical ngx.ctx table.

Regards,
-agentzh
.

agentzh

Hi David

On Tue, Dec 22, 2015 at 12:48 PM, David Birdsong wrote:
>
> I'm trying to balance minimal example with actual use-case--I'll try again.
>
> I'm using ngx.ctx to attach counters to for metrics which will get sent
> somewhere useful in log_by_lua. If there's a place to store counter values
> discovered during a request, I'd love to hear it. For ngx.ctx to be useful,
> I have to avoid any internal rewrites.
>

One kinda hacky solution is to insert a dummy request header via
ngx.req.set_header(). Request headers survive across internal
redirects but module contexts don't (including ngx_lua's ngx.ctx, of
course).

Usually we should do internal redirects earlier rather than later so
that we don't have any meaningful ngx.ctx data to preserve.

> During a request, I have info on how to satisfy the request also attached to
> ngx.ctx. I want to try one backend and use balancer_by_lua to retry against
> a set of peers until I exhaust some condition at which point I want try
> another backend which takes a different URL pattern. balancer_by_lua would
> have me use a dummy upstream and set the peer accordingly to my backend,
> which would be fine, but the URL needs to be changed.
>

Nope, nginx's next_upstream (or upstream retries) do not support
changing the upstream requests. But maybe we can expose a Lua API to
forcibly overwrite the upstream request buffer so that the next trial
can pick the new upstream request buffer up. This is an interesting
feature. Will you work on a patch for this?

Hope these help.

Best regards,
-agentzh

david.birdsong

On Tue, Dec 22, 2015 at 12:58 PM, Yichun Zhang (agentzh) <age...@gmail.com> wrote:

Hi David

On Tue, Dec 22, 2015 at 12:48 PM, David Birdsong wrote:
>
> I'm trying to balance minimal example with actual use-case--I'll try again.
>
> I'm using ngx.ctx to attach counters to for metrics which will get sent
> somewhere useful in log_by_lua. If there's a place to store counter values
> discovered during a request, I'd love to hear it. For ngx.ctx to be useful,
> I have to avoid any internal rewrites.
>

One kinda hacky solution is to insert a dummy request header via
ngx.req.set_header(). Request headers survive across internal
redirects but module contexts don't (including ngx_lua's ngx.ctx, of
course).

Usually we should do internal redirects earlier rather than later so
that we don't have any meaningful ngx.ctx data to preserve.

> During a request, I have info on how to satisfy the request also attached to
> ngx.ctx. I want to try one backend and use balancer_by_lua to retry against
> a set of peers until I exhaust some condition at which point I want try
> another backend which takes a different URL pattern. balancer_by_lua would
> have me use a dummy upstream and set the peer accordingly to my backend,
> which would be fine, but the URL needs to be changed.
>

Nope, nginx's next_upstream (or upstream retries) do not support
changing the upstream requests. But maybe we can expose a Lua API to
forcibly overwrite the upstream request buffer so that the next trial
can pick the new upstream request buffer up. This is an interesting
feature. Will you work on a patch for this?

it's interesting and a good chance for me to contribute back. unfortunately my C isn't very good and i need to squeeze in this refactor to use balancer_by_lua* over the holidays while i have some quiet time.

since i own both backend, the faster option for me is to unify the url structure.

Hope these help.

Best regards,

-agentzh.

agentzh

Hello!

On Tue, Dec 22, 2015 at 2:56 PM, David Birdsong wrote:
>
> it's interesting and a good chance for me to contribute back. unfortunately
> my C isn't very good and i need to squeeze in this refactor to use
> balancer_by_lua* over the holidays while i have some quiet time.
>

Maybe we can add a Lua function or functions to ngx.balancer to
enforce upstream request regeneration upon next trial. It seems that
it's as easy as calling u->create_request() upon the next attempt.
Something like this, maybe?

    local balancer = require "ngx.balancer"

    local ok, err = balancer.set_next_upstream_uri(uri)
    local ok, err = balancer.set_next_upstream_uri_args(args)

This use case looks fun to me :)

Regards,
-agentzh

david.birdsong

On Tue, Dec 22, 2015, 3:10 PM Yichun Zhang (agentzh) <age...@gmail.com> wrote:

Hello!

On Tue, Dec 22, 2015 at 2:56 PM, David Birdsong wrote:
>
> it's interesting and a good chance for me to contribute back. unfortunately
> my C isn't very good and i need to squeeze in this refactor to use
> balancer_by_lua* over the holidays while i have some quiet time.
>

Maybe we can add a Lua function or functions to ngx.balancer to
enforce upstream request regeneration upon next trial. It seems that
it's as easy as calling u->create_request() upon the next attempt.
Something like this, maybe?

local balancer = require "ngx.balancer"

local ok, err = balancer.set_next_upstream_uri(uri)
local ok, err = balancer.set_next_upstream_uri_args(args)

This use case looks fun to me :)

nice!

Regards,
-agentzh.

eahmwah

Do we have these methods available to change the upstream uri in balancer_by_lua block?

On Tuesday, December 22, 2015 at 6:10:26 PM UTC-5, agentzh wrote:

Hello!

On Tue, Dec 22, 2015 at 2:56 PM, David Birdsong wrote:
>
> it's interesting and a good chance for me to contribute back. unfortunately
> my C isn't very good and i need to squeeze in this refactor to use
> balancer_by_lua* over the holidays while i have some quiet time.
>

Maybe we can add a Lua function or functions to ngx.balancer to
enforce upstream request regeneration upon next trial. It seems that
it's as easy as calling u->create_request() upon the next attempt.
Something like this, maybe?

local balancer = require "ngx.balancer"

local ok, err = balancer.set_next_upstream_uri(uri)
local ok, err = balancer.set_next_upstream_uri_args(args)

This use case looks fun to me :)

Regards,
-agentzh