ngx.redirect in body_filter_by_lua_block

rohitratri · 2018-08-10T07:07:37+00:00

Hi all,OpenResty Bay Area meetup is coming back on Aug 23rd, it's been a while since our last meetup, and we have two exciting talks for our next gatheri...

ngx.redirect in body_filter_by_lua_block

rohitratri

Hey folks,

Is there any plan to support usage of ngx.redirect() in body_filter_by_lua_block()?

For a project I am working on, I need to be able to send redirect responses (3XX) to the client based on analysis of the data received from upstream (of course, before I start streaming/letting the upstream response /flow through to the client) from within a body filter.

It may be because I am new to nginx but at first glance it doesn't seem to be straight-forward. I suspected the lack of support for ngx.redirect() within body_filter_by_lua_block() is for the same reason.

If so, any expert suggestions on how best to tackle the situation?

My half-baked (yet to think through it thoroughly) idea is to -

1. Hold back from sending out upstream response headers in the header filter, analyze the upstream response body in the body filter & if required to redirect,

2. Overwrite the headers held back to have a 3XX status code & set the location header, send it out & somehow

3. Close the upstream connection & clean up things [easier to say, I know! Or is it in fact easy?]

Thanks,

Rohit

rohitratri

I think if I set r->headers_only when sending out the redirect headers, item #3 below should be taken care of.

On Thursday, 9 August 2018 16:07:37 UTC-7, Rohit Atri wrote:

Hey folks,

Is there any plan to support usage of ngx.redirect() in body_filter_by_lua_block()?

For a project I am working on, I need to be able to send redirect responses (3XX) to the client based on analysis of the data received from upstream (of course, before I start streaming/letting the upstream response /flow through to the client) from within a body filter.

It may be because I am new to nginx but at first glance it doesn't seem to be straight-forward. I suspected the lack of support for ngx.redirect() within body_filter_by_lua_block() is for the same reason.

If so, any expert suggestions on how best to tackle the situation?

My half-baked (yet to think through it thoroughly) idea is to -
1. Hold back from sending out upstream response headers in the header filter, analyze the upstream response body in the body filter & if required to redirect,
2. Overwrite the headers held back to have a 3XX status code & set the location header, send it out & somehow
3. Close the upstream connection & clean up things [easier to say, I know! Or is it in fact easy?]

Thanks,
Rohit

rpaprocki

Nginx’s design doesn’t allow for this on purpose, because of its streaming nature. If you want to adjust the response based on some arbitrary condition within the response body, you need to buffer be whole body before sending it (including headers) to the client.

Generally this is done by request the body in a content by lua handler and inspecting he response. Be aware of the memory cost of buffer large response bodies.

Sent from my iPhone

On Aug 9, 2018, at 16:22, Rohit Atri <rohi...@gmail.com> wrote:

I think if I set r->headers_only when sending out the redirect headers, item #3 below should be taken care of.

On Thursday, 9 August 2018 16:07:37 UTC-7, Rohit Atri wrote:
Hey folks,

Is there any plan to support usage of ngx.redirect() in body_filter_by_lua_block()?

For a project I am working on, I need to be able to send redirect responses (3XX) to the client based on analysis of the data received from upstream (of course, before I start streaming/letting the upstream response /flow through to the client) from within a body filter.

It may be because I am new to nginx but at first glance it doesn't seem to be straight-forward. I suspected the lack of support for ngx.redirect() within body_filter_by_lua_block() is for the same reason.

If so, any expert suggestions on how best to tackle the situation?

My half-baked (yet to think through it thoroughly) idea is to -
1. Hold back from sending out upstream response headers in the header filter, analyze the upstream response body in the body filter & if required to redirect,
2. Overwrite the headers held back to have a 3XX status code & set the location header, send it out & somehow
3. Close the upstream connection & clean up things [easier to say, I know! Or is it in fact easy?]

Thanks,
Rohit

.

rohitratri

Yeah, I am prepared to buffer the headers & body. I will probably thrown in a sensible buffer size threshold as well.

But as you highlight, I am a little concerned I am breaking the law per se & wonder if there are going to be any unforeseen consequences. I will probably just go ahead & give it a shot. But if its a criminal offense, please shout at me!

Cheers,

Rohit

On Thursday, 9 August 2018 16:25:14 UTC-7, rpaprocki wrote:

Nginx’s design doesn’t allow for this on purpose, because of its streaming nature. If you want to adjust the response based on some arbitrary condition within the response body, you need to buffer be whole body before sending it (including headers) to the client.

Generally this is done by request the body in a content by lua handler and inspecting he response. Be aware of the memory cost of buffer large response bodies.

Sent from my iPhone

On Aug 9, 2018, at 16:22, Rohit Atri <rohit...@gmail.com> wrote:

I think if I set r->headers_only when sending out the redirect headers, item #3 below should be taken care of.

On Thursday, 9 August 2018 16:07:37 UTC-7, Rohit Atri wrote:
Hey folks,

Is there any plan to support usage of ngx.redirect() in body_filter_by_lua_block()?

For a project I am working on, I need to be able to send redirect responses (3XX) to the client based on analysis of the data received from upstream (of course, before I start streaming/letting the upstream response /flow through to the client) from within a body filter.

It may be because I am new to nginx but at first glance it doesn't seem to be straight-forward. I suspected the lack of support for ngx.redirect() within body_filter_by_lua_block() is for the same reason.

If so, any expert suggestions on how best to tackle the situation?

My half-baked (yet to think through it thoroughly) idea is to -
1. Hold back from sending out upstream response headers in the header filter, analyze the upstream response body in the body filter & if required to redirect,
2. Overwrite the headers held back to have a 3XX status code & set the location header, send it out & somehow
3. Close the upstream connection & clean up things [easier to say, I know! Or is it in fact easy?]

Thanks,
Rohit

.

rpaprocki

To be clear, what I am referring to is doing this with custom Lua, and not using the nginx proxy_pass mechanism. There is no way to do this is nginx, without a custom module.

Doing this is also likely to slow throughout a but as there is more concurrent work going on inside each worker process. Just something to be aware of. Testing and profiling your config and code is the best way to avoid unintended hiccups.

Sent from my iPhone

On Aug 9, 2018, at 16:33, Rohit Atri <rohi...@gmail.com> wrote:

Yeah, I am prepared to buffer the headers & body. I will probably thrown in a sensible buffer size threshold as well.
But as you highlight, I am a little concerned I am breaking the law per se & wonder if there are going to be any unforeseen consequences. I will probably just go ahead & give it a shot. But if its a criminal offense, please shout at me!

Cheers,
Rohit

On Thursday, 9 August 2018 16:25:14 UTC-7, rpaprocki wrote:
Nginx’s design doesn’t allow for this on purpose, because of its streaming nature. If you want to adjust the response based on some arbitrary condition within the response body, you need to buffer be whole body before sending it (including headers) to the client.

Generally this is done by request the body in a content by lua handler and inspecting he response. Be aware of the memory cost of buffer large response bodies.

Sent from my iPhone

On Aug 9, 2018, at 16:22, Rohit Atri <rohit...@gmail.com> wrote:

I think if I set r->headers_only when sending out the redirect headers, item #3 below should be taken care of.

On Thursday, 9 August 2018 16:07:37 UTC-7, Rohit Atri wrote:
Hey folks,

Is there any plan to support usage of ngx.redirect() in body_filter_by_lua_block()?

For a project I am working on, I need to be able to send redirect responses (3XX) to the client based on analysis of the data received from upstream (of course, before I start streaming/letting the upstream response /flow through to the client) from within a body filter.

It may be because I am new to nginx but at first glance it doesn't seem to be straight-forward. I suspected the lack of support for ngx.redirect() within body_filter_by_lua_block() is for the same reason.

If so, any expert suggestions on how best to tackle the situation?

My half-baked (yet to think through it thoroughly) idea is to -
1. Hold back from sending out upstream response headers in the header filter, analyze the upstream response body in the body filter & if required to redirect,
2. Overwrite the headers held back to have a 3XX status code & set the location header, send it out & somehow
3. Close the upstream connection & clean up things [easier to say, I know! Or is it in fact easy?]

Thanks,
Rohit

.