ngx.location.capture changes

aviram · 2013-03-18T03:37:22+00:00

What are the best options for websocket backends? I'm working with an embedded platform so I'm somewhat restricted to something lightweight in C/C++. The t...

ngx.location.capture changes

aviram

Hi,

I use ngx.location.capture and I am worried about huge buffers laying in
nginx/lua memory, just like James Hurst in this discussion
- https://groups.google.com/forum/?fromgroups=#!searchin/openresty-en/lua$20body$20filter/openresty-en/v4ZezGqvxWI/xmXEEIAAd3AJ
.
So, I would like to implement the idea that agentzh suggested in his last
response - adding a "max_body_size" option to the capture's option table.
The thing is, that if the subrequest's response body size exceeds the size
I'd like to handle in the Lua script, I'd like to stream the data back to
the client, without it being transferred back to Lua.

The implementation that I thought of is the following - while reading the
subrequest's response, if it exceeds max_body_size, call ngx.print with the
data.

This raises two questions:
1. Is it possible to call ngx.print (or the apt C function) from the
context in which the subrequest's body is read?
2. I've looked at the code, and it seems from there that the body is copied
to a Lua buffer in ngx_http_lua_subrequest_resume, which happens
after ngx_http_lua_post_subrequest is invoked. My question is -
when ngx_http_lua_post_subrequest is invoked, is the entire subrequest's
response body in the memory? If so, I would like to find a way to read the
subrequest's body as it returns and then decide what to do (like a body
filter). Is that possible?

Thanks,
Aviram

agentzh

Hello!

On Sun, Mar 17, 2013 at 12:37 PM,  aviram wrote:
> I use ngx.location.capture and I am worried about huge buffers laying in
> nginx/lua memory, just like James Hurst in this discussion
> -

Well, this is a hard problem :) I'm glad you're looking into this :)

> .
> So, I would like to implement the idea that agentzh suggested in his last
> response - adding a "max_body_size" option to the capture's option table.
> The thing is, that if the subrequest's response body size exceeds the size
> I'd like to handle in the Lua script, I'd like to stream the data back to
> the client, without it being transferred back to Lua.

Actually, in my original proposal, I was thinking about simply
truncating the subrequest's response body without transferring it
downstream.

I think your approach is functionally similar to the echo_location
directive in the ngx_echo module, although the implementation details
differ:

    http://wiki.nginx.org/HttpEchoModule#echo_location

Actually I'm thinking about something more general (and more
powerful), that is, a streaming processing API for subrequests'
responses in the iterator style:

    local res = ngx.location.capture_stream("/foo",
                                { args = "...", chunk_size = 4096 })
    my_process_response_header(res.header)
    local it = res.body_iterator
    while true then
        local chunk, err = it()
        if not chunk then
            if err then
                ngx.log(ngx.ERR, "bad thing happened: ", err)
                ngx.exit(ngx.ERROR)
            end
            -- reached the end of the stream successfully
            break
        end
        chunk = my_process_data_chunk(chunk)
        ngx.print(chunk)
    end

What do you think of this?

On the other hand, I'm afraid your proposal is just too specific to be
really useful for others. But I could be wrong :)

>
> The implementation that I thought of is the following - while reading the
> subrequest's response, if it exceeds max_body_size, call ngx.print with the
> data.
>
> This raises two questions:
> 1. Is it possible to call ngx.print (or the apt C function) from the
> context in which the subrequest's body is read?

Yes.

> 2. I've looked at the code, and it seems from there that the body is copied
> to a Lua buffer in ngx_http_lua_subrequest_resume, which happens
> after ngx_http_lua_post_subrequest is invoked. My question is -
> when ngx_http_lua_post_subrequest is invoked, is the entire subrequest's
> response body in the memory?

Yes.

> If so, I would like to find a way to read the
> subrequest's body as it returns and then decide what to do (like a body
> filter). Is that possible?
>

You have to do this in the body filter for the subrequests, i.e., in
the existing ngx_http_lua_capture_body_filter function. When you have
accumulated enough data in the body filter for the subrequest, you
just need to wake up its parent request and transfer the data chunk to
it. You'll surely need coordinate everything here via the ctx data, of
course.

If all you need is something like the "echo_location" directive for
Lua, then maybe we can just introduce an new "ngx.location.echo()" API
that could be implemented in a more efficient manner (than the
interator API for reading subrequest responses).

Best regards,
-agentzh

brian

This is one reason we've started moving away from using
ngx.location.capture. To get the level of control you describe, it
would greatly complicate the code, I think. We've started using Lua
modules for almost everything rather than ngx.location.capture. We've
been pleasantly surprised by the performance and the decreased memory
footprint and enjoy much greater control of the data flow.

--Brian

agentzh

Hello!

On Sun, Mar 17, 2013 at 2:05 PM, Brian Akins wrote:
> This is one reason we've started moving away from using
> ngx.location.capture. To get the level of control you describe, it
> would greatly complicate the code, I think. We've started using Lua
> modules for almost everything rather than ngx.location.capture. We've
> been pleasantly surprised by the performance and the decreased memory
> footprint and enjoy much greater control of the data flow.
>

I must say this is a good move that I'd love to see :)

The Nginx subrequest approach is mostly for reusing the existing Nginx
C modules like ngx_proxy, which is (much) more complicated (and thus
potentially slower) than the pure Lua solution based on the cosocket
API.

But given that one of ngx_lua's goals is to script the existing (pure
C) Nginx ecosystem, I'm still willing to enhancing the existing
subrequest API in ngx_lua, if not adding too much complication to the
code base.

Best regards,
-agentzh

aviram

Hi!

agentzh, I thought about your proposal of processing the subrequest's body chunked via an iterator or a readable object, but it seems too complicated to implement (as in - it requires some major changes and would take more time).

I know subrequests aren't the best practice, but I could not find a better alternative.

Maybe you can help me? Here's what I want to do -

- I've got nginx defined as a reverse proxy to another site.

- I want to be able to read the requests coming in to the proxy, read them by chunks, use cosockets (to communicate with another process), and possibly alter the request. This is possible with ngx_lua.

- In some cases, I want to be able to read the responses coming from the proxied server in chunks, use cosockets (to communicate with another process), and possibly change the response. For that to work efficiently enough, I would sometimes like to decide whether I want to read the entire body or not based on the response header or even on the partial body of the response. If I won't handle it - I'd like to pass it along to the client. This is the problematic part, mainly because the header and body filters do not support cosockets.

My solution was to create a location for the proxy (for which the content handler is proxy_pass), and in the main location use lua content handler, that performs a subrequest, and that's why the change I described here fits my purpose (I know it doesn't really sound general, but maybe in the future it could be enhanced to be more general, i.e. having a lua callback called when maximum body size has reached).

I thought about the following alternative ideas:

1. Making cosockets available from lua header and body filters.

2 .Writing the subrequest and its processing in C (in which case, I'd have to integrate with the nginx reactor for communication operations).

Both ideas seem to be much more work (obviously, I'd like to write as much as I can in Lua rather than in C).

Do you have any alternatives?

Regards,

Aviram

On Sunday, March 17, 2013 11:58:50 PM UTC+2, agentzh wrote:

Hello!

On Sun, Mar 17, 2013 at 2:05 PM, Brian Akins wrote:
> This is one reason we've started moving away from using
> ngx.location.capture. To get the level of control you describe, it
> would greatly complicate the code, I think. We've started using Lua
> modules for almost everything rather than ngx.location.capture. We've
> been pleasantly surprised by the performance and the decreased memory
> footprint and enjoy much greater control of the data flow.
>

I must say this is a good move that I'd love to see :)

The Nginx subrequest approach is mostly for reusing the existing Nginx
C modules like ngx_proxy, which is (much) more complicated (and thus
potentially slower) than the pure Lua solution based on the cosocket
API.

But given that one of ngx_lua's goals is to script the existing (pure
C) Nginx ecosystem, I'm still willing to enhancing the existing
subrequest API in ngx_lua, if not adding too much complication to the
code base.

Best regards,
-agentzh

agentzh

Hello!

On Sun, Mar 17, 2013 at 3:23 PM,  aviram wrote:
> agentzh, I thought about your proposal of processing the subrequest's body
> chunked via an iterator or a readable object, but it seems too complicated
> to implement (as in - it requires some major changes and would take more
> time).
>

I know, I know :) But I think it is the right way to go if we want to
stick with the subrequest approach.

> I know subrequests aren't the best practice, but I could not find a better
> alternative.
> Maybe you can help me?

If you do not use https nor nginx's file-based http_cache thing for
your backend services, then you may want to use (or enhance) a
lua-resty-http library to do all the proxing thing  (and other related
processing) in Lua. I think this is what Brian Akins was suggesting in
his earlier reply to this thread :)

> I thought about the following alternative ideas:
>   1. Making cosockets available from lua header and body filters.

This requires redesigning half of the Nginx core, which is not
practical at all :)

>   2 .Writing the subrequest and its processing in C (in which case, I'd have
> to integrate with the nginx reactor for communication operations).
> Both ideas seem to be much more work (obviously, I'd like to write as much
> as I can in Lua rather than in C).
>

I don't think you want to mud with C here with Nginx subrequests
directly. It is HARD to get right :)

> Do you have any alternatives?
>

See above :)

Best regards,
-agentzh

aviram

Hi,

Thanks for the replies, Brian and agentzh. Unfortunately, the site(s) to which I'm proxying is both HTTP and HTTPS, so SSL support is required.

lua-resty-http seems cool, but still under development, and therefore I don't think it's suitable (I'm talking about handling massive amounts of traffic).

Anyway, if I find a good solution, I'll share it with you.

Thanks very much :)

On Sunday, March 17, 2013 9:37:22 PM UTC+2, avi...@adallom.com wrote:

Hi,

I use ngx.location.capture and I am worried about huge buffers laying in
nginx/lua memory, just like James Hurst in this discussion
- https://groups.google.com/forum/?fromgroups=#!searchin/openresty-en/lua$20body$20filter/openresty-en/v4ZezGqvxWI/xmXEEIAAd3AJ
.
So, I would like to implement the idea that agentzh suggested in his last
response - adding a "max_body_size" option to the capture's option table.
The thing is, that if the subrequest's response body size exceeds the size
I'd like to handle in the Lua script, I'd like to stream the data back to
the client, without it being transferred back to Lua.

The implementation that I thought of is the following - while reading the
subrequest's response, if it exceeds max_body_size, call ngx.print with the
data.

This raises two questions:
1. Is it possible to call ngx.print (or the apt C function) from the
context in which the subrequest's body is read?
2. I've looked at the code, and it seems from there that the body is copied
to a Lua buffer in ngx_http_lua_subrequest_resume, which happens
after ngx_http_lua_post_subrequest is invoked. My question is -
when ngx_http_lua_post_subrequest is invoked, is the entire subrequest's
response body in the memory? If so, I would like to find a way to read the
subrequest's body as it returns and then decide what to do (like a body
filter). Is that possible?

Thanks,
Aviram

aviram

agentzh,

> If so, I would like to find a way to read the
> subrequest's body as it returns and then decide what to do (like a body
> filter). Is that possible?
>

You have to do this in the body filter for the subrequests, i.e., in
the existing ngx_http_lua_capture_body_filter function. When you have
accumulated enough data in the body filter for the subrequest, you
just need to wake up its parent request and transfer the data chunk to
it. You'll surely need coordinate everything here via the ctx data, of
course.

Thanks for tip, but how do you wake the parent request from ngx_http_lua_capture_body_filter?

I guess that the data copy can be direct - copying the chain of the response of the subrequest to the chain of the response of the parent request. Is it okay?

I'm thinking about adding two options to the capture interface - "max_body_size" and "truncate_on_max_body_size". If max_body_size exceeds, based on the value of truncate_on_max_body_size the request would either be truncated or passed through back to the client. The default would be true (which means that the request would be truncated). Do you think that's general enough?

brian

On Mon, Mar 18, 2013 at 2:48 AM,  <av...@adallom.com> wrote:
> therefore I
> don't think it's suitable (I'm talking about handling massive amounts of
> traffic).

Define "massive." I think you'd be surprised what some of us are doing
with "development" code...

yes, lack of ssl support is an issue for us as well. I figure there's
a way to figure that out at the lua socket layer, rather than each
module figuring it out.

I'd like an http module (in Lua) that behaved like libcurl -
callbacks, etc - as someone earlier in this thread described. The HTTP
code I was hacking on could be made to work that way, but then it
wouldn't be so "simple," I suppose.

aviram

Define "massive." I think you'd be surprised what some of us are doing
with "development" code...

By "massive" I mean something like a medium company's (hundreds to thousands of employees) entire traffic, but I guess the issue with the development code is not the amount of traffic, but both the quality of the code (without reading and testing the code I can't really refer to that) and the fact that the API is unstable. All in all, these aren't as main issues as the SSL issue.

yes, lack of ssl support is an issue for us as well. I figure there's
a way to figure that out at the lua socket layer, rather than each
module figuring it out.

I agree.

I'd like an http module (in Lua) that behaved like libcurl -
callbacks, etc - as someone earlier in this thread described. The HTTP
code I was hacking on could be made to work that way, but then it
wouldn't be so "simple," I suppose.

Again I agree, and whoever implements it should support reading the responses chunked...

brian

On Mon, Mar 18, 2013 at 7:53 AM,  <av...@adallom.com> wrote:
> By "massive" I mean something like a medium company's (hundreds to thousands
> of employees) entire traffic,

FWIW, that's not massive by our definition. Other's on the list have
larger definitions of "massive" as well.

>  but I guess the issue with the development
> code is not the amount of traffic, but both the quality of the code (without
> reading and testing the code I can't really refer to that) and the fact that
> the API is unstable.

Sure. We admittedly are more comfortable with our own "development"
code than that of others.

SSL is a weak point for me, to be honest, so I'm afraid I may not be
much help there - at least not in a timely manner.

--Brian

appa

Hmm, just to take advantage of the discussion about SSL. I guess agentzh has not yet found time to

address the SSL part. Since SSL happens below HTTP there's no way to get around Nginx default SSL

handling even if we use LuaJIT's FFI with OpenSSL or GnuTLS, or is it?

Thanks,

----appa

On Mon, Mar 18, 2013 at 5:15 PM, Brian Akins <br...@akins.org> wrote:

On Mon, Mar 18, 2013 at 7:53 AM, <av...@adallom.com> wrote:
> By "massive" I mean something like a medium company's (hundreds to thousands
> of employees) entire traffic,

FWIW, that's not massive by our definition. Other's on the list have
larger definitions of "massive" as well.

> but I guess the issue with the development
> code is not the amount of traffic, but both the quality of the code (without
> reading and testing the code I can't really refer to that) and the fact that
> the API is unstable.

Sure. We admittedly are more comfortable with our own "development"
code than that of others.

SSL is a weak point for me, to be honest, so I'm afraid I may not be
much help there - at least not in a timely manner.

--Brian

.

agentzh

Hello!

On Mon, Mar 18, 2013 at 2:07 AM,  aviram wrote:
>
> Thanks for tip, but how do you wake the parent request from
> ngx_http_lua_capture_body_filter?

By calling ngx_http_post_request. Just like how
ngx_http_finalize_request wakes up the parent request when finalizing
a subrequest. But you surely also need to update the state machine in
ngx_lua for the parent request to actually get ready to run.

> I guess that the data copy can be direct - copying the chain of the response
> of the subrequest to the chain of the response of the parent request. Is it
> okay?
>

Yes, the parent request and its subrequests share the same memory
pool. But you need to be careful about the non-buffered processing
rules in Nginx output filters. The upstream modules, for example, will
recycle the chain links in its own right.

> I'm thinking about adding two options to the capture interface -
> "max_body_size" and "truncate_on_max_body_size". If max_body_size exceeds,
> based on the value of truncate_on_max_body_size the request would either be
> truncated or passed through back to the client. The default would be true
> (which means that the request would be truncated). Do you think that's
> general enough?
>

Passing the subrequest body data directly to the downstream when
exceeding a threshold looks weird to me for the existing "capturing"
semantics of the ngx.location.capture API. Even truncating the body
data is a form of "capturing" because the exceeding part of the
subrequest response data are indeed "captured" and then discarded
away.

Maybe we can add separate options here, like "max_resp_body" and
"echo_on_exceeding_resp_body_limit". But as we can see, they must be
long enough to really make sense and I do not like long option names
:)

Best regards,
-agentzh

agentzh

Hello!

On Mon, Mar 18, 2013 at 10:06 AM, António P. P. Almeida wrote:
> Hmm, just to take advantage of the discussion about SSL. I guess agentzh has
> not yet found time to
> address the SSL part.

I've been trying (hard) to find some time for that :)

> Since SSL happens below HTTP there's no way to get
> around Nginx default SSL
> handling even if we use LuaJIT's FFI with OpenSSL or GnuTLS, or is it?
>

There're two directions here, upstream and downstream. Regarding the
downstream https handling, we surely cannot bypass Nginx's own stuffs
as long as we want ngx_lua to stay a well formed Nginx HTTP module.
But for the upstream https handling (I think that is what we are
talking about here in this thread), I think we do have much more
freedom :)

Best regards,
-agentzh