ngx.location.capture -> resty.http for internal/subrequest URLs?

igor.clark · 2016-02-02T09:45:59+00:00

Hello! On Mon, Feb 1, 2016 at 10:50 AM, Guilherme wrote: > I tried to do this in header_filter_by_lua but its not working properly. It > overwrites Ca...

ngx.location.capture -> resty.http for internal/subrequest URLs?

igor.clark

Hi there, hope all well!

We've discovered a few places in our app where ngx.location.capturedoesn't seem to be working so well for us, and so we've startedconverting the calls to use lua-resty-http instead. It works great, andwhen it's talking to external URLs, it makes perfect sense.

Our problem is that we've been using ngx.location.capture to makesubrequests to internal urls - e.g. /some/url - which was nice becauseit just passed through the request headers & body, and didn't need tothink about hostnames or anything environment-dependent. Withresty.http, right now I'm constructing the 'external' URL I need tocall, creating a new set of HTTP request headers and working out thenaming etc, and while it works well too, it's having to negotiate SSLbecause all of our URLs are on https, and our setup is quite heavilyname-based so just connecting to 127.0.01 on ngx.var.server_port won't work.

Is there a way to make resty.http call through 'internally' to a URL,like a ngx.location.capture subrequest? All the examples I could find inhttps://github.com/pintsized/lua-resty-http/tree/master/t seem to assumean external URL or 127.0.0.1 - I tried just setting 'path = /some/url'in the request_uri() params table, but I just get 'bad uri'.


Thanks very much :-)
Igor

igor.clark

Hi again! Of course, it turns out that our underlying problem was nothing at all to do with ngx.location.capture, but rather a header mismatch error in our code. Whoops. So, no need to move to resty.http for the time being. Given that I've seen a few references here to how that's going to be the preferred approach going forward, though, I wonder if that'll be the preference for external URLs only, or if it's for local URLs too, whether there might be some way to get access to subrequests without going up and down the stack, in the way ngx.location.capture does?

Cheers,

Igor

On Tuesday, February 2, 2016 at 1:46:01 AM UTC, Igor Clark wrote:

Hi there, hope all well!

We've discovered a few places in our app where ngx.location.capture
doesn't seem to be working so well for us, and so we've started
converting the calls to use lua-resty-http instead. It works great, and
when it's talking to external URLs, it makes perfect sense.

Our problem is that we've been using ngx.location.capture to make
subrequests to internal urls - e.g. /some/url - which was nice because
it just passed through the request headers & body, and didn't need to
think about hostnames or anything environment-dependent. With
resty.http, right now I'm constructing the 'external' URL I need to
call, creating a new set of HTTP request headers and working out the
naming etc, and while it works well too, it's having to negotiate SSL
because all of our URLs are on https, and our setup is quite heavily
name-based so just connecting to 127.0.01 on ngx.var.server_port won't work.

Is there a way to make resty.http call through 'internally' to a URL,
like a ngx.location.capture subrequest? All the examples I could find in
https://github.com/pintsized/lua-resty-http/tree/master/t seem to assume
an external URL or 127.0.0.1 - I tried just setting 'path = /some/url'
in the request_uri() params table, but I just get 'bad uri'.

Thanks very much :-)
Igor

agentzh

Hello!

On Wed, Feb 3, 2016 at 2:04 AM, Igor Clark wrote:
> I wonder if that'll be the
> preference for external URLs only, or if it's for local URLs too, whether
> there might be some way to get access to subrequests without going up and
> down the stack, in the way ngx.location.capture does?
>

I don't understand the "going up and down the stack" part.

The ngx.location.capture* API will be actively maintained in the
future. People should just use it for reusing existing NGINX C modules
(if there's no counterparts in the lua-resty-* world) and only for
relatively small subrequest responses (because ngx.location.capture*
always fully buffers the response).

Regards,
-agentzh

igor.clark

Hi agentzh, thanks, useful info.

Re "up and down", I might be getting the wrong end of the stick. What Imeant was that as far as I'd understood,ngx.location.capture("/local/url") fetches the results of /local/url viaan internal nginx subrequest, without making an actual HTTP call out viathe network or even via localhost, meaning less work overall - whereasif I call resty.http's request_uri("https://127.0.0.1/local/url";), ithas to construct and execute a complete HTTP request, including SSLnegotation in this case, which seems like it could be a fair bit morework for the program to do. So I was wondering whether, if that's right,it still makes sense to use resty.http in the case of fetching smallresponses as part of a larger script, or stick to ngx.location.capturefor this?


Hope that makes sense...
Cheers,
Igor

On 04/02/2016 00:26, Yichun Zhang (agentzh) wrote:

Hello!

On Wed, Feb 3, 2016 at 2:04 AM, Igor Clark wrote:

I wonder if that'll be the
preference for external URLs only, or if it's for local URLs too, whether
there might be some way to get access to subrequests without going up and
down the stack, in the way ngx.location.capture does?

I don't understand the "going up and down the stack" part.

The ngx.location.capture* API will be actively maintained in the
future. People should just use it for reusing existing NGINX C modules
(if there's no counterparts in the lua-resty-* world) and only for
relatively small subrequest responses (because ngx.location.capture*
always fully buffers the response).

Regards,
-agentzh

agentzh

Hello!

On Thu, Feb 4, 2016 at 9:33 AM, Igor Clark wrote:
> Re "up and down", I might be getting the wrong end of the stick. What I
> meant was that as far as I'd understood, ngx.location.capture("/local/url")
> fetches the results of /local/url via an internal nginx subrequest, without
> making an actual HTTP call out via the network or even via localhost,
> meaning less work overall - whereas if I call resty.http's
> request_uri("https://127.0.0.1/local/url";), it has to construct and execute
> a complete HTTP request, including SSL negotation in this case, which seems
> like it could be a fair bit more work for the program to do. So I was
> wondering whether, if that's right, it still makes sense to use resty.http
> in the case of fetching small responses as part of a larger script, or stick
> to ngx.location.capture for this?

Are you using subrequests to fetch local files on the local file
system? That's not really more efficient (actually often much less
efficient) than calling into the file syscalls directly from within
Lua (the only exception might be the use of AIO in the subrequest but
ngx.location.capture* do not support AIO mode yet).

If you are using proxy_pass or something similar in the subrequest,
then you still do network I/O in the subrequest anyway.

Eventually, I'd like to expose the NGINX file API via Lua, with AIO
support (both native AIO or OS thread pooling). I also welcome
patches.

Regards,
-agentzh

igor.clark

Hey agentzh, thanks again, more good info.

I'm not requesting local files via subrequests, only small responses from an internal-only PHP API, connecting to HHVM over unix sockets. We have a codebase in PHP with a bunch of library code that all works fine on the back end, and I'm gradually exposing parts of it to a client-side API, but with all the routing, auth & related logic implemented in OpenResty. This way I get to write all the new API code in Lua, keeping the separation quite clean, and also without re-writing PHP logic in Lua before I decide to migrate specific/complete sections. So e.g. \My\PHP\Library::function() does a bunch of work in PHP, which continues to be called by the server-side PHP code, and also by Lua where necessary via a thin internal layer as e.g. ngx.location.capture("/internal-api/my/php/library/function").

So yes, absolutely it's already doing some network I/O there, over unix sockets - I was just hoping to keep that to a minimum :-)

Anyway - ngx.location.capture* works fine for this now I sorted out my bugs 😁 so this is all good.

The AIO support sounds great, extremely useful. I'm afraid my C's not up to patching that right now but I'll be really happy to help test if & when it hits.

Thanks for the responses!

Igor

On Thursday, February 4, 2016 at 7:36:24 PM UTC, agentzh wrote:

Hello!

On Thu, Feb 4, 2016 at 9:33 AM, Igor Clark wrote:
> Re "up and down", I might be getting the wrong end of the stick. What I
> meant was that as far as I'd understood, ngx.location.capture("/local/url")
> fetches the results of /local/url via an internal nginx subrequest, without
> making an actual HTTP call out via the network or even via localhost,
> meaning less work overall - whereas if I call resty.http's
> request_uri("https://127.0.0.1/local/url"), it has to construct and execute
> a complete HTTP request, including SSL negotation in this case, which seems
> like it could be a fair bit more work for the program to do. So I was
> wondering whether, if that's right, it still makes sense to use resty.http
> in the case of fetching small responses as part of a larger script, or stick
> to ngx.location.capture for this?

Are you using subrequests to fetch local files on the local file
system? That's not really more efficient (actually often much less
efficient) than calling into the file syscalls directly from within
Lua (the only exception might be the use of AIO in the subrequest but
ngx.location.capture* do not support AIO mode yet).

If you are using proxy_pass or something similar in the subrequest,
then you still do network I/O in the subrequest anyway.

Eventually, I'd like to expose the NGINX file API via Lua, with AIO
support (both native AIO or OS thread pooling). I also welcome
patches.

Regards,
-agentzh

agentzh

Hello!

On Sun, Feb 7, 2016 at 12:23 PM, Igor Clark wrote:
> So yes, absolutely it's already doing some network I/O there, over unix sockets - I was just hoping to keep that to a minimum :-)
>

FWIW I've been also thinking about writing a lua-resty-fastcgi library
so that we can talk to FastCGI backends directly through cosockets,
which should be more efficient than the ngx.location.capture +
ngx_fastcgi combination, especially for large FastCGI responses.

Regards,
-agentzh

ben.agricola

Hi agentzh,

I had a use-case for this a couple of years ago and started writing an implementation of lua-resty-fastcgi - https://github.com/benagricola/lua-resty-fastcgi. It's incredibly light on documentation and tests, wasn't ever used in production and I haven't touched it in 2 years but maybe it's a good starting point? :)

Ben.

On Monday, 8 February 2016 19:11:53 UTC, agentzh wrote:

Hello!

On Sun, Feb 7, 2016 at 12:23 PM, Igor Clark wrote:
> So yes, absolutely it's already doing some network I/O there, over unix sockets - I was just hoping to keep that to a minimum :-)
>

FWIW I've been also thinking about writing a lua-resty-fastcgi library
so that we can talk to FastCGI backends directly through cosockets,
which should be more efficient than the ngx.location.capture +
ngx_fastcgi combination, especially for large FastCGI responses.

Regards,
-agentzh

agentzh

Hi Ben

On Tue, Feb 9, 2016 at 8:14 AM, Ben Agricola wrote:
> I had a use-case for this a couple of years ago and started writing an
> implementation of lua-resty-fastcgi -
> https://github.com/benagricola/lua-resty-fastcgi. It's incredibly light on
> documentation and tests, wasn't ever used in production and I haven't
> touched it in 2 years but maybe it's a good starting point? :)
>

Interesting. Thanks for sharing :)

Best regards,
-agentzh

igor.clark

FWIW, a lua-resty-fastcgi implementation that was better than ngx.location.capture*->ngx_fastcgi would be pretty great for my case. Not just because of performance but because of a lot less nginx config, passing ngx.var.* data back to nginx after a rewrite_by_lua_*, etc. Again I'd be super happy to help test this out, were it to come to pass.

Cheers :-)
Igor

On Tuesday, February 9, 2016 at 8:14:42 PM UTC, agentzh wrote:

Hi Ben

On Tue, Feb 9, 2016 at 8:14 AM, Ben Agricola wrote:
> I had a use-case for this a couple of years ago and started writing an
> implementation of lua-resty-fastcgi -
> https://github.com/benagricola/lua-resty-fastcgi. It's incredibly light on
> documentation and tests, wasn't ever used in production and I haven't
> touched it in 2 years but maybe it's a good starting point? :)
>

Interesting. Thanks for sharing :)

Best regards,
-agentzh

ben.agricola

On Tuesday, 9 February 2016 20:14:42 UTC, agentzh wrote:

Hi Ben

On Tue, Feb 9, 2016 at 8:14 AM, Ben Agricola wrote:
> I had a use-case for this a couple of years ago and started writing an
> implementation of lua-resty-fastcgi -
> https://github.com/benagricola/lua-resty-fastcgi. It's incredibly light on
> documentation and tests, wasn't ever used in production and I haven't
> touched it in 2 years but maybe it's a good starting point? :)
>

Interesting. Thanks for sharing :)

Maybe it's useful - if people think this is actually viable and a good base then I'm happy to pick it back up again, although the internal use-case hasn't changed much so there's nothing really to test it on.

I had a play-around to refresh my memory on this getting it working with PHP-FPM again. I seem to remember it makes some assumptions (like always sending the request ID as '1' because php-fpm blindly returns the request ID 1 for everything) - my basic testing works (although the test suite actually fails depending on which order the fastcgi headers are parsed in, so there's some thought required to make the tests work regardless of parse order).

I seem to remember we had some issues with chunked POST request bodies so that is likely an area that'll require further testing / work too, but I've pushed some local commits I had from ages ago into the master branch of the repo above.

@Igor: On the config front, it's neat-ish but still requires some boilerplate code dependent on the App or FCGI program that you're passing the request to. The example I tested with this morning looks like this (obviously not applicable to another PHP app but gives an idea of the config method): https://gist.github.com/benagricola/75a4b3a23a564575a787

Ben.

igor.clark

Hi Ben, hope all's well! Sorry I never replied to this, I don't think I really digested the last part.

I just had reason to think of it again though, as I'm repeatedly having problems with ngx.location.capture()'ing FastCGI->PHP resources when using HTTP/2. So I came back and found this and noticed the gist and the config.

That amount of boilerplate seems fine to me, easily bundle-up-able in a named location or just an include file.

I'll try and take a look at getting in working in the next couple weeks and will let you know how I get on, if you're interested.

Is there anything else I should know about it?

Thanks!

Best,

Igor

On 10/02/2016 12:05, Ben Agricola wrote:

On Tuesday, 9 February 2016 20:14:42 UTC, agentzh wrote:

Hi Ben

On Tue, Feb 9, 2016 at 8:14 AM, Ben Agricola wrote:
> I had a use-case for this a couple of years ago and started writing an
> implementation of lua-resty-fastcgi -
> https://github.com/benagricola/lua-resty-fastcgi. It's incredibly light on
> documentation and tests, wasn't ever used in production and I haven't
> touched it in 2 years but maybe it's a good starting point? :)
>

Interesting. Thanks for sharing :)

Maybe it's useful - if people think this is actually viable and a good base then I'm happy to pick it back up again, although the internal use-case hasn't changed much so there's nothing really to test it on.

I had a play-around to refresh my memory on this getting it working with PHP-FPM again. I seem to remember it makes some assumptions (like always sending the request ID as '1' because php-fpm blindly returns the request ID 1 for everything) - my basic testing works (although the test suite actually fails depending on which order the fastcgi headers are parsed in, so there's some thought required to make the tests work regardless of parse order).

I seem to remember we had some issues with chunked POST request bodies so that is likely an area that'll require further testing / work too, but I've pushed some local commits I had from ages ago into the master branch of the repo above.

@Igor: On the config front, it's neat-ish but still requires some boilerplate code dependent on the App or FCGI program that you're passing the request to. The example I tested with this morning looks like this (obviously not applicable to another PHP app but gives an idea of the config method): https://gist.github.com/benagricola/75a4b3a23a564575a787

Ben.

.

igor.clark

Hey Ben - FYI, I started to try this out, but as it doesn't currently seem to support unix domain sockets I've put it on hold for now. All our system setup relies on talking to HHVM over unix sockets, and I don't have the time to swap that round (or get into ngx.socket.udp!) for now. Another time!

Cheers,
Igor

On 30/07/2016 19:27, Igor Clark wrote:

Hi Ben, hope all's well! Sorry I never replied to this, I don't think I really digested the last part.

I just had reason to think of it again though, as I'm repeatedly having problems with ngx.location.capture()'ing FastCGI->PHP resources when using HTTP/2. So I came back and found this and noticed the gist and the config.

That amount of boilerplate seems fine to me, easily bundle-up-able in a named location or just an include file.

I'll try and take a look at getting in working in the next couple weeks and will let you know how I get on, if you're interested.

Is there anything else I should know about it?

Thanks!

Best,

Igor

On 10/02/2016 12:05, Ben Agricola wrote:

On Tuesday, 9 February 2016 20:14:42 UTC, agentzh wrote:

Hi Ben

On Tue, Feb 9, 2016 at 8:14 AM, Ben Agricola wrote:
> I had a use-case for this a couple of years ago and started writing an
> implementation of lua-resty-fastcgi -
> https://github.com/benagricola/lua-resty-fastcgi. It's incredibly light on
> documentation and tests, wasn't ever used in production and I haven't
> touched it in 2 years but maybe it's a good starting point? :)
>

Interesting. Thanks for sharing :)

Maybe it's useful - if people think this is actually viable and a good base then I'm happy to pick it back up again, although the internal use-case hasn't changed much so there's nothing really to test it on.

I had a play-around to refresh my memory on this getting it working with PHP-FPM again. I seem to remember it makes some assumptions (like always sending the request ID as '1' because php-fpm blindly returns the request ID 1 for everything) - my basic testing works (although the test suite actually fails depending on which order the fastcgi headers are parsed in, so there's some thought required to make the tests work regardless of parse order).

I seem to remember we had some issues with chunked POST request bodies so that is likely an area that'll require further testing / work too, but I've pushed some local commits I had from ages ago into the master branch of the repo above.

@Igor: On the config front, it's neat-ish but still requires some boilerplate code dependent on the App or FCGI program that you're passing the request to. The example I tested with this morning looks like this (obviously not applicable to another PHP app but gives an idea of the config method): https://gist.github.com/benagricola/75a4b3a23a564575a787

Ben.

.