Subrequest in init_worker with timer

bogdan · 2015-02-25T19:00:52+00:00

Hello! On Tue, Feb 24, 2015 at 1:33 PM, Abdul Hakeem wrote: > How does Openresty performance compared to the 1m+ concurrent connections & > multi-...

Subrequest in init_worker with timer

bogdan

Hello, all

I need to make an HTTP request from the "ngx.timer.at" context, started from "init_worker_by_lua_file"
I tried to configure an internal proxy location and to call it with "ngx.location.capture" but I get the error "API disabled in the context of ngx.timer"
How can I make an HTTP request from "ngx.timer.at", in "init_worker_by_lua_file" ? I would prefer not using yet another lua module.

Thank you

Bogdan

agentzh

Hello!

On Wed, Feb 25, 2015 at 3:00 AM, Bogdan Irimia wrote:
> I need to make an HTTP request from the "ngx.timer.at" context, started from
> "init_worker_by_lua_file"
> I tried to configure an internal proxy location and to call it with
> "ngx.location.capture" but I get the error "API disabled in the context of
> ngx.timer"
> How can I make an HTTP request from "ngx.timer.at", in
> "init_worker_by_lua_file" ?

The timer context does not support subrequests because subrequests
require *real* requests. And there are no real requests associated
with any timer created by ngx.timer.at(), for obvious reasons.

You need to use a lua-resty-http library based on the cosocket API for
such requirements.

Regards,
-agentzh

bogdan

I see. It makes sense now :)
Which would be the lightest cosocket http library you would recommend?

Thank you

Yichun Zhang (agentzh) wrote:

Hello!

On Wed, Feb 25, 2015 at 3:00 AM, Bogdan Irimia wrote:

I need to make an HTTP request from the "ngx.timer.at" context, started from
"init_worker_by_lua_file"
I tried to configure an internal proxy location and to call it with
"ngx.location.capture" but I get the error "API disabled in the context of
ngx.timer"
How can I make an HTTP request from "ngx.timer.at", in
"init_worker_by_lua_file" ?

The timer context does not support subrequests because subrequests
require *real* requests. And there are no real requests associated
with any timer created by ngx.timer.at(), for obvious reasons.

You need to use a lua-resty-http library based on the cosocket API for
such requirements.

Regards,
-agentzh

agentzh

Hello!

On Wed, Feb 25, 2015 at 2:22 PM, Bogdan Irimia wrote:
> I see. It makes sense now :)
> Which would be the lightest cosocket http library you would recommend?
>

Personally I like Brian Akins's lua-resty-http-simple library the most:

    https://github.com/bakins/lua-resty-http-simple

while James Hurst's lua-resty-http has a lot more features like
streaming processing (which even ngx.location.capture + ngx_proxy
lacks):

    https://github.com/pintsized/lua-resty-http

Well, just my personal preference. This should not be taken as
OpenResty's official recommendations :) I'd like the community to make
the decision on 3rd-party libraries.

Regards,
-agentzh

bogdan

Great!
I think we'll go with lua-resty-http-simple because we don't need "exotic" features for now, only basic http requests.
Nevertheless, I will take a look at the alternatives.

Thank you!

Yichun Zhang (agentzh) wrote:

Hello!

On Wed, Feb 25, 2015 at 2:22 PM, Bogdan Irimia wrote:

I see. It makes sense now :)
Which would be the lightest cosocket http library you would recommend?

Personally I like Brian Akins's lua-resty-http-simple library the most:

https://github.com/bakins/lua-resty-http-simple

while James Hurst's lua-resty-http has a lot more features like
streaming processing (which even ngx.location.capture + ngx_proxy
lacks):

https://github.com/pintsized/lua-re sty-http

Well, just my personal preference. This should not be taken as
OpenResty's official recommendations :) I'd like the community to make
the decision on 3rd-party libraries.

Regards,
-agentzh

james_9172

On 25 February 2015 at 22:26, Yichun Zhang (agentzh) <age...@gmail.com> wrote:

Hello!

On Wed, Feb 25, 2015 at 2:22 PM, Bogdan Irimia wrote:
> I see. It makes sense now :)
> Which would be the lightest cosocket http library you would recommend?
>

Personally I like Brian Akins's lua-resty-http-simple library the most:

https://github.com/bakins/lua-resty-http-simple

while James Hurst's lua-resty-http has a lot more features like
streaming processing (which even ngx.location.capture + ngx_proxy
lacks):

https://github.com/pintsized/lua-resty-http

Well, just my personal preference. This should not be taken as
OpenResty's official recommendations :) I'd like the community to make
the decision on 3rd-party libraries.

As an aside, my module was originally based on Brian's excellent work, and I kept his "simple" interface as I could see people would need that. I *added* in a more generic interface which supports streaming, as well as SSL, pipelining, trailers and other corner cases like handling "100 Continue".

The main reason to choose my module would be when your requests / responses are less "known". Proxying the client request body for example (which could be huge), or returning a potentially large response, or handling "Expect: 100-continue" etc. It aims to be a complete HTTP client, and thus handles a greater number of corner cases.

If your requests / responses are known and present no risk to your application, then you may not need streaming.

This said, and no disrespect to Brian (as I said, I based my module on his!), I don't really see much gain in having multiple modules for the same thing, and I'm not sure why one might choose a less complete HTTP client over a more complete one?

If performance is a concern I'm really up for addressing any issues where possible - the aim was always to make something generic for the community, and indeed it includes features I've never needed myself! Any feedback / criticism is most welcome.

Regards,

James Hurst

ja...@pintsized.co.uk

aapo.talvensaari

On Thursday, 26 February 2015 12:27:56 UTC+2, James Hurst wrote:

This said, and no disrespect to Brian (as I said, I based my module on his!), I don't really see much gain in having multiple modules for the same thing, and I'm not sure why one might choose a less complete HTTP client over a more complete one?

Yes, when I was constructing this list:

https://github.com/bungle/awesome-resty/#networking

I was a quite confusing to find so many HTTP libs, and three of them with the same name:

lua-resty-http by @pintsized (last update 3 months ago)

lua-resty-http by @liseen (last updated 8 days ago)

lua-resty-http by @DprianGray (last updated 1 year ago, abandoned fork, maybe a pull-request fork?)

Then we have proxy prebuilt in Nginx:

https://groups.google.com/d/msg/openresty-en/6OwhJioFVoo/LTxMumhbFIUJ

Plus alternatives in the list above. At least the naming issue should be solved at some point.

james_9172

Yes, when I was constructing this list:
https://github.com/bungle/awesome-resty/#networking

I was a quite confusing to find so many HTTP libs, and three of them with the same name:

lua-resty-http by @pintsized (last update 3 months ago)
lua-resty-http by @liseen (last updated 8 days ago)
lua-resty-http by @DprianGray (last updated 1 year ago, abandoned fork, maybe a pull-request fork?)

Yes, I'm pretty sure the @DorianGray one is a pull-request fork from the @pintsized module - I'm not aware that it contains additional features / improvements. Great work compiling that list though BTW, I found it really useful!

AFAIK, the @liseen module predates mine and Brian's, and seems quite widely used, but lacks key features like chunked encoding, and wasn't written in a way that made it easy for me to extend, which is why I started with Brian's "simple" version instead.

Some of this confusion simply arises from HTTP being quite a cumbersome protocol in the wild. There are lots of dusty corners. It's almost like there are two distinct use-cases for such a library in our eco-system; one which is used to perform REST-like requests, perhaps for some JSON configuration and things of that nature; and one which is more directly related to the current client request / response generation, perhaps acting as a proxy for example.

The latter case is way more complex, because we must deal with whatever the client asks of us, and the same variability can be true from a backend. The lack of streaming in this scenario, may work fine in proof-of-concept development, but leaves us wide open to vulnerabilities in the wild.

But even given this distinction, I don't see why we can't create one module that deals with simple cases and the various corner cases - that's what I've been trying to do at least ;)

James Hurst

ja...@pintsized.co.uk

dsgnvictor

Hello!

On Wednesday, February 25, 2015 at 11:26:54 PM UTC+1, agentzh wrote:

Personally I like Brian Akins's lua-resty-http-simple library the most:
https://github.com/bakins/lua-resty-http-simple

lua-resty-http-simple is great but could return a table from res.headers["Set-Cookie"] instead of concatenated string.

ngx.header["Set-Cookie"] = a_cookies_table works but ngx.header["Set-Cookie"] = all_cookies_joined_with_; doesn't

Well, it's not so hard to modify the http.simple.lua to get proper results :-)

Anyways, thank you for AWESOME Openresty !

best,

Victor