Re: Chuckin and POSTs

agentzh · 2012-10-02T07:10:28+00:00

On Monday, October 1, 2012 3:10:27 PM UTC-7, agentzh wrote:Hi, guys! I've been hacking on the "thread" branch of the ngx_lua module in the last few weeks:...

Re: Chuckin and POSTs

agentzh

Hello!

On Mon, Oct 1, 2012 at 1:55 PM, Mike Zeller wrote:
> We are using your chuckin module for nginx.  It works great

Good to know :)

> however I had a
> couple of questions about how it works.
> If I understand it correctly it is buffering chunked encoding messages until
> it knows the content length then passing it along?

Yes, just like the standard request body reader in nginx. Request
bodies are always fully buffered in Nginx (either in memory or on
disk) unless you're using the ngx.req.socket() API provided by our
ngx_lua module:

    http://wiki.nginx.org/HttpLuaModule#ngx.req.socket

> Is it possible to have it just pass the chunk to the backend directly
> without buffering it up?

No.

>  Currently nginx works great minus the
> fact that we cant send chunks to the backend (using proxy_pass) in real
> time.

Even the standard Nginx request body reader cannot do that. You'll
need ngx_lua's ngx.req.socket() for that, as mentioned above.

>  Or maybe you know if Igor is adding true chunked
> encoding to the core of nginx?
>

Nope :)

BTW, I'm cc'ing the openresty-en mailing list so that other users may
benefit from our discussion here:

    http://groups.google.com/group/openresty-en

Best regards,
-agentzh

agentzh

Hello!

On Mon, Oct 1, 2012 at 9:51 PM, Mike Zeller wrote:
> Awesome.  So I managed to compile nginx with the lua support and I have been
> looking at the documentation for ngx.req.socket.  How would I go about using
> that to pass incoming connections to a node.js application listening on the
> same box?

Just use ngx.req.socket + ngx.socket.tcp. That is, reading one data
chunk from the downstream via the former and then send this chunk to
the upstream via the latter. Everything here is of 100% nonblocking
I/O.

The only caveat here is that ngx.req.socket does not understand
chunked encoding automatically. You need to patch the Nginx core to
prevent it from croaking on chunked request bodies and parse the
chunked body in Lua  yourself :)

If no chunked request bodies are involved, then everything should work
out of the box just fine.

See

    http://wiki.nginx.org/HttpLuaModule#ngx.socket.tcp

for more details.

>  Not sure if what I want to do is possible.  Basically what I want
> to do is have:
>
> incoming data (SSL) -> nginx (remove SSL) -> node.js application.
>
> The important part is that the data gets to node in real time and not
> buffered like the chunkin module.  It seems like the examples for mysql and
> other applications allow ngxin/lua to pass things off in real time in a non
> blocking fashion.
>

Well, you can access mysql/redis/memcached/etc directly from within
ngx_lua in a 100% nonblocking manner. The Lua libraries
lua-resty-mysql, lua-resty-redis, and lua-resty-memcached are all
written atop the nonblocking ngx.socket.tcp API (aka the cosocket API)
that was mentioned above.

See https://github.com/agentzh/lua-resty-mysql for example.

Best regards,
-agentzh

mike.zeller

Awesome. So I managed to compile nginx with the lua support and I have been looking at the documentation for ngx.req.socket. How would I go about using that to pass incoming connections to a node.js application listening on the same box? Not sure if what I want to do is possible. Basically what I want to do is have:

incoming data (SSL) -> nginx (remove SSL) -> node.js application.

The important part is that the data gets to node in real time and not buffered like the chunkin module. It seems like the examples for mysql and other applications allow ngxin/lua to pass things off in real time in a non blocking fashion.

-Mike

On Mon, Oct 1, 2012 at 11:10 PM, agentzh <age...@gmail.com> wrote:

Hello!

On Mon, Oct 1, 2012 at 1:55 PM, Mike Zeller wrote:
> We are using your chuckin module for nginx. It works great

Good to know :)

> however I had a
> couple of questions about how it works.
> If I understand it correctly it is buffering chunked encoding messages until
> it knows the content length then passing it along?

Yes, just like the standard request body reader in nginx. Request
bodies are always fully buffered in Nginx (either in memory or on
disk) unless you're using the ngx.req.socket() API provided by our
ngx_lua module:

http://wiki.nginx.org/HttpLuaModule#ngx.req.socket

> Is it possible to have it just pass the chunk to the backend directly
> without buffering it up?

No.

> Currently nginx works great minus the
> fact that we cant send chunks to the backend (using proxy_pass) in real
> time.

Even the standard Nginx request body reader cannot do that. You'll
need ngx_lua's ngx.req.socket() for that, as mentioned above.

> Or maybe you know if Igor is adding true chunked
> encoding to the core of nginx?
>

Nope :)

BTW, I'm cc'ing the openresty-en mailing list so that other users may
benefit from our discussion here:

http://groups.google.com/group/openresty-en

Best regards,
-agentzh

agentzh

Hello!

On Tue, Oct 2, 2012 at 2:05 PM, Dave Eddy wrote:
> However, i realized my mistake instantly, that the data provided by
> ngx.req.socket():receive is just the POST data, and not all of the headers
> (which makes sense).  I'm thinking I will need to add in logic to
> reconstruct the headers before forwarding to Node.js, adding in
> X-Forwarded-For, etc.
>

Yes. Check out the ngx.req.get_headers() API provided by ngx_lua:

    http://wiki.nginx.org/HttpLuaModule#ngx.req.get_headers

Also, you'll need to receive the TCP response from your upstream
servers (i.e., your nodejs app in this case) and forward the data
chunk to the downstream. (Check out the ngx.print and ngx.flush
functions provided by ngx_lua.)

Essentially, you're just coding up a nonbuffered HTTP proxy in pure
Lua running inside Nginx ;)

> Is this the right approach to tackle this problem? or am I thinking about
> this wrong.
>

This approach should work. To maximize the performance, you may want
to consider migrating all your nodejs app code right over to ngx_lua
(if possible) ;)

Best regards,
-agentzh

dave.eddy

Hello agentzh!

thanks for your quick replies and your awesome work for nginx!

Ideally, we would keep all of the body parsing logic outside of lua, and just pass it directly to our node.js app. I looked through the documentation, and was able to get a broken prototype to accomplish this goal.

The goal here is to have the POST data stream directly to our backend service (a tcp socket) asap, rather than buffering it all. here's the code thus far: (attached as opposed to pastebin for the benefit of the mailing list)

location / {

content_by_lua '

-- connect to node

local node_sock, err = ngx.socket.connect("127.0.0.1", 8080)

if err then

ngx.say("failed to connect to node: ", err)

return

end

-- get the req.socket

local req_sock, err = ngx.req.socket()

if not req_sock then

ngx.say("failed to get to req.socket: ", err)

return

end

-- send the request from nginx to node

while true do

local line, err, partial = req_sock:receive();

if err then

if partial then

node_sock:send(partial)

end

break;

end

node_sock:send(line)

end

-- all the data has been sent to node, now there must

-- be a while loop here to recv() from node and send

-- back out through nginx

node_sock:close()

}

However, i realized my mistake instantly, that the data provided by ngx.req.socket():receive is just the POST data, and not all of the headers (which makes sense). I'm thinking I will need to add in logic to reconstruct the headers before forwarding to Node.js, adding in X-Forwarded-For, etc.

Is this the right approach to tackle this problem? or am I thinking about this wrong.

Thanks,

dave

On Mon, Oct 1, 2012 at 10:43 PM, agentzh <age...@gmail.com> wrote:

Hello!

On Mon, Oct 1, 2012 at 9:51 PM, Mike Zeller wrote:
> Awesome. So I managed to compile nginx with the lua support and I have been
> looking at the documentation for ngx.req.socket. How would I go about using
> that to pass incoming connections to a node.js application listening on the
> same box?

Just use ngx.req.socket + ngx.socket.tcp. That is, reading one data
chunk from the downstream via the former and then send this chunk to
the upstream via the latter. Everything here is of 100% nonblocking
I/O.

The only caveat here is that ngx.req.socket does not understand
chunked encoding automatically. You need to patch the Nginx core to
prevent it from croaking on chunked request bodies and parse the
chunked body in Lua yourself :)

If no chunked request bodies are involved, then everything should work
out of the box just fine.

See

http://wiki.nginx.org/HttpLuaModule#ngx.socket.tcp

for more details.

> Not sure if what I want to do is possible. Basically what I want
> to do is have:
>
> incoming data (SSL) -> nginx (remove SSL) -> node.js application.
>
> The important part is that the data gets to node in real time and not
> buffered like the chunkin module. It seems like the examples for mysql and
> other applications allow ngxin/lua to pass things off in real time in a non
> blocking fashion.
>

Well, you can access mysql/redis/memcached/etc directly from within
ngx_lua in a 100% nonblocking manner. The Lua libraries
lua-resty-mysql, lua-resty-redis, and lua-resty-memcached are all
written atop the nonblocking ngx.socket.tcp API (aka the cosocket API)
that was mentioned above.

See https://github.com/agentzh/lua-resty-mysql for example.

Best regards,
-agentzh

dave.eddy

Hey,

So I have it about 80% of the way there (thanks for all your help!). I foreach over the headers and send them upstream to our node.js app, however, i'm curious if a header is repeated (ie Set-Cookie) how to check for such a case.

Also, when the Node.js app responds, because it is a TCP socket, it sends all the normal HTTP headers with it. however, ngx.print() will attempt to send headers for me. Is there a way to write to the ngx socket (ie ngx.say(), ngx.print()) without it first sending headers on my behalf? Ideally, i would like ngx.print to stream the data as it comes in from the node.js app, without trying to do any HTTP parsing logic.

Thanks again,

dave

ps: here is the content_by_lua portion I have currently

-- connect to node

local node_sock, err = ngx.socket.connect("127.0.0.1", 8080)

if err then

ngx.say("failed to connect to node: ", err)

return

end

-- get the req.socket

local req_sock, err = ngx.req.socket()

if not req_sock then

ngx.say("failed to connect to req.socket: ", err)

return

end

-- send the request to node

local method = ngx.req.get_method()

node_sock:send(method .. " " .. ngx.var.uri .. " " .. "HTTP/1.1\\r\\n")

-- headers

local h = ngx.req.get_headers()

for k, v in pairs(h) do

node_sock:send(k .. ": " .. v .. "\\r\\n")

end

node_sock:send("X-Forwarded-For: nginx\\r\\n")

-- body

if method == "POST" or method == "PUT" then

while true do

local line, err, partial = req_sock:receive();

if err then

if partial then

node_sock:send(partial)

end

break;

end

node_sock:send(line);

end

node_sock:send("\\r\\n")

end

node_sock:send("\\r\\n")

-- receive the response

while true do

local line, err, partial = node_sock:receive();

if err then

if partial then

ngx.print(partial)

ngx.flush(true)

end

break;

end

ngx.print(line)

ngx.flush(true)

end

-- close the node socket (??)

ngx.req.finish_body()

node_sock:close()

On Tue, Oct 2, 2012 at 2:34 PM, agentzh <age...@gmail.com> wrote:

Hello!

On Tue, Oct 2, 2012 at 2:05 PM, Dave Eddy wrote:
> However, i realized my mistake instantly, that the data provided by
> ngx.req.socket():receive is just the POST data, and not all of the headers
> (which makes sense). I'm thinking I will need to add in logic to
> reconstruct the headers before forwarding to Node.js, adding in
> X-Forwarded-For, etc.
>

Yes. Check out the ngx.req.get_headers() API provided by ngx_lua:

http://wiki.nginx.org/HttpLuaModule#ngx.req.get_headers

Also, you'll need to receive the TCP response from your upstream
servers (i.e., your nodejs app in this case) and forward the data
chunk to the downstream. (Check out the ngx.print and ngx.flush
functions provided by ngx_lua.)

Essentially, you're just coding up a nonbuffered HTTP proxy in pure
Lua running inside Nginx ;)

> Is this the right approach to tackle this problem? or am I thinking about
> this wrong.
>

This approach should work. To maximize the performance, you may want
to consider migrating all your nodejs app code right over to ngx_lua
(if possible) ;)

Best regards,
-agentzh

agentzh

Hello!

On Wed, Oct 3, 2012 at 1:10 PM, Dave Eddy wrote:
> Hey,
>
> So I have it about 80% of the way there (thanks for all your help!).  I
> foreach over the headers and send them upstream to our node.js app, however,
> i'm curious if a header is repeated (ie Set-Cookie) how to check for such a
> case.
>

When you have multiple response headers with the same name, just Lua
array-like tables to assign it. For example,

    ngx.header['Set-Cookie'] = {'a=32; path=/', 'b=4; path=/'}

See the documentation for ngx.header.HEADER for details:

    http://wiki.nginx.org/HttpLuaModule#ngx.header.HEADER

> Also, when the Node.js app responds, because it is a TCP socket, it sends
> all the normal HTTP headers with it. however, ngx.print() will attempt to
> send headers for me.  Is there a way to write to the ngx socket (ie
> ngx.say(), ngx.print()) without it first sending headers on my behalf?

No, you should always parse the HTTP response from your backend
service, and set the response headers via the ngx.header.HEADER API
yourself. Otherwise you'll never know the end of the data stream and
you won't be able to support connection pooling (and no HTTP 1.1
keepalive) for your backend connections.

> Ideally, i would like ngx.print to stream the data as it comes in from the
> node.js app, without trying to do any HTTP parsing logic.
>

No, it's not good at all. HTTP 1.1 keepalive support will then no
longer be possible.

Best regards,
-agentzh