ngx_proxy, balancer_by_lua and http keepalives

andreaslubbe · 2016-05-14T16:46:06+00:00

I found the issue. It checks for "ipv6=" token in #if (NGX_HAVE_INET6) block. so if NGX_HAVE_INET6 is not enabled, it treats it as...

ngx_proxy, balancer_by_lua and http keepalives

andreaslubbe

(Porting https://github.com/openresty/openresty/issues/176)

Original Question (edited for http keepalive):

Assuming that the domains in urls resolve to the ones in ips, what are the differences between these two configurations? Does the first one use any http keepalives? Does the second one use 100 keepalives per IP (totalling a maximum 300 keepalives) or 100 in total (giving about 30-40 for each IP, assuming a uniform distribution of requests)? Anything else that's different?

location / {
    set $proxy_pass_loc '';
    content_by_lua_block {
        local urls = ['http://example.com', 'http://api.example.com', 'http://www.example.com']
        ngx.var.proxy_pass_loc = urls[math.random(3)]  .. ngx.var.request_uri
        ngx.exec "/foo"
    }
}
location /foo {
    internal;
    proxy_pass $proxy_pass_loc;
}

upstream foo {
    server 0.0.0.1;
    balancer_by_lua_block {
        local ips = ['93.184.216.34', '93.184.216.35', '93.184.216.36']
        require "balancer".set_current_peer(ips[math.random(3)], 80)
     }
    keepalive 100;
}

server {
    location / {
        proxy_pass http://foo;
     }
 }

Original answer:

your first example does not involve any connection pool for your backend HTTP connections since there is no corresponding upstream {} configuration blocks and thus no "keepalive" configurations.

In the 2nd example, all your IPs share the same connection pool with the size limit of 100. But only connections with *matching addresses" can get reused. This means that if your ask for a peer for 92.184.216.34, then the in-pool connections with the address 93.184.216.35 can never get picked up for it.

andreaslubbe

So in the first example, every single request will go through a 3-way tcp handshake to send the request and through a 2-way tcp close after receiving the response, correct? Meaning that this approached should be avoided if possible?

For the 2nd example, how long are these keepalives cached/preserved?

Also, just to make sure I fully understand what's going on (having read https://www.nginx.com/blog/load-balancing-with-nginx-plus-part2/#keepalive), let's pretend for a second the above keepalive limit was 5 and there are 7 simultaneous requests coming in . Here is what I currently believe happens:

#req randomly chosen IP keepalive pool id

#1 93.184.216.34 --, but gets cached afterwards with id 1
#2 93.184.216.35 --, but gets cached afterwards with id 2
#3 93.184.216.36 --, but gets cached afterwards with id 3
#4 93.184.216.34 --, but gets cached afterwards with id 4
#5 93.184.216.35 --, but gets cached afterwards with id 5
#6 93.184.216.36 --, connection gets closed afterwards
#7 93.184.216.34 --, connection gets closed afterwards

Then, after all of those requests are done, another set of 7 simultaneous requests come in. Again, here is what I currently believe happens:

#req randomly chosen IP keepalive pool id

#1 93.184.216.34 Reuses id 1 (or 4)
#2 93.184.216.35 Reuses id 2 (or 5)
#3 93.184.216.36 Reuses id 3
#4 93.184.216.34 Reuses id 4 (or 1)
#5 93.184.216.35 Reuses id 5 (or 2)
#6 93.184.216.36 --, connection gets closed afterwards
#7 93.184.216.34 --, connection gets closed afterwards

Is that about right?

On Saturday, May 14, 2016 at 10:46:06 AM UTC+2, AL wrote:

(Porting https://github.com/openresty/openresty/issues/176)

Original Question (edited for http keepalive):
Assuming that the domains in urls resolve to the ones in ips, what are the differences between these two configurations? Does the first one use any http keepalives? Does the second one use 100 keepalives per IP (totalling a maximum 300 keepalives) or 100 in total (giving about 30-40 for each IP, assuming a uniform distribution of requests)? Anything else that's different?
location / {
    set $proxy_pass_loc '';
    content_by_lua_block {
        local urls = ['http://example.com', 'http://api.example.com', 'http://www.example.com']
        ngx.var.proxy_pass_loc = urls[math.random(3)]  .. ngx.var.request_uri
        ngx.exec "/foo"
    }
}
location /foo {
    internal;
    proxy_pass $proxy_pass_loc;
}
upstream foo {
    server 0.0.0.1;
    balancer_by_lua_block {
        local ips = ['93.184.216.34', '93.184.216.35', '93.184.216.36']
        require "balancer".set_current_peer(ips[math.random(3)], 80)
     }
    keepalive 100;
}

server {
    location / {
        proxy_pass http://foo;
     }
 }
Original answer:
your first example does not involve any connection pool for your backend HTTP connections since there is no corresponding upstream {} configuration blocks and thus no "keepalive" configurations.
In the 2nd example, all your IPs share the same connection pool with the size limit of 100. But only connections with *matching addresses" can get reused. This means that if your ask for a peer for 92.184.216.34, then the in-pool connections with the address 93.184.216.35 can never get picked up for it.

agentzh

Hello!

On Sat, May 14, 2016 at 2:08 AM, AL wrote:
> So in the first example, every single request will go through a 3-way tcp
> handshake to send the request and through a 2-way tcp close after receiving
> the response, correct?

Yes.

> Meaning that this approached should be avoided if
> possible?
>

Yes.

> For the 2nd example, how long are these keepalives cached/preserved?

As long as it can. Usually up to the remote keepalive timeout settings
and keepalive max requests settings.

> Also, just to make sure I fully understand what's going on (having read
> https://www.nginx.com/blog/load-balancing-with-nginx-plus-part2/#keepalive),
> let's pretend for a second the above keepalive limit was 5 and  there are 7
> simultaneous requests coming in . Here is what I currently believe happens:
> #req  randomly chosen IP  keepalive pool id
> #1     93.184.216.34          --, but gets cached afterwards with id 1
> #2     93.184.216.35          --, but gets cached afterwards with id 2
> #3     93.184.216.36          --, but gets cached afterwards with id 3
> #4     93.184.216.34          --, but gets cached afterwards with id 4
> #5     93.184.216.35          --, but gets cached afterwards with id 5
> #6     93.184.216.36          --, connection gets closed afterwards
> #7     93.184.216.34          --, connection gets closed afterwards
>
> Then, after all of those requests are done, another set of 7 simultaneous
> requests come in. Again, here is what I currently believe happens:
> #req  randomly chosen IP  keepalive pool id
> #1     93.184.216.34          Reuses id 1 (or 4)
> #2     93.184.216.35          Reuses id 2 (or 5)
> #3     93.184.216.36          Reuses id 3
> #4     93.184.216.34          Reuses id 4 (or 1)
> #5     93.184.216.35          Reuses id 5 (or 2)
> #6     93.184.216.36          --, connection gets closed afterwards
> #7     93.184.216.34          --, connection gets closed afterwards
>
> Is that about right?

Yeah, something like that.

Regards,
-agentzh

andreaslubbe

Thanks!

On Wednesday, May 18, 2016 at 6:37:44 AM UTC+2, agentzh wrote:

Hello!

On Sat, May 14, 2016 at 2:08 AM, AL wrote:
> So in the first example, every single request will go through a 3-way tcp
> handshake to send the request and through a 2-way tcp close after receiving
> the response, correct?

Yes.

> Meaning that this approached should be avoided if
> possible?
>

Yes.

> For the 2nd example, how long are these keepalives cached/preserved?

As long as it can. Usually up to the remote keepalive timeout settings
and keepalive max requests settings.

> Also, just to make sure I fully understand what's going on (having read
> https://www.nginx.com/blog/load-balancing-with-nginx-plus-part2/#keepalive),
> let's pretend for a second the above keepalive limit was 5 and there are 7
> simultaneous requests coming in . Here is what I currently believe happens:
> #req randomly chosen IP keepalive pool id
> #1 93.184.216.34 --, but gets cached afterwards with id 1
> #2 93.184.216.35 --, but gets cached afterwards with id 2
> #3 93.184.216.36 --, but gets cached afterwards with id 3
> #4 93.184.216.34 --, but gets cached afterwards with id 4
> #5 93.184.216.35 --, but gets cached afterwards with id 5
> #6 93.184.216.36 --, connection gets closed afterwards
> #7 93.184.216.34 --, connection gets closed afterwards
>
> Then, after all of those requests are done, another set of 7 simultaneous
> requests come in. Again, here is what I currently believe happens:
> #req randomly chosen IP keepalive pool id
> #1 93.184.216.34 Reuses id 1 (or 4)
> #2 93.184.216.35 Reuses id 2 (or 5)
> #3 93.184.216.36 Reuses id 3
> #4 93.184.216.34 Reuses id 4 (or 1)
> #5 93.184.216.35 Reuses id 5 (or 2)
> #6 93.184.216.36 --, connection gets closed afterwards
> #7 93.184.216.34 --, connection gets closed afterwards
>
> Is that about right?

Yeah, something like that.

Regards,
-agentzh