Any recommendations for better dns resolved balancer_by_lua?

adel · 2017-03-21T04:54:10+00:00

How do I get the list of active clients connections/sockets from the pool?On Monday, 20 March 2017 13:43:21 UTC-4, RJoshi wrote:Hello, I am using&#xA0...

Any recommendations for better dns resolved balancer_by_lua?

adel

Hello Team,

Would be great to have your thoughts on load balancing on the openresty as a proxy dns load-balance internal servers. Below is what I have done so far:

[1]. I have collected all available servers by resolving an internal dns. Below code sets ngx.ctr.upstream as below table:

{
  ["192.168.1.75"] = 80,
  ["192.168.129.178"] = 80,
  ["192.168.129.182"] = 80
}

location / {

rewrite_by_lua_block {

local resolver = require "resty.dns.resolver"

local r, err = resolver:new {

nameservers = {{"8.8.8.8", 53}},

retrans = 3, -- 3 retransmissions on receive timeout

timeout = 1000, -- 1 sec

}

local domain = ngx.var.prefix .. ngx.var.domain .. "." .. os.getenv("CNS") -- xx.com

if not r then

ngx.log(ngx.ERR, "failed to instantiate the resolver: ", err)

ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)

end

-- dns query

local answers, err = r:query(domain)

if not answers then

ngx.log(ngx.ERR, "failed to query the DNS server: ", err)

ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)

end

if answers.errcode then

ngx.log(ngx.ERR, "dns server returned error code: ", answers.errcode, ": ", answers.errstr)

ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)

end

local http = require "resty.http"

local httpc = http.new()

local hosts = {}

-- default to http first

local scheme = 'http'

local upstream_port = 80

if ngx.var.scheme == 'https' then

scheme = 'https'

upstream_port = 443

end

function try(port)

_hosts = {}

httpc:set_timeout(300)

for i, _answer in ipairs(answers) do

local ok, err = httpc:connect(_answer["address"], port)

if ok then

_hosts[_answer["address"]] = port

end

-- If hosts are empty, it should try again to connect to other ports

if next(_hosts) == nil then

if port == 443 then

_hosts = try(80)

if next(_hosts) ~= nil then

scheme = 'http'

end

elseif port == 80 then

_hosts = try(443)

if next(_hosts) ~= nil then

scheme = 'https'

end

return _hosts

end

hosts = try(upstream_port)

if next(hosts) == nil then

local inspect = require "inspect"

ngx.status = 404

ngx.say("Cannot connect to following servers on port 80 or 443")

ngx.say(inspect(scheme))

ngx.say(inspect(answers))

ngx.exit(404)

end

-- set hosts & scheme for proxy_pass & upsteam load_balanced_by_lua

ngx.var.scheme_custom = scheme

ngx.ctx.hosts = hosts

}

proxy_pass $scheme_custom://internal_upsteam;

#error_page 502 500 = @error;

}

[2]. I can get all available servers from upstream. what would be ideal load balancing method? round-robin? Maybe test response time somewhere in review_by_lua_block and rearrange them? does balancer_by_lua_block set all of the upstream here or only set the first or last ip as upstream. ? How do I know what is current upstream without setting custom headers in the servers?

upstream internal_upsteam {

server 0.0.0.1;

balancer_by_lua_block {

local balancer = require "ngx.balancer"

local hosts = ngx.ctx.hosts

for h,p in pairs(hosts) do

balancer.set_current_peer(h, p)

end

}

keepalive 10; # connection pool

}

Kind Regards,

[+]Adel Abula

[-] Founder | Solution Architect

[-] PH:+64-09-9722636 | MOB:+64-212888791
[-] Email: Adel@magebinary.com | Web: www.MageBinary.co.nz
[-] 11/145 Kitchener Rd, Milford, Auckland 0620 |

LinkedIn | Facebook | Twitter

IMPORTANT NOTICE: The information contained in this file is confidential and may also be LEGALLY PRIVILEGED, intended only for the addressee. If you are not the addressee, any use or dissemination of the information and/or copying of this file is strictly prohibited. If you are not the addressee, please immediately notify us.CONDITIONS: MageBinary accepts no liability or responsibility whatsoever for any loss or damage suffered by the recipient arising out of, or in connection with, the use or misuse of this data. All reasonable means have been used by MageBinary to ensure that the transfer medium and its contents are free of computer viruses. The copyright of the original documentation belongs to MageBinary.

thibaultcha

Hi,

You might want to checkout lua-resty-dns-client, a library we built at Mashape to provide out of the box load balancing capabilities for balancer_by_lua from DNS records resolution (A, SRV). It currently only supports round-robin and weighted round-robin load balancing strategies, with plans for more complex ones in the near future.

https://github.com/Mashape/lua-resty-dns-client

You might be interested by the 'src/resty/dns/balancer.lua' module in this library.

As for an example usage of this library, you might want to checkout Kong, and this module in particular:

https://github.com/Mashape/kong/blob/master/kong/core/balancer.lua

Best,
Thibault

On 3/20/17 1:53 PM, Adel Abula wrote:

Hello Team,

Would be great to have your thoughts on load balancing on the openresty as a proxy dns load-balance internal servers. Below is what I have done so far:

[1]. I have collected all available servers by resolving an internal dns. Below code sets ngx.ctr.upstream as below table:
{
  ["192.168.1.75"] = 80,
  ["192.168.129.178"] = 80,
  ["192.168.129.182"] = 80
}
location / {

rewrite_by_lua_block {

local resolver = require "resty.dns.resolver"

local r, err = resolver:new {

nameservers = {{"8.8.8.8", 53}},

retrans = 3, -- 3 retransmissions on receive timeout

timeout = 1000, -- 1 sec

}

local domain = ngx.var.prefix .. ngx.var.domain .. "." .. os.getenv("CNS") -- xx.com

if not r then

ngx.log(ngx.ERR, "failed to instantiate the resolver: ", err)

ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)

end

-- dns query

local answers, err = r:query(domain)

if not answers then

ngx.log(ngx.ERR, "failed to query the DNS server: ", err)

ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)

end

if answers.errcode then

ngx.log(ngx.ERR, "dns server returned error code: ", answers.errcode, ": ", answers.errstr)

ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)

end

local http = require "resty.http"

local httpc = http.new()

local hosts = {}

-- default to http first

local scheme = 'http'

local upstream_port = 80

if ngx.var.scheme == 'https' then

scheme = 'https'

upstream_port = 443

end

function try(port)

_hosts = {}

httpc:set_timeout(300)

for i, _answer in ipairs(answers) do

local ok, err = httpc:connect(_answer["address"], port)

if ok then

_hosts[_answer["address"]] = port

end

end

-- If hosts are empty, it should try again to connect to other ports

if next(_hosts) == nil then

if port == 443 then

_hosts = try(80)

if next(_hosts) ~= nil then

scheme = 'http'

end

elseif port == 80 then

_hosts = try(443)

if next(_hosts) ~= nil then

scheme = 'https'

end

end

end

return _hosts

end

hosts = try(upstream_port)

if next(hosts) == nil then

local inspect = require "inspect"

ngx.status = 404

ngx.say("Cannot connect to following servers on port 80 or 443")

ngx.say(inspect(scheme))

ngx.say(inspect(answers))

ngx.exit(404)

end

-- set hosts & scheme for proxy_pass & upsteam load_balanced_by_lua

ngx.var.scheme_custom = scheme

ngx.ctx.hosts = hosts

}

proxy_pass $scheme_custom://internal_upsteam;

#error_page 502 500 = @error;

}

[2]. I can get all available servers from upstream. what would be ideal load balancing method? round-robin? Maybe test response time somewhere in review_by_lua_block and rearrange them? does balancer_by_lua_block set all of the upstream here or only set the first or last ip as upstream. ? How do I know what is current upstream without setting custom headers in the servers?

upstream internal_upsteam {

server 0.0.0.1;

balancer_by_lua_block {

local balancer = require "ngx.balancer"

local hosts = ngx.ctx.hosts

for h,p in pairs(hosts) do

balancer.set_current_peer(h, p)

end

}

keepalive 10; # connection pool

}

Kind Regards,

[+]Adel Abula

[-] Founder | Solution Architect

[-] PH:+64-09-9722636 | MOB:+64-212888791
[-] Email: Adel@magebinary.com | Web: www.MageBinary.co.nz
[-] 11/145 Kitchener Rd, Milford, Auckland 0620 |

LinkedIn | Facebook | Twitter

IMPORTANT NOTICE: The information contained in this file is confidential and may also be LEGALLY PRIVILEGED, intended only for the addressee. If you are not the addressee, any use or dissemination of the information and/or copying of this file is strictly prohibited. If you are not the addressee, please immediately notify us.CONDITIONS: MageBinary accepts no liability or responsibility whatsoever for any loss or damage suffered by the recipient arising out of, or in connection with, the use or misuse of this data. All reasonable means have been used by MageBinary to ensure that the transfer medium and its contents are free of computer viruses. The copyright of the original documentation belongs to MageBinary.

.