how to achieve nginx upstream connection pool

BijanPadhi

We are developing nginx openresty app that will have a high load upstream traffic. ex - https://upstream.com
However are spending around 100 ms every time we connect to upstream.
To leverage connection pool, I started using
http://nginx.org/en/docs/http/ngx_http_upstream_module.html with keep alive directive.
in this directive we have tried to make the server name dynamic using the balancer by lua block.
https://github.com/openresty/lua-nginx-module#balancer_by_lua_block
and using https://github.com/openresty/lua-resty-dns#tcp_query for dns resolution.

Though these things works for me, I am trying to find out if there is anyway we can achieve dns resolution on openresty + nginx opensource and still avoid the hard coupling of not entering the host dns directly in upstream block.
Because if we explicitly mention the server dns for upstream then nginx+openresty app fails to start when it cant resolve the dns.

Sample code

upstream upstream1 {
server 127.0.0.1 fail_timeout=10;

balancer_by_lua_block {
local balancer = require "ngx.balancer"
 -- use Lua to do something interesting here
 -- as a dynamic balancer
 local host, port = ngx.var.upstream:match( "([^://]+):(%d+)" )
 if(host == nil)
 then
    host = string.gsub(ngx.var.upstream, "https://", "")
 end;
 --local maps to cache host and ip
 local targetIp = upstreamSvcHostIpMap[host]
 local port = upstreamSvcHostPortsMap[host]
 if(port == nil) then port = 443 end;
 --ngx.log(ngx.ERR, "Set the current peer: ip -- ", targetIp , " port --", port)
 local ok, err = balancer.set_current_peer(targetIp, port)
 if not ok then
     ngx.log(ngx.ERR, "failed to set the current peer: ", err)
     return ngx.exit(500)
 end

}
keepalive 20;
keepalive_timeout 60s;
keepalive_time 1h;

}

dns query
local resolver = require "resty.dns.resolver"
local _M = {}

function _M.getIpFromDNS(host)
local nameServerIp = "8.8.8.8"
if(nameServer ~= nil) then nameServerIp = nameServer; end
ngx.log(ngx.ERR, "nameServerIp: ", nameServerIp)
local r, err = resolver:new{
nameservers = { nameServerIp}
}
if not r then
ngx.log(ngx.ERR, "failed to instantiate the resolver: ", err)
return
end

local ans, err = r:tcp_query(host, { qtype = r.TYPE_A })
if not ans then
    ngx.log(ngx.ERR, "failed to query: ", err)
    return
end
 for i, ans1 in ipairs(ans) do
    ngx.log(ngx.ERR, ans1.name, " ", ans1.address or ans1.cname,
            " type:", ans1.type, " class:", ans1.class,
            " ttl:", ans1.ttl)
    return ans1.address or ans1.cname
end

return nil

end

function _M.readNameServer()
local nameServerLocal = nil
local fp = assert(io.open("/etc/resolv.conf"))
for line in fp:lines() do
if line.find(line, "nameserver") then
nameServerLocal = line:match(" (.*)")
end

end
fp:close()
ngx.log(ngx.ERR, "nameServerLocal:", nameServerLocal)
return nameServerLocal

end

Any help in this regard will be appreciated.

zhuizhuhaomeng

OpenResty can resolve dns directly. Which version of openresty are you using?

BijanPadhi

zhuizhuhaomeng Thanks for responding.
Putting a server name directly creates hard dependency on startup.
if the dns cant be resolved in the upstream directive then the application fails at startup.
is there a way to avoid the startup failure while putting the server dns in upstream directive
and server dns resolution fails during startup

Robinh12

Even though these methods seem to work for me, I'm wondering if there's a way to use openresty + nginx opensource for DNS resolution without having to enter the host dns directly into the upstream block.